Where does the formula for gravitational potential come from? Why the minus sign?

We know Newton's Law of Gravitation, which states that the force F between two bodies of masses M and m respectively, separated by distance R, can be given by: F = GMm/R²where G is the gravitational constant. We're looking to get to gravitational potential, denoted by 'V', which is defined as the gravitational potential energy per unit mass of an object as its current point in space. We know that energy = work done = force x perpendicular distance. Using our first eqn, we get: energy = work done = GMm/R, & therefore the magnitude of V for an object of mass m is GM/R. However, we also know that V by definition is the work done in moving our object from 'infinity' (the hypothetical point at which gravity = 0) to its current position in space. Also by definition, we know that the force of gravity is purely attractive, therefore rather than expending energy by moving our object from infinity to its current position, we release energy. Think of 2 magnets, with opposing poles facing each other. If you're holding one magnet away from the other, it takes effort to keep them separated: if you were to let go of one it would move towards the other by itself, i.e. the work done in moving the two magnets towards each other is negative because they pull each other. The same is true for gravitational force, negative work is done in moving our object away from infinity because it's attracted towards other masses. Applying this theory to our eqn derived above, we get: V = -GM/R, as desired.