How do you differentiate by first principles?

Firstly, what are we trying to achieve by differentiating a function? Differentiation allows us to compute the rate of change of a function at any point along a well behaved function.

Consider the curve y = f(x)

Imagine moving along the curve to a point with coordinates (x₁,y₁) and then moving to a new point (x₂,y₂). What is the gradient between these points? Trivially, it is y₂-y₁/x₂-x₁

What happens if we draw a line between those two points? Assuming the points are a suitable distance apart and the curve is suitably curvy and well behaved, this line is unlikely to follow the curve, nor be tangent to the curve. Let's think about our definition of differentiation again, we want to describe/compute the rate of change (or gradient) of the function at a point. This is the tangent to the function. Why is it the tangent though? Since we require the rate of change at a point, we need the gradient at a point. But surely that is contradictory? In fact yes, there cannot be a gradient at a true point, but... There is a gradient between a point and a point infinitessimally nearby. Now we are getting somewhere. So in order to differentiate we must choose the coordinates (x₂,y₂) such that:

x₂ = x₁ + dx where dx is infinitessimally small. 

y₂ = f(x₂) = f(x₁ + dx)

Then trivially the gradient is simply:

( (f(x₁+dx)-f(x₁) ) / (x₁+dx-x₁) = ( f(x + dx) - f(x) ) / dx

(note: for clarity I have rewritten x₁ as x) 

For example lets plug in a simply function like f(x) = x²

The numerator then looks like:

x² + 2x*dx + dx² - x²

The divisor:

dx

The whole fraction is then just:

2x + dx.

Since dx is infinitessimal we recover the expected result of 2x using the 'bring down power rule'

Try to convince yourself the power rule holds for all xⁿ where n is an integer using binomial expansion and taking suitable approximations (hint: dxⁿ is always negligable!