Calculus and Optimization

Critical points and monotonicity

A critical point of a differentiable function f is an interior point x_0 of its domain where f'(x_0)=0 or where f' fails to exist. Between consecutive critical points, f' keeps a constant sign, so f is increasing where f'>0 and decreasing where f'<0. A local extremum can occur only at a critical point. The sign change of f' across that point decides the type: a change from + to - gives a local maximum, and a change from - to + gives a local minimum. To locate the critical points, factor f' and test one point in each interval. For example, if g(x)=x^3-3x, then g'(x)=3(x-1)(x+1), so g decreases on (-1,1) and increases elsewhere. Thus x=-1 is a local maximum and x=1 is a local minimum. On a closed interval, the global maximum and minimum are found by comparing the values at interior critical points and at the endpoints. A value at an endpoint that is excluded from the domain is never attained. The maximum or minimum need not be unique, since two different points can share the same extreme value.

Second derivative test

If f'(x^*)=0 and f''(x^*)>0, then x^* is a local minimum. If f'(x^*)=0 and f''(x^*)<0, then x^* is a local maximum. The test is local, because it describes the behaviour of f near x^* and says nothing about values far away, so it cannot by itself establish a global minimum or maximum. When f''(x^*)=0, the test is inconclusive and one must examine the sign of f' or higher derivatives. For instance, h(x)=x^4 has h'(0)=h''(0)=0, yet 0 is a global minimum. For k(x)=-(x-2)^2, we have k'(2)=0 and k''(2)=-2<0, so x=2 is a local maximum.

Sigmoid function derivative

The logistic (sigmoid) function \sigma(x)=\dfrac{1}{1+e^{-x}} satisfies the self-referential identity \sigma'(x)=\sigma(x)\bigl(1-\sigma(x)\bigr). This follows from the quotient rule, which gives \sigma'(x)=\dfrac{e^{-x}}{(1+e^{-x})^2}, together with 1-\sigma(x)=\dfrac{e^{-x}}{1+e^{-x}}. The derivative can therefore be computed from the function value alone, without knowing x explicitly. The derivative is largest when \sigma(x)=\tfrac12, where it equals \tfrac14, and it tends to 0 as \sigma(x) approaches 0 or 1. For example, if \sigma(x)=0.7, then \sigma'(x)=0.7\times0.3=0.21.

Continuity and differentiability of piecewise functions

For a piecewise function, continuity at a junction point c requires that the left limit, the right limit and the function value at c all coincide. Differentiability at c requires, in addition, that the left and right derivatives agree, which means the slopes of the adjacent pieces must match at c. Continuity is necessary for differentiability, so the value conditions should always be checked first. Each junction gives one equation from the values and one from the slopes. Consider p(x)=x for x<1 and p(x)=ax^2+bx+c for x\ge1. Continuity at x=1 gives a+b+c=1, and differentiability gives 2a+b=1, since the left slope is 1.

Limits by series expansion

Small-argument Taylor expansions convert limits of the form 0/0 into comparisons of leading powers. The standard expansions are \ln(1+u)=u-\dfrac{u^2}{2}+\cdots, \cos x=1-\dfrac{x^2}{2}+\cdots, e^x=1+x+\dfrac{x^2}{2}+\cdots and \sin x=x-\dfrac{x^3}{6}+\cdots. The logarithm of a product splits into a sum of logarithms, so each factor can be expanded separately. The expansion must be carried to the order that survives in the denominator, and higher-order terms of the form O(x^4) can be ignored. For example, \lim_{x\to0}\dfrac{e^x-1-x}{x^2}=\dfrac12, because the numerator equals \dfrac{x^2}{2}+O(x^3).

Taylor series derivatives at zero

If a function has the Maclaurin series \sum_k a_k x^k, then its n-th derivative at 0 equals n!\,a_n. The problem of finding f^{(n)}(0) therefore reduces to finding the coefficient of x^n. For \sinh x=\dfrac{e^x-e^{-x}}{2}, the series contains only odd powers, so f^{(n)}(0)=0 for every even n. The odd-order derivatives satisfy \sinh^{(2k+1)}(0)=\cosh 0=1. Similarly, the derivatives of \cos x vanish at 0 for all odd orders, and the even-order ones equal (-1)^{n/2}. Remember that n! multiplies the coefficient and that the factorial in the answer does not appear unless the coefficient itself carries it.