Linear Algebra and Calculus

Eigenvalues, determinant and trace

For a square matrix A of order n with eigenvalues \lambda_1,\dots,\lambda_n counted with multiplicity, two identities always hold: the product of the eigenvalues equals the determinant, \prod_i \lambda_i=\det A, and the sum of the eigenvalues equals the trace, \sum_i\lambda_i=\operatorname{tr}A, where the trace is the sum of the principal diagonal entries. Together these let you find a ratio such as \det A/\operatorname{tr}A without computing each eigenvalue.

For a 2\times 2 matrix the characteristic equation is \lambda^2-(\operatorname{tr}A)\lambda+\det A=0, so the trace and determinant alone determine the eigenvalues. For example, a matrix with trace 4 and determinant 3 has roots \lambda=1 and \lambda=3.

The eigenvalues of a triangular matrix, upper or lower, are its diagonal entries, because \det(A-\lambda I) is the product of the diagonal factors. If a matrix is written as J-I with J the all-ones matrix, its eigenvalues are those of J shifted by -1; since J has rank 1 and trace 3, its eigenvalues are 3,0,0, so J-I has eigenvalues 2,-1,-1.

A matrix is singular exactly when \det A=0. A homogeneous system A\mathbf{x}=\mathbf{0} has a non-trivial solution exactly in this case, which is how an unknown parameter in the coefficient matrix is found.

• trace — the sum of the diagonal entries; equals the sum of eigenvalues. e.g. \operatorname{tr}\begin{bmatrix}2&1\\0&5\end{bmatrix}=7
• singular — having zero determinant; at least one eigenvalue is zero.

If every row of A sums to the same value \lambda, then A(1,1)^T=\lambda(1,1)^T and (1,1)^T is an eigenvector. When the row sums of a 2\times 2 matrix are equal, the second eigenvalue then follows from the trace. Eigenvalues of A and A^T are identical, but 1/\lambda is an eigenvalue of A^{-1}, not in general of A. Eigenvectors belonging to distinct eigenvalues are linearly independent, so n distinct eigenvalues give n independent eigenvectors.

Divergence and curl of vector fields

The divergence of a vector field \vec V=u\hat i+v\hat j+w\hat k is the scalar \nabla\cdot\vec V=\dfrac{\partial u}{\partial x}+\dfrac{\partial v}{\partial y}+\dfrac{\partial w}{\partial z}. A field is called divergence-free (solenoidal) when this is zero everywhere. For the planar field x\hat i+y\hat j the divergence is 2, not zero.

The curl is the vector \nabla\times\vec V. For a planar field, only the z-component \partial v/\partial x-\partial u/\partial y can be nonzero. A field with zero curl is irrotational (curl-free). A field can be curl-free but not divergence-free, and the two properties must be checked separately.

Vorticity is defined as \vec\omega=\nabla\times\vec V, and enstrophy is the square of its magnitude, |\vec\omega|^2. To evaluate it at a point, first compute each component of the curl from the partial derivatives and then substitute the coordinates.

The Gauss divergence theorem converts the outward flux through a closed surface into a volume integral, \oint \vec v\cdot\hat n\,dS=\iiint \nabla\cdot\vec v\,dV. It is the standard shortcut when the divergence is simple.

If the curl of a field vanishes on a simply connected region, the field is conservative, \vec F=\nabla\phi. Then the line integral around any closed contour is zero, because it depends only on the endpoints of the path.

• irrotational — having zero curl.
• solenoidal — having zero divergence.
• circulation — the closed line integral \oint \vec F\cdot d\vec l.

Continuity and differentiability of functions

A function is continuous at a point when its left limit, right limit and function value all coincide there. Compositions and products of continuous functions are continuous, so \sin(|x|) and e^{-x}|\cos x| are continuous wherever their building blocks are.

The absolute value |g(x)| is continuous wherever g is, but it creates a corner at each zero of g, where the function may fail to be differentiable. Where g\neq 0, the function |g(x)| equals \pm g(x) locally and is differentiable.

A piecewise function can be continuous but have a derivative that jumps at the joining point. Compare the one-sided derivatives: for f(x)=x^2 on x<0 and f(x)=x on x\geq 0, the left derivative at 0 is 0 while the right derivative is 1, so the derivative is discontinuous at 0 even though f is continuous.

To make a piecewise function continuous at a joining point, equate the left-hand limit with the function value there. If f(x)=A+x for x<2 and f(x)=1+x^2 for x\geq 2, then A+2=5 gives A=3.

If a continuous, monotonic function has a single root inside an interval, the function changes sign across that root. Therefore the endpoint values have opposite signs, and their product is negative.

• continuous at a point — the limit from both sides equals the value at that point.
• corner (kink) — a point where the one-sided derivatives differ; the function is continuous there but not differentiable.

Extrema of function on closed interval

For a quadratic y=ax^2+bx+c with a>0 the parabola opens upward, so its only stationary point dy/dx=2ax+b=0 is a minimum. For y=(x+3)(x-2)=x^2+x-6, the derivative 2x+1 vanishes at x=-1/2, which is the minimum.

To find the maximum or minimum of a differentiable function on a closed interval [a,b], solve f'(x)=0 for the critical points inside the interval, then compare f at each critical point with the values at the endpoints. The largest value gives the maximum and the smallest gives the minimum.

A function built from absolute values such as |x|+|2x+3| is convex and piecewise linear, with breakpoints where each absolute-value term changes sign. Such a function attains its minimum at one of its breakpoints. Evaluating it there gives the minimum value.

• critical point — a point in the domain where f'(x)=0 or f' does not exist.
• breakpoint — a point where the expression inside an absolute value is zero.

Rank and consistency of linear systems

A homogeneous system A\mathbf{x}=\mathbf{0} is always consistent, because the trivial solution \mathbf{x}=\mathbf{0} always satisfies it. It can therefore never have no solution.

If the number of unknowns is n and the rank r of the coefficient matrix satisfies r<n, the homogeneous system has n-r free parameters and thus infinitely many solutions. Equivalently, the coefficient matrix has zero determinant. A unique solution (only the trivial one) requires r=n, a nonsingular matrix.

An outer product A=uv^T of two non-zero vectors has every column proportional to u, so its column space is spanned by one vector. Hence the rank of A is 1.

• rank — the number of linearly independent rows or columns of a matrix.
• consistent system — a system with at least one solution.
• free parameter — an unknown that can take any value, giving a family of solutions.

Limit evaluation

When a limit has the form 0/0 or a difference of singular terms, expand the functions in a series or use standard limits. For instance, \dfrac{1}{\sin x}-\dfrac{1}{x}=\dfrac{x-\sin x}{x\sin x}, and with \sin x = x-x^3/6+\cdots the expression tends to 0 as x\to 0.

The identity 1-\cos 2x = 2\sin^2 x converts a cosine difference into a square of the standard form. Since \lim_{x\to 0}\sin x/x=1, the limit \lim_{x\to 0}(1-\cos 2x)/x^2 equals 2.

• standard limit — a known result such as \lim_{x\to 0}\sin x/x=1 used as a building block.
• indeterminate form — a form such as 0/0 that requires further manipulation.