Linear Algebra, Calculus and Numerical Methods

Limits of functions

A limit as x\to a describes the value that a function approaches near a, and it may be finite, infinite, or nonexistent. Two standard patterns matter for exam problems. First, when the expression has the form \frac{g(x)-g(a)}{x-a}, the limit is by definition g'(a), so it can be evaluated by differentiation. Second, for trigonometric ratios as x\to 0, one uses \lim_{x\to 0}\frac{\sin ax}{x}=a and \lim_{x\to 0}\frac{\tan ax}{x}=a, so that \lim_{x\to 0}\frac{\sin ax}{\sin bx}=\frac{a}{b} for b\neq 0. Since \tan(kx)\approx kx near zero, the limit of a ratio of two tangents is governed by the ratio of their coefficients. For example, \lim_{x\to 0}\frac{\sin 2x}{\sin 7x}=\frac{2}{7}. Multiplying a limit by a constant multiplies the answer by that same constant, so \lim_{x\to 0}\frac{n\sin 5x}{\sin 3x}=\frac{5n}{3}.

Positive definite matrices

A real symmetric matrix A is positive definite if \mathbf{x}^{T}A\mathbf{x}>0 for every nonzero vector \mathbf{x}. For a symmetric matrix this is equivalent to every eigenvalue being strictly positive. Its determinant, being the product of the eigenvalues, is then positive, so the matrix is nonsingular. A zero eigenvalue instead signals a singular matrix that is only positive semidefinite. The opposite case is negative definite, where \mathbf{x}^{T}A\mathbf{x}<0 for every nonzero \mathbf{x} and every eigenvalue is negative. Each diagonal entry a_{ii} is then negative, so the trace is negative. For an n\times n negative definite matrix the determinant has sign (-1)^n, so its sign depends on the order n and is not fixed as negative for every order. For example, \mathrm{diag}(2,5) is positive definite, while \mathrm{diag}(-1,-3) is negative definite.

Simpson's 1/3 rule

Simpson's 1/3 rule approximates \int_a^b f(x)\,dx by integrating a parabola through three equally spaced points. For one panel with h=\frac{b-a}{2} it gives I\approx\frac{h}{3}\left[f_0+4f_1+f_2\right]. For n subintervals, n must be even, and the weights are 1,4,2,4,\dots,4,1 multiplied by \frac{h}{3}. With two subintervals the three nodes are the two endpoints and the midpoint, and the value of the integral is obtained by substituting the function values at these nodes. The error of the rule is proportional to h^4, and the rule integrates polynomials up to degree three exactly. The error therefore vanishes for any cubic polynomial, whatever the step length. This property gives a quick check: for a cubic integrand the Simpson estimate equals the exact integral.

Fourier series coefficients

For a function of period 2\pi, the Fourier series is f(x)=\frac{a_0}{2}+\sum_{n\ge 1}(a_n\cos nx+b_n\sin nx). The constant term \frac{a_0}{2} is the average value of f over one period, given by \frac{a_0}{2}=\frac{1}{2\pi}\int_{-\pi}^{\pi}f(x)\,dx. If the positive and negative areas over a period cancel, the constant term is zero. A function with opposite constant values on (-\pi,0) and (0,\pi), for instance, has zero mean over a symmetric period. Symmetry decides which coefficients survive. An even function satisfies f(-x)=f(x), and then all sine coefficients b_n vanish because f(x)\sin nx is odd. Only the constant term and the cosine terms remain. An odd function satisfies f(-x)=-f(x), so f(x)\cos nx is odd and every a_n, including the constant term, vanishes. Only sine terms remain. For example, f(x)=x on (-\pi,\pi) is odd, and its series contains only sine terms.

Mean value theorem bounds

The mean value theorem states that if f is continuous on [a,b] and differentiable on (a,b), then f(b)-f(a)=(b-a)f'(c) for some c\in(a,b). This gives bounds on f(b) from bounds on the derivative. If m\le f'(x)\le M on the interval, then f(a)+m(b-a)\le f(b)\le f(a)+M(b-a). The largest possible value of f(b) therefore comes from the upper bound on the derivative, and it is attained by a linear function with that slope. For example, if g(0)=2 and 0\le g'(x)\le 3 on [0,4], then 2\le g(4)\le 14.

Integration by parts

Integration by parts follows from \int u\,dv=uv-\int v\,du. A good choice of u is a function that simplifies on differentiation, such as a logarithm, while dv should be an easily integrated factor. For \int x\ln(1+x)\,dx, one takes u=\ln(1+x) and dv=x\,dx, which reduces the problem to an integral of a rational function. When the remaining rational term is improper, polynomial division is applied first. For example, \frac{x^2}{1+x}=x-1+\frac{1}{1+x}, and each term is then integrated separately. For definite integrals, the boundary term [uv]_a^b must be evaluated as well. Over [0,1] the integral \int_0^1 x\ln(1+x)\,dx equals \frac{1}{4}.

Integration of exponential rational functions

Integrals of the form \int \frac{f'(x)}{f(x)}\,dx equal \ln|f(x)|+C. The skill is recognizing when the numerator is a constant multiple of the derivative of the denominator. When the denominator is e^{x}+x^{e}, its derivative is e^{x}+e\,x^{e-1}. A numerator written as \frac{1}{e}\left(e^{x}+e\,x^{e-1}\right) therefore produces the factor \frac{1}{e} in front of the logarithm. The constant coefficient must be kept, and the argument of the logarithm must match the denominator exactly. A sign change inside the logarithm is not permitted unless the integrand actually has that form. For example, \int \frac{2x+2}{x^2+2x}\,dx=\ln|x^2+2x|+C, because the numerator is exactly the derivative of the denominator.

Continuity and differentiability of piecewise functions

An absolute value function such as |x-a| is continuous everywhere, so any finite sum of such terms is continuous. Such a function is piecewise linear, and it can fail to be differentiable only at the junction points where the expression inside an absolute value changes sign. At x=a the left slope of |x-a| is -1 and the right slope is +1, which creates a corner. A sum of k absolute value terms with distinct junction points therefore has k corners, where it is continuous but not differentiable. To test a breakpoint, first compare the values of the pieces to check continuity, and then compare the left and right derivatives to check differentiability.

Consistency of linear systems

A system of linear equations A\mathbf{x}=\mathbf{b} is consistent, meaning it has at least one solution, exactly when the rank of A equals the rank of the augmented matrix [A\,|\,\mathbf{b}]. If the common rank equals the number of unknowns, the solution is unique. If the common rank is less than the number of unknowns, infinitely many solutions exist. Geometrically, each linear equation in two unknowns is a straight line. Several lines have a common solution only if they all pass through one point. If two lines meet at a point that a third line does not pass through, the system is inconsistent. A homogeneous system always has the zero solution and is therefore never inconsistent.

Definite integral of trigonometric powers

Integrals of the type \int \cos x\,\sin^{m}x\,dx are handled by the substitution u=\sin x, since \cos x\,dx then becomes du. After the substitution the limits of integration must be converted. The new limits are the values of \sin x at the original limits, and the integrand becomes a power of u. For example, \int_0^{\pi/2}\cos x\,\sin^{2}x\,dx=\left[\frac{\sin^{3}x}{3}\right]_0^{\pi/2}=\frac{1}{3}. Failing to change the limits is the most common error in such problems.