Physlib

PhyslibAlpha.Mathematics.PartialDerivativeTest

The Second Partial Derivatives Test

We prove a version of the second partial derivative test from calculus for analytic functions `f : V → ℝ`, where `V` is a finite-dimensional vector space.

Main results

* `second_derivative_test`: Suppose `f` is a real-valued function on a finite-dimensional inner product space that has vanishing gradient at `x₀`, and has a power series on a ball of positive radius around `x₀`. If the second Frechét derivative is positive definite at `x₀` then `f` has local minimum at `x₀`.

Tags

24 declarations

theorem

Updating index 00 of a vector of length 2 results in (c,b)(c, b)

For any type α\alpha and elements a,b,cαa, b, c \in \alpha, updating the first component (index 00) of the vector (a,b)(a, b) with the value cc results in the vector (c,b)(c, b).

theorem

Updating the second entry of a 2-vector yields (a,c)(a, c)

Let a,ba, b, and cc be elements of a type α\alpha. Updating the second component (index 1) of the vector (a,b)(a, b) with the value cc results in the vector (a,c)(a, c).

definition

Hessian bilinear companion of ff at x0x_0

Let VV be a real normed vector space and f:VRf: V \to \mathbb{R} be a function. For a point x0Vx_0 \in V, the **Hessian bilinear companion** is the bilinear map B:V×VRB: V \times V \to \mathbb{R} defined by B(u,v)=D2f(x0)(u,v)+D2f(x0)(v,u) B(u, v) = D^2 f(x_0)(u, v) + D^2 f(x_0)(v, u) where D2f(x0)D^2 f(x_0) is the second Fréchet derivative of ff at x0x_0 viewed as a multilinear map on V2V^2.

definition

Second Fréchet derivative as a quadratic map yD2f(x0)(y,y)y \mapsto D^2 f(x_0)(y, y)

Let VV be a real normed vector space. For a function f:VRf: V \to \mathbb{R} and a point x0Vx_0 \in V, this definition constructs the quadratic map Q:VRQ: V \to \mathbb{R} such that for any vector yVy \in V, Q(y)=D2f(x0)(y,y)Q(y) = D^2 f(x_0)(y, y), where D2f(x0)D^2 f(x_0) denotes the second iterated Fréchet derivative of ff at x0x_0.

definition

Continuous bilinear map of a continuous multilinear map on V2V^2

Let k\mathbb{k} be a complete nontrivially normed field and VV be a finite-dimensional normed vector space over k\mathbb{k}. This definition converts a continuous multilinear map g:V×Vkg : V \times V \to \mathbb{k} into its corresponding continuous bilinear map B:VL(V,k)B: V \to \mathcal{L}(V, \mathbb{k}), defined such that for any x,yVx, y \in V, B(x)(y)=g(x,y)B(x)(y) = g(x, y).

definition

Multilinear map of a quadratic map QQ

Let VV be a real vector space. This definition associates a quadratic map Q:VRQ : V \to \mathbb{R} with its corresponding multilinear map B:V×VRB : V \times V \to \mathbb{R} (where the domain is represented as the space of functions from {0,1}\{0, 1\} to VV). The resulting map is defined such that for any v0,v1Vv_0, v_1 \in V, B(v0,v1)B(v_0, v_1) is equal to the polar bilinear form of QQ evaluated at v0v_0 and v1v_1.

definition

Half-polar bilinear form of a quadratic map QQ as a multilinear map

Let VV be a vector space over R\mathbb{R}. For a quadratic map Q:VRQ: V \to \mathbb{R}, this definition constructs the associated multilinear map M:V×VRM: V \times V \to \mathbb{R} corresponding to half of the polar bilinear form of QQ. Specifically, for any v0,v1Vv_0, v_1 \in V, the map is defined as M(v0,v1)=12(Q(v0+v1)Q(v0)Q(v1))M(v_0, v_1) = \frac{1}{2} (Q(v_0 + v_1) - Q(v_0) - Q(v_1)).

theorem

The multilinear map associated with a quadratic map is continuous

Let VV be a finite-dimensional real normed vector space. For any quadratic map Q:VRQ : V \to \mathbb{R}, its associated multilinear map B:V×VRB : V \times V \to \mathbb{R} (the polar bilinear form) is continuous.

theorem

The half-polar bilinear form of a quadratic map QQ is continuous

Let VV be a finite-dimensional real normed vector space and Q:VRQ: V \to \mathbb{R} be a quadratic map. Then the associated half-polar bilinear form B:V×VRB: V \times V \to \mathbb{R}, defined by B(v0,v1)=12(Q(v0+v1)Q(v0)Q(v1)),B(v_0, v_1) = \frac{1}{2} (Q(v_0 + v_1) - Q(v_0) - Q(v_1)), is continuous.

definition

Continuous polar bilinear form of a quadratic map QQ

Let VV be a finite-dimensional real normed vector space. For any quadratic map Q:VRQ: V \to \mathbb{R}, this definition constructs the corresponding continuous multilinear map B:V×VRB: V \times V \to \mathbb{R} (represented as a map from the index set {0,1}\{0, 1\} to VV). The resulting map is defined by the polar bilinear form of QQ, such that for any x,yVx, y \in V, B(x,y)=Q(x+y)Q(x)Q(y)B(x, y) = Q(x + y) - Q(x) - Q(y).

definition

Continuous half-polar bilinear form of a quadratic map QQ

Let VV be a finite-dimensional real normed vector space and let Q:VRQ: V \to \mathbb{R} be a quadratic map. This definition constructs the continuous bilinear map B:V×VRB: V \times V \to \mathbb{R} corresponding to one-half of the polar bilinear form associated with QQ. For any vectors v0,v1Vv_0, v_1 \in V, the map is defined as: B(v0,v1)=12(Q(v0+v1)Q(v0)Q(v1))B(v_0, v_1) = \frac{1}{2} (Q(v_0 + v_1) - Q(v_0) - Q(v_1)) The continuity of this map is guaranteed by the finite-dimensionality of VV.

theorem

Evaluation of the continuous multilinear map of QQ is its polar bilinear form

Let VV be a finite-dimensional real normed vector space and Q:VRQ : V \to \mathbb{R} be a quadratic map. For any x,yVx, y \in V, the continuous multilinear map associated with QQ (constructed via the polar identity) evaluated at the pair (x,y)(x, y) is equal to the polar bilinear form of QQ, defined as B(x,y)=Q(x+y)Q(x)Q(y)B(x, y) = Q(x + y) - Q(x) - Q(y).

theorem

The half-polar bilinear map evaluated at (x,y)(x, y) equals 12B(x,y)\frac{1}{2} B(x, y)

Let VV be a finite-dimensional real normed vector space and Q:VRQ: V \to \mathbb{R} be a quadratic map. Let B1/2:V×VRB_{1/2} : V \times V \to \mathbb{R} be the continuous bilinear map constructed as one-half of the polar bilinear form associated with QQ. For any vectors x,yVx, y \in V, the value of B1/2B_{1/2} applied to the pair (x,y)(x, y) is equal to 12B(x,y)\frac{1}{2} B(x, y), where BB is the polar bilinear form of QQ.

theorem

A positive definite quadratic map on a finite-dimensional space is coercive

Let VV be a finite-dimensional real normed vector space. Suppose F:VRF: V \to \mathbb{R} is a positive definite quadratic map. Then the associated continuous bilinear form B:V×VRB: V \times V \to \mathbb{R}, defined as one-half of the polar form of FF (such that B(v,v)=F(v)B(v, v) = F(v) for all vVv \in V), is coercive. That is, there exists a constant c>0c > 0 such that B(v,v)cv2B(v, v) \geq c \|v\|^2 for all vVv \in V.

theorem

The polar bilinear form of yD2f(x0)(y,y)y \mapsto D^2 f(x_0)(y, y) equals D2f(x0)(x,y)+D2f(x0)(y,x)D^2 f(x_0)(x, y) + D^2 f(x_0)(y, x)

Let VV be a real normed vector space and f:VRf : V \to \mathbb{R} be a function. For a point x0Vx_0 \in V, let QQ be the quadratic map associated with the second Fréchet derivative at x0x_0, defined by Q(y)=D2f(x0)(y,y)Q(y) = D^2 f(x_0)(y, y). Then the polar bilinear form BB of this quadratic map satisfies B(x,y)=D2f(x0)(x,y)+D2f(x0)(y,x)B(x, y) = D^2 f(x_0)(x, y) + D^2 f(x_0)(y, x) for all x,yVx, y \in V. This identity expresses the polar bilinear form of the second-derivative quadratic map as the symmetrized Hessian.

theorem

Symmetry of the second Fréchet derivative (Schwarz's theorem)

Let VV be a real normed space and f:VRf: V \to \mathbb{R} be a function. If ff is CC^\infty (infinitely differentiable) at a point x0Vx_0 \in V, then the second iterated Fréchet derivative D2f(x0)D^2 f(x_0) is symmetric. That is, for any vectors x,yVx, y \in V, the following equality holds: D2f(x0)(y,x)=D2f(x0)(x,y)D^2 f(x_0)(y, x) = D^2 f(x_0)(x, y) This result is known as Schwarz's theorem (or Clairaut's theorem) regarding the symmetry of second derivatives.

theorem

Coercivity of D2f(x0)D^2 f(x_0) from Positive Definiteness and Differentiability

Let VV be a finite-dimensional real normed vector space and f:VRf: V \to \mathbb{R} be a function. Suppose ff is infinitely differentiable (CC^\infty) at a point x0Vx_0 \in V. If the quadratic form associated with the second Fréchet derivative of ff at x0x_0, defined by yD2f(x0)(y,y)y \mapsto D^2 f(x_0)(y, y), is positive definite, then the bilinear map D2f(x0)D^2 f(x_0) is coercive. That is, there exists a constant c>0c > 0 such that for all yVy \in V: D2f(x0)(y,y)cy2D^2 f(x_0)(y, y) \geq c \|y\|^2

theorem

Positive Definiteness of D2f(x0)D^2 f(x_0) Implies Coercivity

Let VV be a finite-dimensional real normed vector space and f:VRf: V \to \mathbb{R} be a function. If the second Fréchet derivative of ff at x0Vx_0 \in V is positive definite—meaning the quadratic map Q(y)=D2f(x0)(y,y)Q(y) = D^2 f(x_0)(y, y) satisfies Q(y)>0Q(y) > 0 for all nonzero yVy \in V—then the corresponding bilinear form B(x,y)=D2f(x0)(x,y)B(x, y) = D^2 f(x_0)(x, y) is coercive. (A bilinear form BB is coercive if there exists a constant c>0c > 0 such that B(y,y)cy2B(y, y) \geq c \|y\|^2 for all yVy \in V).

theorem

f(x0)f(x)f(x_0) \leq f(x) via Second-Order Taylor Remainder Bound

Let VV be a real inner product space and f:VRf: V \to \mathbb{R} be a function. For any points x0,xVx_0, x \in V and a constant CRC \in \mathbb{R}, suppose the following conditions are satisfied: 1. The second Frechét derivative of ff at x0x_0 satisfies the inequality Cxx02D2f(x0)(xx0,xx0)C \|x - x_0\|^2 \leq D^2 f(x_0)(x - x_0, x - x_0). 2. The first Frechét derivative of ff at x0x_0 satisfies Df(x0)(x)=Df(x0)(x0)Df(x_0)(x) = Df(x_0)(x_0) (which, by linearity, implies Df(x0)(xx0)=0Df(x_0)(x - x_0) = 0). 3. The error of the second-order Taylor expansion is bounded by f(x)i=021i!Dif(x0)(xx0,,xx0)C2xx02,\left\| f(x) - \sum_{i=0}^2 \frac{1}{i!} D^i f(x_0)(x - x_0, \dots, x - x_0) \right\| \leq \frac{C}{2} \|x - x_0\|^2, where Dif(x0)D^i f(x_0) denotes the ii-th Frechét derivative of ff at x0x_0. Then, f(x0)f(x)f(x_0) \leq f(x).

theorem

A Positive Definite Hessian and o(xx02)o(\|x-x_0\|^2) Remainder Imply a Local Minimum

Let VV be a finite-dimensional real inner product space and f:VRf: V \to \mathbb{R} be a function that is CC^\infty at x0Vx_0 \in V. Suppose the following conditions hold: 1. The gradient of ff vanishes at x0x_0, i.e., f(x0)=0\nabla f(x_0) = 0. 2. The function ff satisfies a second-order Taylor expansion condition at x0x_0: f(x)i=021i!Dif(x0)(xx0,,xx0)=o(xx02)f(x) - \sum_{i=0}^2 \frac{1}{i!} D^i f(x_0)(x - x_0, \dots, x - x_0) = o(\|x - x_0\|^2) as xx0x \to x_0, where Dif(x0)D^i f(x_0) is the ii-th Fréchet derivative of ff at x0x_0. 3. The second Fréchet derivative at x0x_0 is positive definite, i.e., the quadratic map vD2f(x0)(v,v)v \mapsto D^2 f(x_0)(v, v) is positive definite. Then ff has a local minimum at x0x_0.

theorem

Power Series Representation Implies o(xx02)o(\|x - x_0\|^2) Quadratic Approximation Remainder

Let VV be a normed real vector space and f:VRf: V \to \mathbb{R} be a function. Suppose ff has a formal power series representation pp on a ball of radius rr centered at x0Vx_0 \in V. Then the difference between the function f(x)f(x) and its partial sum of degree 2 (the quadratic approximation) is little-o of the square of the distance from x0x_0. That is, f(x)i=02pi(xx0,,xx0)=o(xx02)f(x) - \sum_{i=0}^2 p_i(x - x_0, \dots, x - x_0) = o(\|x - x_0\|^2) as xx0x \to x_0, where pip_i denotes the ii-th term of the formal multilinear series pp.

theorem

Every point in a subsingleton space is a local minimum.

Let VV be a topological space that is a subsingleton (meaning it contains at most one element). For any function f:VRf : V \to \mathbb{R} and any point x0Vx_0 \in V, ff has a local minimum at x0x_0.

theorem

Second Derivative Test for Local Minimum

Let VV be a finite-dimensional real inner product space and f:VRf: V \to \mathbb{R} be a function. Suppose x0Vx_0 \in V is a point such that the gradient of ff at x0x_0 vanishes, f(x0)=0\nabla f(x_0) = 0. If ff has a formal power series representation on a ball of radius r>0r > 0 centered at x0x_0, and the second Fréchet derivative at x0x_0 (represented by the quadratic map yD2f(x0)(y,y)y \mapsto D^2 f(x_0)(y, y)) is positive definite, then ff has a local minimum at x0x_0.

theorem

Second Derivative Test for Local Minimum of Analytic Functions

Let VV be a finite-dimensional real inner product space. Suppose f:VRf: V \to \mathbb{R} is a function that is analytic at a point x0Vx_0 \in V. If the gradient of ff vanishes at x0x_0 (i.e., f(x0)=0\nabla f(x_0) = 0) and the second Fréchet derivative of ff at x0x_0 is positive definite as a quadratic map (meaning D2f(x0)(y,y)>0D^2 f(x_0)(y, y) > 0 for all non-zero yVy \in V), then ff has a local minimum at x0x_0.