Calculus of Variations

CDT Paul Falcone, CDT James Morgan, CDT Dominic Senteno

Introduction

It is obvious that thte shortest distance between two points is a straight line. That is why most most people, when they are traveling cross country tend to use interstates because they are reltively straight. But let's say you wanted to travel across the country using only local roads, also known as the sienic route. How do you calculate the shortest distance between two points with constraints? Is this idea even used in the "real-word"? These questions and a lot more will be answered when we talk about the Calculus of Variations.

Summery of a Direct Application of Calculus of Variations

Caclculus of Variations is a field in mathematics that deals with finding functionals that have maximums and minimums [1]. Within this field of calculus there are two basic subsets: unconstrained and constrained [3].

Unconstrained

The simiplist way to describe unconstrained is if you are tasked with going from point A to point B, and there is nothing in your way. A funcion is considered unconstrained when it does not have to satisfy any conditions, which is the reason why a strainght line is the shortest distance between two points.

Constrained

A function is considered constrained when it is trying to satisfy a certain condidtion. A "real-world" example trying to drive cross-country using only local roads. A classic example in Calculus of Variations is trying to find the minimum distance between the origin to some surface by following a given constraint, $\phi$.

Example

Find the minimum distance from the origin to the surface $x^2-z^2=1$ subject to $\phi =x^2-z^2-1=0$
First you need to solve for $z$: $z^2=x^2-1$ and then by substitution we can get

(1)
\begin{equation} f=2x^2+y^2-1 \end{equation}

Next we will show you how to solve for an unconstrained problem.
What you will do next is differentiate (1) and you will get:

(2)
\begin{equation} f_x=0=4x \end{equation}
(3)
\begin{equation} f_y=0=2y \end{equation}

This implies that $f_x=f_y=0$ at the minimizing point, which is not possible due to the fact that $z^2=-1$. This proves that there is no solution, or minimum distance, for an unconstrained function. Which we already knew because the problem statement told us that it was constrained.
Now we will solve through constrained optimization. Let $\phi =0$ and let $X_0$ be a relative minimum such that there is a constant, $\lambda$, and the function $F: \mathbb{R}\to\mathbb{R}$ that is defined as

(4)
\begin{align} F=f+\lambda\phi \end{align}

By using Lagrange Multiplers, we can say:

(5)
\begin{align} \sum_{i=1} ^n F_x_ih_i=0 \end{align}

For all vectors in $H$.
From the previous problem (1), we are able to form the function:

(6)
\begin{align} F=x^2+y^2+z^2+\lambda(x^2-z^2-1) \end{align}

We can now form a system of equations from 6 and get:

(7)
\begin{align} 0=F_x=2x+2\lamda x=2x(1+\lambda) \end{align}
(8)
\begin{equation} 0=F_y=2y \end{equation}
(9)
\begin{align} 0=f_z=2z-2\lambda z=2z(1-\lambda) \end{align}
(10)
\begin{align} \phi=x^2-z^2-1=0 \end{align}

We have to add the constraint into the system of equations because it is part of the system because it affects the outcome of our function 6. By using basic algebra, we can solve this system of equations and find that the only possible points on the hyperbola that would allow for the shortest distance to the origin are at $(1,0,0)$ and $(-1,0,0)$.

In the calculus of variations, you also seek a function $y=f(x)$ to minimize some cost or to maximize some product. This max/min will usually be expressed as an integral involving $x,y,$ and $y'$:

(11)
\begin{align} \int^{b}_{a}F(x,y,y')dx \end{align}

where $F$ is a function of three variables, which we will assume has continuous first and second partial derivatives.

Relavance to Real Analysis

Since the calculus of variations is a direct extension and continuation of single and multivariable calculus, it is not surprising that real analysis plays a critical role in the development of the central proofs and theorems found in the calculus of variations. As in regular calculus, some theorems are not very interesting in and of themselves, but they are often useful in proving other more powerful and applicable theorems. The Leibniz's Rule and the Fundamental Lemma of the Calculus of Variations are two such theorems.

Leibniz's Rule: Given $f(x,t)$and $\frac{\partial f}{\partial t}$ are continuous for x in $[a,b]$ and t in $[t_0 -\delta, t_0 +\delta]$ for some$\delta>0$. Then

(12)
\begin{align} \frac{d}{dt} \int^{b}_{a}{f(x)dt=\int^{b}_{a}{\frac{\partial f}{\partial t}(x,t)dx \end{align}

This theorem is similar to Theorem 2 in chapter two of Real Mathematical Analysis in that it allows one to switch the order of operation between differentiation and integration, just as in Theorem 2 converging sequences are sent to convergent sequences by continuous functions acting on them.

The Fundamental Theorem of the Calculus of Variations: If $f(x)$ and $\eta(x)$ are continuously differentiable functions on $[a,b]$ with $\eta(x)$ equaling zero at a and b, then

(13)
\begin{align} \int^{b}_{a}f \eta' = - \int^{b}_{a}f' \eta \end{align}

This is a special application of integration by parts and it essentially says that given the special condition that $\eta$ vanishes at its end points, the derivatives of the functions $f$ and $\eta$ can be switched with a single negation.

Both of the above theorems are used to prove the following theorem, the Euler's Equation. This theorem and its corollaries are extremely useful in solving optimization problems, as will soon be demonstrated.

Euler's Equation: A local minimum or maximum y=$f(x)$ from all $C^2$ functions of a cost

(14)
\begin{align} \int^{b}_{a}F(x,y,y')dx \end{align}

given $f(a)$ and $f(b)$, satisfies Euler's Equation:

(15)
\begin{align} \frac{\partial F}{\partial y}=\frac{d}{dx}\frac{\partial F}{\partial y'} \end{align}

This equation enables one to find solutions to the cost function $\int^{b}_{a}F(x,y,y')dx$, provided such solutions exist. Unfortunately, this equation does not guarantee the existince or uniqueness of the solution, but it is an extremely important tool in finding a local minimum or maximum function y to solve the cost function. To illustrate how this theorem can be used, we will use the theorem to find the function $y=f(x)$ such that the following integral is minimized:

(16)
\begin{align} \int^{2}_{1}\frac{y'^2}{x}dx \end{align}

with initial conditions $f(1)=1$ and $f(2)=2$.
Since in this example, $F(x,y,y')=\frac{y'^2}{x}$, we know that $\frac{\partial F}{\partial y}=0$ since y does not appear explicitly in $F(x,y,y')$. Now, since $\frac{d}{dx}\frac{\partial F}{\partial y'}=0$, we know that $\frac{\partial F}{\partial y'}$ must be constant since its derivative is equal to zero. Thus we obtain

(17)
\begin{align} \frac{\partial F}{\partial y'}=2\frac{y'}{x}=c \end{align}
(18)
\begin{equation} y'=c x \end{equation}
(19)
\begin{align} y=\frac{1}{2}c x^2+d \end{align}

Then using the initial conditions, we can solve for c and d to obtain:

(20)
\begin{align} y=\frac{1}{3} x^2+\frac{2}{3} \end{align}

When F does not depend on x or y explicitly, the Euler's Equation can be simplified, as the above example demonstrated. The function $F(x,y,y')$ did not depend explicitly on y, so the First Integral theorem applies to it.

First Integral: If F does not depend explicitly on y, then Euler's Equation becomes

(21)
\begin{align} \frac{\partial F}{\partial y'}=C \end{align}

Likewise, if F does not depend on x explicitly, then Euler's Equation is equivalent to

(22)
\begin{align} F-\frac{\partial F}{\partial y'} y'=C \end{align}

This can be proven by simply taking the derivative of both sides with respect to x and thereby arriving at Euler's Equation.

The First Integral theorem is very practical in solving one of the most famous problems in the calculus of variations, namely the brachistochrone problem. This problem was first posed by John Bernoulli in 1697 and was solved by some of the world's greatest mathematicians at the time including Newton, Leibniz, and both John and James Bernoulli. The brachistochrone problem ask for the curve that minimizes the amount of time it takes for a puck to slide from one point to another point (assuming no friction), as demonstrated below.

Brachistochrone.png

If we set the axes up so that the y axis points downward, and we let the puck start at the origin, we know from the Conservation of Energy Law that the potential energy of the puck will be translated into kinetic energy. That is,

(23)
\begin{align} \frac{1}{2}m v^2=m g y \end{align}

where m is the mass of the object and g is the gravitational constant. Simplifying this and using some basic calculus facts, we obtain the following:

(24)
\begin{align} v=\sqrt{2 g y} \end{align}
(25)
\begin{align} v=\frac{ds}{dt}=\sqrt{2 g y} \end{align}
(26)
\begin{align} dt=\frac{ds}{\sqrt{2 g y}}=\sqrt{\frac{1+y'^2}{2 g y}} \end{align}

Integrating both sides from $t_0=0$ to $t_1=a$, we get:

(27)
\begin{align} T=\int^{a}_{0}\sqrt{\frac{1+y'^2}{2 g y}} \end{align}

where T is the total time elapsed. Recognizing that the function F does not depend on x explicitly, we apply the First Integral theorem to find the minimal curve that satisfies the above equation. After doing some basic algebra, we obtain:

(28)
\begin{align} \sqrt{\frac{y}{2c-y}}=dx \end{align}

Making the substitution $y=2 c \sin^2{\theta}$ allows us to solve for x after some basic algebra and making use of some trigonometric identities.

(29)
\begin{align} dx=2 c (1-\cos{2 \theta})d\theta \end{align}
(30)
\begin{align} x=2 c (\theta-\frac{1}{2}\sin{2 \theta})+C \end{align}

Since initially x and y are zero, the constant C in the above equation must be zero. Then substituting $\phi=2 \theta$ into the equations for y and x, we arrive at the following parametric equations:

(31)
\begin{align} y=2 c \sin^2{\frac{\phi}{2}=c (1-\cos{\phi}) \end{align}
(32)
\begin{align} x=c (\phi-\sin{\phi}) \end{align}

This equation is the equation for the cycloid, which amazed mathematicians at the time because the cycloid is famous for having many other unique properties and was well studied by other mathematicians at the time. The brachistochrone problem demonstrates the power and utility of the calculus of variations not just in applied optimization problems, but also in pure mathematics problems.

Relevance

The relevance of Calculus of Variations begins with the Optimal Control Theory, which focuses on “finding a control law for a given system such that a certain optimality criterion is achieved.” Mathematically, “optimal control is a set of differential equations describing the paths of the control variables that minimize the cost functional.” The cost functional is written as

(33)
\begin{align} J=\phi(x(T)) + \int^{T}_{0} L(x,u,t)dt \end{align}

where $T$ is the terminal time of the system and $L(x,u,t)$ is the Lagrangian [2].

From the Optimal Control Theory, we can begin to see how we appply calculus of variations to real-world situations. Some real-world applications that branch from the Optimal Control Theory are Economic Strategies and Utility of Consumption. For Economic Strategies we say, “our goal is to minimize the total cost, the sum of production and inventory.” We interpret this as

(34)
\begin{align} C=\int^{T}_{0}(c_1 x'^2 + c_2 x)dt \end{align}

we then use the Euler equation (15) to get the general solution

(35)
\begin{align} x=B{\frac {t}{T} - {\frac {c_2}{4c_1} t(T-t)}} \end{align}

Another economic strategy is to delay production meanwhile our money earns interest, which is represented by

(36)
\begin{align} C=\int^{T}_{0}e^{-rt}(c_1 x'^2 + c_2 x)dt \end{align}

After applying Euler’s equation (15), we get the general solution

(37)
\begin{align} x''={\frac {c_2}{2c_1}+rx'} \end{align}

Utility of Consumption is a method used to determine the maximum utility or “happiness” in ones life. This is interpreted as

(38)
\begin{align} \int^{T}_{0}U(C(t))dt \end{align}

where $C$ is the consumption and $U(C)$is the utility. To maximize lifetime utility/happiness we plan to spend all the money we have before we die. What we find is that in order to maximize ones utility “one should so arrange one’s finances, so that spending grows at the same rate as investments.”[5].
One example of the Optimal Control Theory is Henry Ford’s use of the economic strategy. Henry Ford used the idea of economic strategy in industry. Everyone knows of Henry Fords innovation of the assembly line, but not everyone know that after making the Model T his plants were shut down in order to redesign the equipment to allow more production. Ford realized that in order to maximize his profit and avoid redesigning his equipment, he had to make specialized equipment that would be flexible and programmable, “multi-task production equipment.” As a result of calculations, Ford industries now use “‘transfer line’ technology.” This allows Ford plant to “produce a variety of outputs efficiently."[6].

References

1. "Calculus of variations", [wikipedia: Wikipedia entry on Calculus of Variations].
2. "Optimal Control", [wikipedia: Wikipedia entry on Optimal Control Theory].
3. "Calculus of Variations", MA 4311 Lecture Notes. I.B. Russak, Dept. of Mathematics, Naval Postgraduate School, Monterey, CA, 9 July, 2002.
4. "Leonhard Euler", Wikipedia entry on Leonhard Euler.
5. "Real Analysis and Applications: Including Fourier Series and the Calculus of Variations", Frank Morgan. American Mathematical Society 2005.
6. "The Economics of Modern Manufacturing: Technology, Strategy, and Organization", Paul Milgrom and John Roberts. American Economic Association June 1990.
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License