One efficient way to compute functional differential (draft)

In many advanced optimization field, you need to compute the 1st or 2nd differential of a functional. The rigor analysis of functional calculus is complex and difficult. But luckily, in most application, we just need to get the 1st order differential of a functional to compute the stable “point” (in fact it’s a function in the functional case). And further, to compute the 2nd order differential functional to check if the stable is the optimized point that we’re searching. From this perspective, what we need to do is making the process of computing 1st and 2nd order differential as convenient as possible.

Lots of books introduce the calculus of functional by variational method. But that’s a great amount of work. One efficient computation way is through the Taylor expansion in appendix of G.H. Fredrickson’s book The equilibrium theory of inhomogeneous polymers. I’d like to give some details of this great method.

Introduction to Dirac function

One useful method in the functional Taylor expansion is the Dirac $\delta$ function, which is defined by

and also constrained to satisfy the identity $\int_{-\infty}^\infty \delta(x) \, dx = 1$.

From Lebesgue integral with respect to the measure $\delta$, Dirac function satisfies

This function is used widely in engineering field. But its rigorous explanation involves massive knowledge about generalized function and advanced functional analysis, which is beyond the scope of this post. Here, we only treat it as some useful mid tool to help us compute the functional calculus.

Taylor Expansion

Let’s begin with the normal one dimension formula:

Then, let’s treat $f$ as $F$, $x$ as $\delta f$ and $x_0$ as $f$, we can get

Then considering the characteristics of Dirac $\delta$ function, the functional can be transformed like the book format:

where the $\Gamma_1(x)$ and $\Gamma_2(x, x')$ corresponding to 1st and 2nd order functional differentials. Also I guess, the general $n^{th}$ term may be like (it’s welcome to discuss if anyone disagree with this):

Now, we only need to compare the expressions with our own functional form, we can get the 1st and 2nd order differential functional.

Example of computing functional

Let’s first try to compute functional $F[f] = \int_a^b\ [f(x)]^n\ dx \stackrel{abbr.}{=} \int_a^b[f]^n\ dx$.

For the 1st order differential, we just need to scrutinize the equation $\int_a^b [n f^{n-1}\ \delta f] dx = \int^{b}_a \Gamma_1(x)\ \delta f(x)\ dx$. It’s obvious that

For the 2nd order differential, we need to consider the equation:

as the left side can be

the equation can be reduced as

Use the measure characteristics of Dirac function, and treat the $C_n^2 f^{n-2}\delta f$ as $x$’s function, the right side can be

Now, it’s easy to see the 2nd order differential of our functional:

where we used the symmetry of Dirac function.

Another useful example is to compute the entropy functional.

TK