Variational derivative

Zadorozhny's now reading lectures on what he calls "variational methods and random processes". Despite of some common abuse of notation the course seems to be qute useful. I'll post some summaries of what he's teaching us. Yet I'll use slightly different notation and thus as always, all the good is his and all the wrong is mine. This is the part about variational derivatives.


Let be one of the following Banach spaces:

  • the space of bounded continuous functions defined on the interval with the norm:

  • the space of functions possessing bounded continuous derivatives:

  • the space of classes of equivalent functions with the norm:


We consider a functional and we consider it the context of variational problems or optimal control problems. The common approach to investigate these problems is linearization. Now we could quite naturally begin with consideration of Frechet derivative of (assuming it has one): where is a bounded linear operator and is the Frechet derivative of at the function . Yet it turns out there's something even more convenient. As we saw earlier12 the functional is often an integral one. This and some further exploration leads to the following definition:

The function is called a functional or variational derivative of if the Frechet derivative at any function is a bounded integral operator with the kernel

In simpler words, the variational derivative is a kernel of the Frechet derivative.

Nota Bene: while the Frechet derivative is unique it may have infinitely many equivalent kernels.


There's an established notation for the variational derivative. As you could've guessed it's a bit messy. Although aware of the distinction, in his works Zadorozhny usually doesn't distinguish a function from its value a derivative from the increment of the value associated with specific increment of the argument --- that is the derivative applied to that increment. The wikipedia page3 is prone to a similar abuse of notation.

Thus so far we will use the following notation for the value of the functional derivative at specific function and time :

Sadly, Zadorozhny often calls this thing a function which of course it isn't. He also uses this same symbol for the value of the derivative and for all of the functions , , It's common though. The real problem comes from the fact that he also calls both a derivative and a differential (and not just their value) the following expression:

I suppose that's because he's talking too much with physics folks. The common notion of a derivative is that of a linear mapping which approximates the function under consideration as good as possible. The integral above isn't a derivative of but the function is. The notion of a "differential" is then a non-sense. As opposed to the notion of a "differential associated with specific change ". Which is still excessive. That's a long and old story though which probably deserves its own writeup.

To denote the function itself I'd rather propose the notation

Yet obviously things are going to get more complicated than that when it comes to multivaried functionals and the change of variables. In these cases I believe multiindex notation could be used just as with usual derivatives.



It is important to note that spaces are included in each other as normed spaces so that if is a bounded linear operator in then it is just as well bounded in all . And so are .

Suppose we have two norms which lie in the following relation:


Thus whenever

it's also

In other words if as then as .

Now it's easily seen that


which proves our assertion about inclusions.

Properties or "The rules of variational differentiation"

Here one needs to be precise in what spaces the derivative is bounded and where it's not. Despite of that I'll omit proofs for some trivial cases.

Derivative of a constant functional


If has a variational derivative, then so does and:


If the functionals and have variational derivatives, then so does their sum and the following equality holds:

Derivative of a multiple

If the functionals and have variational derivatives then so does and:

That is .

Chain rule 1

Suppose has a variational derivative and is a function. Consider a functional



Chain rule 2

Suppose has a variational derivative and is a function. Consider a functional

If is , then has a variational derivative and:

Just in case you're not familiar with this notation, let's rewrite this definition using an auxilliary function defined by when we fix some . Then .

Then the statement above can be rewritten as:


Let's consider an increment:

In the second line we used a linear approximation of the value of a differentiable function .

In the third line we've rewritten the preceding expression so that we get the expression of the form , where and are two functions.

This allows us to use a linear approximation in the fourth line.

Note the in the end. It's actually .

It follows then that in (and in by inclusion) the variational derivative of does exist and:


  1. {{ site.url }}link://slug/var-calc-criteria 

  2. {{ site.url }}link://slug/opt-criterions 




Comments powered by Disqus