Variational derivative

Zadorozhny's now reading lectures on what he calls "variational methods and random processes". Despite of some common abuse of notation the course seems to be qute useful. I'll post some summaries of what he's teaching us. Yet I'll use slightly different notation and thus as always, all the good is his and all the wrong is mine. This is the part about variational derivatives.

Spaces

Let be one of the following Banach spaces:

  • the space of bounded continuous functions defined on the interval with the norm:

  • the space of functions possessing bounded continuous derivatives:

  • the space of classes of equivalent functions with the norm:

Definition

We consider a functional and we consider it the context of variational problems or optimal control problems. The common approach to investigate these problems is linearization. Now we could quite naturally begin with consideration of Frechet derivative of (assuming it has one): where is a bounded linear operator and is the Frechet derivative of at the function . Yet it turns out there's something even more convenient. As we saw earlier12 the functional is often an integral one. This and some further exploration leads to the following definition:

The function is called a functional or variational derivative of if the Frechet derivative at any function is a bounded integral operator with the kernel

In simpler words, the variational derivative is a kernel of the Frechet derivative.

Nota Bene: while the Frechet derivative is unique it may have infinitely many equivalent kernels.

Notation

There's an established notation for the variational derivative. As you could've guessed it's a bit messy. Although aware of the distinction, in his works Zadorozhny usually doesn't distinguish a function from its value a derivative from the increment of the value associated with specific increment of the argument --- that is the derivative applied to that increment. The wikipedia page3 is prone to a similar abuse of notation.

Thus so far we will use the following notation for the value of the functional derivative at specific function and time :

Sadly, Zadorozhny often calls this thing a function which of course it isn't. He also uses this same symbol for the value of the derivative and for all of the functions , , It's common though. The real problem comes from the fact that he also calls both a derivative and a differential (and not just their value) the following expression:

I suppose that's because he's talking too much with physics folks. The common notion of a derivative is that of a linear mapping which approximates the function under consideration as good as possible. The integral above isn't a derivative of but the function is. The notion of a "differential" is then a non-sense. As opposed to the notion of a "differential associated with specific change ". Which is still excessive. That's a long and old story though which probably deserves its own writeup.

To denote the function itself I'd rather propose the notation

Yet obviously things are going to get more complicated than that when it comes to multivaried functionals and the change of variables. In these cases I believe multiindex notation could be used just as with usual derivatives.

Examples

Inclusions

It is important to note that spaces are included in each other as normed spaces so that if is a bounded linear operator in then it is just as well bounded in all . And so are .

Suppose we have two norms which lie in the following relation:

Then

Thus whenever

it's also

In other words if as then as .

Now it's easily seen that

and4

which proves our assertion about inclusions.

Properties or "The rules of variational differentiation"

Here one needs to be precise in what spaces the derivative is bounded and where it's not. Despite of that I'll omit proofs for some trivial cases.

Derivative of a constant functional

Homogeneity

If has a variational derivative, then so does and:

Additivity

If the functionals and have variational derivatives, then so does their sum and the following equality holds:

Derivative of a multiple

If the functionals and have variational derivatives then so does and:

That is .

Chain rule 1

Suppose has a variational derivative and is a function. Consider a functional

Then

Proof

Chain rule 2

Suppose has a variational derivative and is a function. Consider a functional

If is , then has a variational derivative and:

Just in case you're not familiar with this notation, let's rewrite this definition using an auxilliary function defined by when we fix some . Then .

Then the statement above can be rewritten as:

Proof

Let's consider an increment:

In the second line we used a linear approximation of the value of a differentiable function .

In the third line we've rewritten the preceding expression so that we get the expression of the form , where and are two functions.

This allows us to use a linear approximation in the fourth line.

Note the in the end. It's actually .

It follows then that in (and in by inclusion) the variational derivative of does exist and:

Bibliography


  1. {{ site.url }}link://slug/var-calc-criteria 

  2. {{ site.url }}link://slug/opt-criterions 

  3. https://en.wikipedia.org/wiki/Functional_derivative 

  4. https://en.wikipedia.org/wiki/H%C3%B6lder%27s_inequality 

Formal series

I intend to write a series of posts on some topics I found often to be vaguely or mis-represented in baccalaureate's math courses. This one's about something called "formal series" in discrete mathematics. It's written in a rough and inaccurate manner because the true reason I'm publishing it now is that I'm afraid to forget about it again. I'll (re)write this whole thing later when I have time. Yet here's the gist (a bit more under the cut): don't try to fool students and call it "application of complex analysis to performing some operations on sequences".

These dudes introduce some magical "formal series" without specifying what they really are and what sequences are and why all of this really works. The way it oughts to be taught is as follows:

  1. Since the very first semester (or even better in school) the folks should learn some (very basic) basics of functional analysis. Sets and functions. And the difference between the a funciton and its value at some point
  2. Then you teach them that a sequence in the set \( X \) is a map from \( \mathbb{N} \) to \( X \).
  3. You teach them that a series in the v.s. \( X \) is essentialy a sequence of partial sums and the series and and sequences in v.s. are interchangeable
  4. You teach them about differentiability and power series
  5. If your for your sequence \( a \) the value \( \limsup_k\sqrt[k]{|a_k|} \) is finite then the corresponding power series \( \sum_k a_k (x - x_0)^k \) is convergent in some disc. That's right, it's Hadamard's bit, works well in Banach spaces
  6. If you add one series to another (or multiple by a constant or do you name what to it) then it's going to just change the radius of convergence somehow. But the series will still converge at least somewhere. Well actually I'm not sure right now what happens if they're centered around different points I'll look through it after I finally get some sleep.
  7. It's an (absolutely) convergent powerseries thus it defines a differentiable function!
  8. You could try and mix and split your series into some parts resembling Taylor series of some well-known functions
  9. How now, you've got combinations of some trivia? I reckon you know something about adding \( \frac{1}{1-ax} \) to \( \frac{2}{1-bx} \).
  10. Mix and expand!

You heard of IPFS? I'm glad it's out there. And I'm not

Just learned about IPFS1. Skimmed through the paper, watched the seminar talk. It's cool. I'm glad it's being adopted. But it's all damn straightforward. All these things I've been thinking through and discussing with friends and some random people at least since 2014. All this time I've been thinking "all right, I'll finish the current thing and then finally just implement these basic ideas as is, plain and straightforward; just to get started". Now I see someone's done it, plain and straightforward. I don't really know how many times I've run into this... I'm back to being depressive. I'm wasting my time. Though I shall learn something about distributions and Banach modules now. I shall not spread myself over all of it at once.


  1. https://ipfs.io 

Projections

There are some things about projections that, it turned out, I was getting all wrong. Specifically, it concerns the norms of projections. All right, I shall confess: I never realized the norm could be greater than one.

Recap

A linear operator \( P: X\to X\) is called a projection if \( P^2 = P \). Suppose \( X \) is a normed v.s. Then we can introduce the "operator norm": \[ \|A\|_{\mathrm{op}} = \inf \{C\in\mathbb{R};\ \|Ax\|\leq C\|x\|\ \text{for all}\ x\in X\}\] for those operators \( A \) for which it's finite. Here \( \|\cdot\|:X\to\mathbb{R} \) is the norm in \( X \). That value happens to equal \[ \|A\|_{\mathrm{op}} = \sup_{x\neq 0} \frac{\|Ax\|}{\|x\|} = \sup_{\|x\|=1} \|Ax\|. \] Such operators form a linear subspace of the space of all operators on \( X \). Moreover they form a Banach Algebra with the usual multiplication (composition) of operators and the submultiplicative norm \( \|\cdot\|_{\mathrm{op}} \): \[ \|AB\|_{\mathrm{op}} \leq \|A\|_{\mathrm{op}}\|B\|_{\mathrm{op}} \]

Wrong intuition

The (wrong) first intuition is that for a non-zero projection there will be \( \|P\|=1 \). Here's where this deception comes from. I'd erroneously think of orthogonal projections and expect \[ Px = \left\{\begin{aligned} & x,\ \text{for}\ x\in X_1,\\ & 0,\ \text{otherwise}.\end{aligned}\right. \] Then obviously \( \|P\| = \sup_{\|x\|=1} \|x\| = 1. \) Yet the projection needs not be of the form above --- it may be oblique.

Here's an example AG Baskakov used to demonstrate this phenomenon to me --- his ignorant student. Consider \( X = \mathbb{R}^n \) and an operator \( P \) defined by the formula: \[ Px = (x, a) b, \] for some fixed \( a, b \in X \). Then \( P^2 x = Py = (y, a) b = (x, a) (b, a) b, \) where \( y = Px = (x, a) b \). Thus chose we such \( a, b \) that \( (a, b) = 1 \) the operator \( p \) would be a projection. Now to its norm: \[ \sup_{\|x\|=1} \|Px\| = \sup_{\|x\|=1} |(x, a)|\|b\| = (\frac{1}{\sqrt{a}}a, a)\|b\| = \|a\|\|b\|. \] Could we pick \( a \) and \( b \) to satisfy \( (a, b) = \|a\|\|b\|\cos\phi = 1 \) and \( \|a\|\|b\| > 1 \)? Most certainly, sir.

Fix

But let's just see what can be told from the definition. Since \( P^2=P \) and since the norm is submultiplicative it's clear that: \[ \|P\|_{\mathrm{op}} = \|P^2\|_{\mathrm{op}} \leq \|P\|_{\mathrm{op}}^2. \] This in turn implies \[ 1\leq \|P\|_{\mathrm{op}} \]

Features of orthogonal projections

Academia, part 1. Research supervisors -- necessity.

Academia is conservative. It's got a central tendency which results in a highly-coupled federated system with a rather high entrance barrier. One of the requirements has so far been that you must follow a scientific supervisor... until it's time to lead. The new communication capabilities make us question the position of this and many other established attributes of Academia. We ask why it's so and conclude that the scientific community needs to learn a decentralized way from software developers

The questions

Sat, Feb 16 2018, visiting AG Baskakov I finally get to try and discuss these problems as well as the perspectives of research in and outside Voronezh.

Here I will just outline some propositions of AG Baskakov addressed to a student standing just in the begining of his path. I believe these propositions pretty much describe the way Academia is today. The quoted are the translated pieces of our dialogue and the rest are my comments.

  • "You won't achieve anything without a supervisor", i.e. you must begin with a supervisor.
    • "Try and find any recognized mathematician who made it his own way. You won't find one! There was Gallois but he grew up in France", i.e. an appeal to history.
  • "You also don't need anyone besides a supervisor", it suffices to have a supervisor.
  • "You don't need people of your generation" --- you could think you need a competitive environment, someone to share and exchange ideas with, an inspiration for something completely new, or solving some applied problems in "background", but you really don't. These things are optional.
  • It doesn't matter if there aren't in the city many other people doing real research or if the most researchers at the university aren't exactly in tune with the rest of the world. Apparently, you're supposed to abide.

How it came to this

Newton said:

"If I have seen further it is by standing on the shoulders of Giants"

It is known that Science isn't built just by a man --- man's life's too short. Science isn't even built by men of a generation --- we only further the knowledge acquired by the predecessors as far our abilities allow and our needs require us to. Hence 'tis natural that humankind's knowledge was commonly being developed by groups of people following the most prominent of us, then by the ones who followed the followers, and so till us. It's an ever evolving system with many branches and clusters. But its key feature to note is that one's first struggle always is finding an entrance. An entrance is someone who'd become their advisor, their teacher, and mentor. Someone whose line they would follow and further.

Is there no fallacy? Do ones have to find themselves what's basically a second father? Do they have to choose a single line and can't one pursue many?

It just wasn't feasible before but it may be now with the new technology. The many pieces that are the Scientific Knowledge today we could build together into a connected system available to everyone. The current state of research, the latest challenges and achievements could be tracked online in a consistent way. We could provide a guidance for newcomers in the form of manuals&tutorials, as well as lists of open issues, ranked and classified. That's what engineers and software developers are used to do. That could be the entrance. That could smoothen this steep learning curve and lower the threshold.

Yet not only could it be an entrance it could also define the workflow. We already have open review systems. We've even got services resembling social networks for scientists where you could track one's path and interact in some ways. We've got git for cooperation. We saw outstanding systems evolve with many independent contributions of many unrelated people in an almost random manner1. We know now that this randomness also provides some validation of the result and increases the robustness. Academia oughts to adopt and improve these decentralized methods.

Conclusion

An appeal to history doesn't seem to be a convicing argument. I believe supervisors as they are now at least can be optional though it'll always be easier to be guided by someone experienced. The immediate concern should be making research contribution-friendly and adopting the existing task-tracking and version control tools for common research. We shall keep the image of FOSS community in mind. It's only an initial stage and we shall learn our needs in progress before we can build more elaborate tools.

See the next writing in this series.


  1. Raymond, Eric. "The cathedral and the bazaar." Knowledge, Technology & Policy 12.3 (1999): 23-49. URL: www.catb.org/esr/writings/cathedral-bazaar/ 

Academia, part 2. Research supervisors -- sufficiency.

We've talked a little about certain aspects of Academia in the previous writing. Specifically we've tried to cover the question of whether it's necessary to find and follow a supervisor. Now we're questioning the sufficiency part. The main argument for insufficiency is the problem of an emotional burnout.

Just like writing or software development, research is a kind of activity that may lead to a burnout. There are different ways to deal with it. For example, one person said1 that

"There are two math schools in Voronezh. There's the school of the followers of Kipriyanov --- these are the ones who drink, and there's a school of Baskakov --- the ones who run"

But if we mean to take the problem seriously, as we do, we must admit that drinking isn't a reliable solution and running's not enough.

To be continued


  1. TODO: Attribution? 

Young Marx [would hate the Soviet Union]

It was Feb 13, 2018. Yet another day of exhaustion and desperation. But later in the evening the "Young Marx" was being streamed in the SPARTAK. It's been a solace.

A play by Richard Bean and Clive Coleman directed by Nicholar Hytner and set in the newly-opened London Bridge theatre tells us a story of young Marx and Engels (Engels and Marx!). Or rather a story of young and ambitious pennyless outlaw genious hiding out in London from debts, his wife's relatives, and the consequences of 1848. Listen to Bean and Coleman to learn about the play: - What's it about? - How much is true?

The play's quite modern setting. It's very rapid and vigorous. The music rocks! I would say that it's shot as good as the Hamlet (the one featuring Cumberbatch). It's a damn Masterpiece!

Theoretical

I've been thinking for two days now and couldn't figure out the part that scratched my mind. Was is the same theme as with JCS --- the theme of a saviour who shall sacrifice himself for the sake of the greater good? Or was it about the motivation Marx had? Well, not just that. In the end I've found Rory Kinnear who played Karl had put it into words for me:

"What do you do that is practical when your whole life is about the theoretical?"

Brutalized

I really appreciated the "brutalized" scene. It's the one where Karl complains to Fridrich about growing brutalized and dehumanized because of his hard work, because of emotinal burnout, and his unfortunes (debts and family problems) and his guilt for the people he had "killed when those have read the Manifesto". Fridrich replies with much anger that it is Manchester where truly brutalized people live: they lead an existence so dehumanized that their children have to play in human shite their yards are covered with, and the adults working all-day at manufactories don't ever go to their homes to see their children. He tells Marx that these poor people need his work on capitalism to be finished --- the work that Marx is yet to begin.

I appreciated it for several reasons. First of all it's of course relatable to the aforementioned problem of dealing with the material world while working on abstracta. Did you just think "procrastination"? The second but no less important reason is that this scene should remind a believer about the problems Marx wanted to solve. Yes, I'm talking about the socialists and communists of this day taking the works of Marx as dogmata. I'm talking about them worshiping and defending the Soviet Union. These are the most terrible mistake you could make, comrades.

First of all, the Soviet Union was all about dehumanized existence. Every single "great achievement" of the Soviet World was associated with thousands and millions dead of exhaustion. The discipline was only built on uttermost fear. On executions with no prosecution. On humiliation. These were never a goal and now we know almost for sure these are no means. It's an empirical knowledge. And it's obvious that Marx would reproach all these things.

Secondly... Well I could rant on the Union being built and run by greedy ignorant morons with sadistic inclinations who have never in their lives read Marx. I could rant about the unscalable state-run economy --- the Plan. About the centralization. About the national problem. I could write it, but we all know about these things. Well, most of us. I'll write a post for the zealots later. The thing I'm saying is that we ought to learn from mistakes. And we know Soviet Union was all about mistakes. And God I'm sure, Marx'd learn. Admit and learn.

Our agenda shall be as follows:

The Soviet Union had nothing to do with socialism, nor communism, nor Marx, nor Engels, nor even Lenin though it'd be a surprise to some unaware people. The model and the problem Marx stated are still relevant are to be thoroughly studied in the context of the new world, the new communication and computation capabilities, and the context of the Great Soviet Failure.

I've been repeating the same thing for the last two years probably: we must take the class struggle model and go through the entire derivation once more and then again! We need to state clearly what the problem is and what the possible solutions are. And screaming "capitalism kills people" is no help.

Jesus Christ the Superstar

Another thing I watched in the last few months is that famous musical called Jesus Christ Superstar. Well yeah, I'm such a hillbilly that I hadn't even heard of it before. I never read a Bible. Also I strongly dislike everything related to religion, living in a country of churches, oligarch priests, saint KGB officers, criminalized offence of religious beliefs and all that. Also this thing about Occam's razor and biased priors... What I want to say now is that I found this musical surprisingly good. I even liked the story.

First of all I watched the 1973 movie: a few decent scenes, Judas's surely cool, the Caiphas too. But Jesus... not a single line to remember! Just a histerical bitch. Boring. So it looked like a bunch of hipsters surrounded another hipster and worshipped him for no particuar reason. Then some other dudes suddenly think this crows's dangerous (oh I can't even guess why!). And the other nigga dude tries to talk sense into his ole hipster friend before it's too late. Just as I said: boring.

A friend of mine then gave me a hint to listen to the original album recording featuring Ian Gillan as Jesus. Sad thing the dude was sort of busy travelling with Deep Purple when the movie was being shot because Gillan's damn good Jesus! The best Jesus I've heard of, actually! I mean, I'm now singing "I only want to say" almost every day after I heard Gillan sing it!

Once I'd found a decent Jesus I could appreciate the rest of the play. Here's what I think of it in the end: - Judas's almost the only good guy here. Perhaps his role is even more important than that of JC. He cares for the poor and he wants to do something about them. He also cares for the party: when he sees JC has just let things go he is trying to fix it, and persuade JC to restore the control over the mutinous crowd. - The musical is mostly about the crowds' evolution. Things begin with harmless "Hey sanna, Ho sanna!" and "Would you smile at me?" but eventually they become "Hey JC, JC, would you die for me?" which causes aforementioned JC to frown and really think about something.

This line culminates in "The Temple" with cripples literally tearing JC apart: "See my skin --- I'm a mass of blood! See my legs, I can hardly stand! I believe You can make me well!"

And JC screaming out: "Heal yourselves!"

In other words, don't do people good, they'll demand ever more. And after you're exhausted they'll make laugh of you. - The crowd doesn't care to help itself. They just want to be "saved" by someone else:

Hey JC, JC, would you fight for me? ... Would you touch me, would you mend me, Christ? ... I believe you can make me whole. ... I believe you can make me mell. - Yeah, "I only want to say" is damn good! But I also liked Judas talking to the priests.

Lost

I've re-watched the first two seasons of Lost a few weeks ago. Or perhaps it'd be more correct to say that I've watched it for the first time given that the time before it I was watching a translated version. Yet it was such a nostalgia! Anyway, a few observations:

  • Most of the characters sound rather british.
  • Everything's "sodding". British, isn't it?
  • Don't tell what I can't do.
  • Party activity idea: "I never" --- the game Sawyer and Kate play. It's quite simple: state a proposition of the form "I never [did smth]". Those who did shall drink.
  • Polar bear, Hurley speaking Korean, the whole dreamt-theme --- it's almost as good as Twin Peaks.