Linearity of Expectations

[SQUEAKING] [RUSTLING] [CLICKING] YUFEI ZHAO: In this video, let us look at a basic yet important concept in probability known as linearity of expectations and use it to deduce some interesting consequences in combinatorics via the probabilistic method. Linearity of expectations says that, if you are given random variables, x1 to xn, and constants, C1 through Cn, then when we take a linear combination of these random variables, as such, C1 times x1-- so imagine these are real valued random variables and these are real constants, for instance-- then the sum of these C1 times x1 plus C2 times x2 and so on plus Cn times xn-- so this sum has expectation-- the following, which can be computed by distributing this expectation symbol across to the individual variables. So this is a basic and important property. And it's worth noting that a similar statement written for products is often not true. So it is not usually the case that the expectation of a product of two random variables is the product of their expectations, unless you're in some special circumstances, such as when x and y are independent or uncorrelated. Anyway, let us focus on the linearity of expectations and see some ways to use this in combinatorial applications. And the first example is the following question, which will be able to give a very quick and clean answer. The question is, what is the average number of fixed points of a permutation of 1 through n chosen uniformly at random? As a reminder, if we have numbers 1, 2, 3, 4-- here, n equals to 4-- then a permutation can be thought as some way to map these numbers, the sets back to themselves, in one to one correspondence. And a fixed point is some number that gets mapped back to themselves. So in this permutation, there are exactly two fixed points. OK, great. Let's answer this question using linearity of expectations. One way to think about the problem is that there are n factorial permutations. And maybe we want to try to go about counting how many of them have zero fixed points, how many of them have one fixed points, how many have two fixed points. But that method can get pretty cumbersome and pretty difficult, quite quickly. However, if we look at this problem through the lens of linearity of expectations, there turns out to be a very quick solution. And the method is to introduce some random variables. Let xi be a random variable that equals to 1 if i is a fixed point, meaning that in this permutation i gets mapped to itself and 0 otherwise. So x sub i is the indicator random variable for the element i being a fixed point of this random permutation. What's the expectation of xi Well, this is the probability that i is a fixed point. Well, this being a uniform permutation chosen uniformly at random, i is sent to each of the elements 1 through n with equal probabilities. So in particular, it is sent back to itself with probability 1 over n. And then the number of fixed points equals to the sum of the xi's-- so x1 plus x2, and so on, to xn. And now, we can take expectation on both sides and apply linearity of expectations and see that, well, each individual term is 1 over n and there are n such terms. So the answer is 1. And that is the answer to this question. So the average number of fixed points of a permutation chosen uniformly at random is exactly 1. And you see that there's a very quick calculation once you get the hang of the idea of linearity of expectations. Let us look at a slightly more interesting example. And for this example, we'll consider the concept of a tournament. So a tournament is a concept in graph theory referring to the following. There are n vertices. Think of n players in some tournament. And between every pair of vertices, we have some directed edge pointing in one of these two directions. So every pair of vertices, there is some edge pointing to one of the two directions. OK, so that's an example of a tournament. I'll need to introduce another concept, which is that of a Hamilton path. So a Hamilton path is a directed path, meaning we travel along the directions of these edges according to their directions. So directed path that passes through each vertex, so every vertex of the graph exactly once, no more, no less. So let's see if we can find any Hamilton paths in this example. Well, I see one. So if we start with a middle vertex and go along this one, this edge, then this edge, and then this edge, so that's a Hamilton path that goes through all four vertices, each vertex passing through it exactly once. And it always traverses along the direction of the edges. Let us prove the following theorem. For every n, there exists a tournament of n vertices with at least n factorial times 2 to the minus n plus 1 Hamilton paths. So in other words, for every n, there is some way to orient the edges of the complete graph on n vertices so that it has lots and lots of Hamilton paths, specifically at least this many. So that's a theorem that we're aiming to prove. We will not prove this theorem by explicitly constructing such a tournament. Instead, we'll invoke the probabilistic method and show that a random tournament has an expectation of this property. So here's the proof. Let's consider a random tournament on n vertices chosen uniformly at random. One way to do this is to take a complete graph on n vertices and for every edge flip a fair coin and use that coin to decide which way the edge orients of the two different directions. Now, let's think about the number of Hamilton paths. So each path of the-- so each-- so each of the n factorial permutations of vertices forms a directed path. OK, so first, consider a permutation of vertices-- for example, 2, 1, 3, 4. And think about what is the probability that the edges that go in the direction of this permutation-- that the edges are oriented according to the permutation. Well, we have to flip n minus 1 coins. And all of them have to come up in such a way so that the edges are pointing in the direction of this permutation, in the order of this permutation. So the probability that this is a directed path for each of these permutations is precisely 2 to the minus parentheses n minus 1. And now, we invoke the linearity of expectations to claim that the expected number of Hamilton paths must then be-- well, each of the n factorial permutations has probability of 2 to the minus n minus 1 of being a directed path. So this is a calculation that is analogous to the calculation that we did in the earlier part of this video. Well, this is what happens in expectation, on average. And thus, there must be some instance where we can beat this average or at least be at least as large as this average. So thus, there exists a tournament with at least this many Hamilton paths. And that concludes the proof of this theorem that we laid out earlier. So this is an example of applying linearity of expectations as a step in the probabilistic method to prove this nice and simple result, that there exists tournaments with lots of Hamilton paths.