Open course ware

PROFESSOR: OK, let us resume. I had no idea how many people would be here today, and I think I made 25 copies of the handout. And I see 25 names on the list. And that means that two people did not get a copy of the syllabus. Does anybody need a copy? That's strange. OK. All right. We covered some introductory material, and I think we've covered enough that you can do a problem set. So it gives me great pleasure to hand out problem set number one. OK, you can think about that. It is the sort of problem that will either take you two minutes or two hours or infinity. So don't spend too much time on it, but I would like you to put your name on it and turn it in either at the end of the hour or next time so I can make comments if there's something that's mostly right but not quite right. Let's return to these three simple patterns that we put on the blackboard. And let me make another point about symmetry. The people who sensed that this pattern and the...

In this video, let us look at a second quantity of interest that has to do with absorbing states. Now that we know how to calculate the probability of getting to a given absorbing state, we would like to know how long it would take to get to it. Let us first deal with that question when we have only one absorbing state. Let us consider the following Markov chain, which is a little simpler than the one that we had in the previous video. We have transient states. One, two, and three are transient states. And we have one recurrent state, four. And that recurrent state four is an absorbing state, because once you get to it you stay there forever. So in this simple example, the absorption probability to four is trivially one. No matter where you start, with probability one you're guaranteed that eventually you will reach four. But the question of interest is to know how long would it take to get to four. In other words, how many transitions would you have to do until you r...

Let us now illustrate, with an example, the calculations of n step transition probabilities that we have just discussed. In this example, we are given a two state Markov chain, and as part of the input, the one step transition probabilities between these two states. So, given that you are in state one, the probability that you will next go to state two is 0.5, and the probability that you will stay in state one is 0.5. And, given that you are in state two, the probability that you will next go to state one is 0.2, and the probability that you will stay in state two is 0.8. Now, suppose that you start in state one, and you would like to calculate the probability of being in state one after n transitions, or after n steps. With our notation here, this is r11 of n. That probability can happen in two ways. After n minus 1 steps, you end up in state one, and then for the last transition, you stay in state one, or after the first n minus 1 transition, you end up in state two, a...

Let us now abstract from our previous example and provide a general definition of what a discrete time, finite state Markov chain is. First, central in the description of a Markov process is the concept of a state, which describes the current situation of a system we are interested in. For example, in the case of the checkout counter example, the number of customers in the queue provided the right level of information needed to define a useful state. Time is assumed to be discrete, that is, divided in discrete time steps. The system starts at time 0 in an initial state, and at each successive time step, the system goes from its current state to a next one chosen with some randomness. As a result, after n such transitions, the state of the system will be random, and so we can think of it as a random variable. Let Xn be this random variable. That is, Xn represents the state in which the system is after n transitions from an initial state in which it started to operate. As a...

So let us start with our example. Suppose that you go to a supermarket, and start observing customers arriving and leaving from a given checkout counter. Assume that there are two customers in the queue when you arrive. For simplicity, assume also the customers come one at a time, that there is a single queue, and that the customer in front of the queue is the one getting served by the clerk. So what events of interest could happen then? A new customer could join the queue, which is an arrival. Or the customer currently being served is done, and leaves-- departure. Or both events could happen. Now for making our model more precise, we need to specify the processes of the customer arrivals and departures. And for that, let's use some simple discrete time stochastic processes, which we have introduced before. So as usual, we first divide time into discrete time steps, say in seconds. Here n equals 0 would correspond to the time when you arrived. For arrivals, let's ...

PROFESSOR: Let us consider the anharmonic oscillator, which means that you're taking the unperturbed Hamiltonian to be the harmonic oscillator. And now, you want to add an extra term that will make this anharmonic. Anharmonic reflects the fact that the perturbations are oscillations of the system are not exactly harmonic. And in the harmonic oscillator, the energy difference between levels is always the same. That's a beautiful property of the harmonic oscillator. That stops happening in an anharmonic oscillator. The energy differences can vary. So the things, if you have a transition from one level, first level to the ground state, or second level to the ground state, one is not the harmonic of the other because they're not exactly twice as big as each other. So let's try to add an x to the 4th perturbation, which is intuitively very clear. You have a potential. And you're adding now an extra piece that behaves like x to the 4th. And it's going to...

Let us now come back to the trajectory estimation problem that we introduced earlier. We have an object that moves vertically. At any given time t, the height at which the object is found is equal to this expression. It corresponds to the following-- the object starts at time 0, at some initial height Theta0, it has an initial velocity of Theta1, but also has a certain acceleration. And if Theta2 is negative, this will be a downwards acceleration, which means that the object eventually will turn and start going down. So this is a typical trajectory of such an object, where here we're plotting the height as a function of time. However, the Thetas are unknown and they are random-- we do not know what they are. So this blue curve is just a simulation where we drew values for those random variables at random. But if we were to simulate again, we might obtain a somewhat different blue curve, because the values of the Thetas might have been different. We do not observe the ...

Let us now discuss in some more detail what it takes to carry out Bayesian inference, when both random variables are discrete. The unknown parameter, Theta, is a random variable that takes values in the discrete set. And we can think of these values as alternative hypotheses. In this case, we know how to do inference. We have in our hands the Bayes rule and we have seen plenty of examples. So instead of going through one more example in detail, let us assume that we have a model, that we have observed the value of X, and that we have already determined the conditional PMF of the random variable Theta. As a concrete example, suppose that Theta can take values 1, 2, or 3. We have obtained our observation, and the conditional PMF takes this form. We could stop at this point or we could continue by asking for a specific estimate of Theta-- our best guess as to what Theta is. One way of coming up with an estimate is to use the maximum a posteriori of probability rule, which lo...

Let us now revisit the subject of expectations and develop an important linearity property for the case where we're dealing with multiple random variables. We already have a linearity property. If we have a linear function of a single random variable, then expectations behave in a linear fashion. But now, if we have multiple random variables, we have this additional property. The expected value of the sum of two random variables is equal to the sum of their expectations. Let us go through the derivation of this very important fact because it is a nice exercise in applying the expected value rule and also manipulating PMFs and joint PMFs. We're dealing with the expected value of a function of two random variables. Which function? If we write it this way, we are dealing with the function g, which is just the sum of its two entries. So now we can continue with the application of the expected value rule. And we obtain the sum over all possible x, y pairs. Here, we nee...

Let us now put to use our understanding of the coin-tossing model and the associated binomial probabilities. We will solve the following problem. We have a coin, which is tossed 10 times. And we're told that exactly three out of the 10 tosses resulted in heads. Given this information, we would like to calculate the probability that the first two tosses were heads. This is a question of calculating a conditional probability of one event given another. The conditional probability of event A, namely that the first two tosses were heads, given that another event B has occurred, namely that we had exactly three heads out of the 10 tosses. However, before we can start working towards the solution to this problem, we need to specify a probability model that we will be working with. We need to be explicit about our assumptions. To this effect, let us introduce the following assumptions. We will assume that the different coin tosses are independent. In addition, we will assume...

Let us now study a very important counting problem, the problem of counting combinations. What is a combination? We start with a set of n elements. And we're also given a non-negative integer, k. And we want to construct or to choose a subset of the original set that has exactly k elements. In different language, we want to pick a combination of k elements of the original set. In how many ways can this be done? Let us introduce some notation. We use this notation here, which we read as "n-choose-k," to denote exactly the quantity that we want to calculate, namely the number of subsets of a given n-element set, where we only count those subsets that have exactly k elements. How are we going to calculate this quantity? Instead of proceeding directly, we're going to consider a somewhat different counting problem which we're going to approach in two different ways, get two different answers, compare those answers, and by comparing them, we're going t...