L24.6 A Numerical Example - Part I
Let us now illustrate, with an example, the calculations of n step transition probabilities that we have just discussed. In this example, we are given a two state Markov chain, and as part of the input, the one step transition probabilities between these two states. So, given that you are in state one, the probability that you will next go to state two is 0.5, and the probability that you will stay in state one is 0.5. And, given that you are in state two, the probability that you will next go to state one is 0.2, and the probability that you will stay in state two is 0.8. Now, suppose that you start in state one, and you would like to calculate the probability of being in state one after n transitions, or after n steps. With our notation here, this is r11 of n. That probability can happen in two ways. After n minus 1 steps, you end up in state one, and then for the last transition, you stay in state one, or after the first n minus 1 transition, you end up in state two, a...