Lecture 9 Limsup, Liminf, and the Bolzano-Weierstrass Theorem

[SQUEAKING] [RUSTLING] [CLICKING] CASEY RODRIGUEZ: All right, so we proved these two theorems last time, and we used them for-- and we had a couple of applications of them. So the first theorem, simple theorem, was that a sequence converges to x if and only if the limit as n goes to infinity of the absolute value of xn minus x goes to 0. And then we also had the squeeze theorem, that if you have two-- if you have three sequences-- a sub n, b sub n, and x sub n-- so that x sub n is in between a sub n and b sub n, and the a sub n and b sub n converge to the same thing, then the sequence x sub n converges, and it converges to the [? common ?] limit of a sub n and b sub n. So x sub n gets squeezed in between a sub n and b sub n. Last time we used this to prove a kind of special limit that this equals 0 if the absolute value of c is less than 1. So I had proved it for c positive, and maybe c less than 1-- but same proof works with absolute values, just because the absolute value of c to the n equals the absolute value of c raised to the n-th power. So let's use this to do a few more special limits if you like. But first, I'm going to state the binomial theorem. I'm not going to prove it, but it's a simple exercise using induction. And the binomial theorem says that [? for all ?] [? N, the ?] natural numbers, x, y in R, x plus y raised to the n is equal to the sum from k equals 0 to n of n choose k, x to the n minus k y to the k. And here n choose k-- this is equal to n factorial over k factorial divided by n minus-- times n minus k factorial. OK. So again, you can prove this by inducting on n. All right. OK. We have a theorem, so we'll prove a few special limits. I have a real number that's positive. Then the limit as n goes to infinity of 1 over n the p equals 0. If p is positive, then limit as n goes to infinity of p to the 1 over n equals 1. And the third is-- it's just that a certain limit exists-- limit as n goes to infinity of n to the 1 over n equals 1. OK? Let me make a small comment here. So far in our discussion of the real numbers, we've only defined what it means to take a real number to an integer power, but-- and n-th roots. So we have n-th powers and n-th roots. So using that, we can then define how to take a real number to a rational power-- although there needs to be something that's checked to make sure this is well-defined, because you can always write a rational number not uniquely as one integer over another. So we can define a positive real number to a rational power, but using really only the elementary facts about the real numbers, the fact that it has the least upper bound property, and the fact that the rationales are somehow dense in the real numbers, we can then define what it means to take a positive real number to a positive real number power. OK? All of that is just to say that to define a positive real number to a positive real number or to a real number of power doesn't require the introduction of the exponential or logarithm-- although they, in the end too-- they both agree, once you do have the exponential and the logarithm. So all of that is just to say that what we've done up to now-- these things actually do make sense. You don't need the exponential and the logarithm to make sense of a positive real number to a real number of power. OK? But we're just going to use the basic properties of exponents throughout all this, so we don't-- we haven't even talked about continuity, or derivatives, or anything like that, so we'll just use elementary means to be able to prove these statements. OK? So for the first one, we'll prove this actually just using the definition of the limit, which, remember, means for every epsilon positive there, we should be able to find a capital number M, such that if n is bigger than or equal to capital M, 1 over n to the p is less than epsilon. So let epsilon be positive. Choose M, a natural number, such that M is greater than or equal to-- is greater than 1 over epsilon to the 1 over p. OK? And if n is bigger than or equal to M, 1 over n to the p minus 0, which equals 1 over n to the p-- this is less than or equal to 1 over capital M to the p. So here, again, I'm using-- an elementary fact that we all know is that, if I have a positive power here, then-- and little n is bigger than or equal to capital M then into the p would be bigger than or equal to capital M to the p. We do know that for integer exponents, but just trust me that one can define non-integer exponents, and that inequality remains true as long as the power that you're using is positive. So I have 1 over n to the p is less than or equal to 1 over capital M to the p. And by our choice of capital M, this is less than epsilon. So that proves number one. So number two we'll do-- we really only need to do two cases. One is p. So p equals 1 is fine. This is clear, because then I just get 1 for the whole sequence. So that's one case. Now let me do p bigger than 1. OK. And if p is bigger than 1, the absolute value of p to the 1 over n minus 1-- this is just p to the 1 over n minus 1. And so I want to show that this quantity here goes to 0. OK? So we have an inequality which we proved actually in the second lecture, I think, using induction, but you can actually get from binomial theorem as well. So let me just recall this right here, that we had this inequality that, if x is bigger than or equal to minus 1, then 1 plus x raised to the n is bigger than or equal to 1 plus nx. OK? And so we use this inequality now with x equals p minus 1. So p-- this is equal to 1 plus p to the 1 over n minus 1 raised to the n-th power. I'm sorry. x is p to the 1 over n minus 1, not p minus 1. And now we use this inequality. This is bigger than or equal to this thing times n plus 1. So then I'll go over here. So now subtracting the 1 and dividing by n tells me that p to the 1 over n minus 1 is less than or equal to p minus 1 over n. And as we noted here, since p is bigger than 1, this is bigger than or equal to 0. OK? And now we apply the squeeze theorem. By squeeze theorem, this is just going to 0. p minus 1 divided by n-- that's just a number over n. This goes to 0 as n goes to infinity. And in, fact that's contained in number 1 for p equals 1, and also by our limit facts that we proved from last time, that the limit respects algebraic operations of the real numbers. So by the squeeze theorem, since this converges to 0 and this converges to 0, this converges to 0, which implies that. OK? So that deals with the case p is bigger than 1. To deal with p less than 1, we use the case p bigger than 1, and again, the fact that limits respect algebraic operations. Then we write the limit as p-- ugly looking p-- write that better-- the limit as n goes to infinity up to the 1/n. Now p is less than 1, so 1/p is bigger than 1. This is equal to the limit as n goes to infinity of 1 over 1 over p to the 1/n. And now 1/p is bigger than 1-- raised to the 1/n power converges to 1 by the case we did before. And so 1 over this converges to 1/1 equals 1. OK? OK, so let me just remark that, although we did prove this inequality for-- by induction, it actually follows from the binomial theorem. And we'll use the binomial theorem to get a little bit of a different inequality that we'll use for number 3. So for number 3, we want to prove limit as n goes to infinity of n over 1 over n equals 1. So rather than keep writing n to the 1 over n minus 1, which I want to show convergence to 0 now, I'm going to write xn. So let xn equal n to the 1/n minus 1, which we note is bigger than or equal to 0 for all n. OK? And so my goal I want to show is limit as x as n goes to infinity of x sub n equals 0, because then that proves that this converges to 0. And since this is equal to its absolute value, that means that n to the 1/n converges to 1. And the way we're going to do that is using an inequality we get from the binomial theorem-- and using this trick here. Now, if we look at plus x to the n raised to the n, this is just-- that's just n-- let me move it over a little bit-- plus x to the n raised the n. x to the n-- x sub n is n to the 1/n minus 1. Add 1. I get into n to the 1/n-- raised to n, I get n. Now, by the binomial theorem, this is equal to sum from k equals 0 to n, n choose k. And I have 1 raised to the n minus k power, x to to the n raised of the k. OK? Now, this is a sum of non-negative things because, x sub n is always non-negative. And these coefficients are just quotients of factorials, so they're always non-negative as well. This sum is always bigger than or equal to 1 term from the sum. So it's bigger than or equal to k equals 2 [INAUDIBLE] y2-- you'll see-- x sub n squared. Now, n choose 2-- this is equal to n factorial over 2 factorial n minus 2 factorial x to the n squares x of n squared, which equals n times n minus 1 over 2 x of n squared. All right? Now, I started off with n and I proved it was bigger than or equal to this quantity here. So now I divide what's in front of the x of in and take square roots. So then that implies that, for n in bigger than 1-- because I need to divide by n minus 1-- 0, which is bigger than or equal to n, x sub n is less than or equal to 2 over n minus 1 square root. OK? And now this is just 0, so it converges to 0. This right-hand side is square root of 2 over n minus 1. Now, the limit as n goes to infinity of 2 over n minus 1 is 0. Square root of that also converges to 0. That's a fact we did from the end of last time. So this whole thing converges to 0. So by the squeeze theorem, limit as n goes to infinity of x sub n-- which, remember, this is just n to the 1 over n minus 1, which implies-- OK? And that completes the proof. All right, so now we are going to study a couple of objects related to a bounded sequence. What's the underlying question we're going to try to answer? Whenever something's introduced, you should think of it in terms of, what's the question that was asked that this is trying to answer? So we're now moving on to the topic of limsup and liminf of a sequence. So here's the question. So we've seen sequences that don't necessarily converge, like minus 1 to the n, but-- and we know about subsequences now, where you just pick entries along the sequence. You pick one, move to the right, and pick the next one. Now, if you look at minus 1 to the n, if I just look at the subsequence consisting of picking the odd entries-- the odd numbered entries in the sequence, then I'd just get minus 1 minus 1 minus 1 for my new subsequence. And this converges. It's just constant. Or if I chose the even ones, I'd get 1, 1, 1, 1, 1. And that converges. So this sequence, which is bounded, has a convergent subsequence. Now, not all sequences have convergent subsequences. For example, if you look at the sequence x sub n equals n-- so x sub n is just 1, 2, 3, 4, 5, 6, and so on-- that will not have any convergent subsequences, because any subsequence of that has to be unbounded. OK? That's pretty easy to show. And we know that a convergent sequence is bounded. OK? So what am I getting at? So the question that we're going to ask and try to answer is the following. Does every bounded sequence have a convergent subsequence? OK? And what I was just going on about right there is that we know this is true, for example, for the sequence minus 1 to the n. We also this is true for convergent sequences. Convergent sequences are bounded, so-- and they have convergent subsequences. Just take the subsequence to be the whole sequence to begin with. And we have to throw in this bounded part, because there are sequences that don't have any convergent subsequences. Like I just said, x sub n equals n is an example of an unbounded sequence that doesn't have any convergent subsequences. So the answer to this question, as we'll see, is yes, this is a very-- really a very powerful statement. And this is due to Bolzano and Weierstrass. And I'll restate it in a little bit when we get to the statement of that theorem-- namely, that every bounded sequence does have a convergent subsequence. There are several different ways to prove this theorem. We're going to prove it by introducing limsup and liminf, because these are also two important objects that arise in analysis. So let's get to the definition of these guys. So let xn be bounded sequence. And we define-- if they exist, there are going to be certain limits, so it's not clear that they exist at all to begin with. But we'll show that they always do exist. We define limsup x sub n. And sometimes I'll write n goes to infinity underneath. Sometimes I'll just write limsup or I'll just have an n underneath it. This is supposed to be a number, and this is equal to the limit as n goes to infinity of a new sequence obtained from the old sequence x sub n. What are the entries of this new sequence? This is sup of x of k, k bigger than or equal to n. OK? And the liminf is similar, except it's now with infs. OK? So for each natural number n, I take the supremum of the set of elements x sub k for k bigger than or equal to n. So this is a bounded set because the sequence is bounded. So this the supremum is well-defined-- same thing for the inf. OK? So I get a number, a new number for each n. OK? And I take the limit of those numbers and I define one to be the limsup, one to be the liminf, if they exist-- because limits don't always exist, so it's not even clear that these two limits actually are meaningful. And the first thing that we'll do is we'll prove that these limits do actually always exist. So rather than keep writing this, I'm going to give these-- write some symbols for these two things. So let a sub n to be-- so of course, if I forget to say this at least in what I'm talking about with respect to limsup and liminf, I'm always talking about a bound sequence. But let me continue to state that as one of my hypotheses. Let x of n be a bounded sequence, and let a sub n be the supremum of the set of all elements x sub k, where k is bigger than or equal to n, and b to be the infimum of x sub k, a bigger than or equal to n. OK? So then there's couple of statements. One is that the sequence a sub n-- so let me just put over here what these things have to do with the limsup and liminf. Then the limsup of x sub n-- this is defined to be the limit as n goes to infinity of the a sub n's. And the liminf of x sub n-- this is defined to be the limit as n goes to infinity of the b sub n's. OK? What we're going to show is that the limit of a sub n as n goes to infinity of a sub n exists, and the limit as n goes to infinity of b sub n exists. So how we're going to show that is we're going to show that a sub n is monotone decreasing and bounded. And so these are the conclusions. So [INAUDIBLE] should say then. The sequence b sub n is monotone increasing and bounded. So in particular, since we know that, if we have a monotone sequence which is bounded, it has to converge, that means these two limits exist. And then the second part of this term is the simple statement that the liminf of x sub n is less than or equal to the limsup [INAUDIBLE] x sub n. OK? OK, to prove one-- so in fact, before I prove this theorem, let me prove a small simple theorem first. Let's put the proof of this theorem on hold just for a second, and now prove a very simple theorem that, if A and B are subsets of real numbers, A, B, both not equal to the empty set, and is A subset of B-- so also need to be bounded. So if we take two non-empty subsets of real numbers, such that [INAUDIBLE] bounded and A is a subset of B, and the conclusion is that the inf of B is less than or equal to the inf of A. And this is always less than or equal to the sup of A, and this is less than or equal to the sup of B. So what this says is that, if I take a subset of B, then that increases the inf and decreases the sup. OK? So the sup of a smaller set is smaller than the sup of the bigger set. And that inequality reverses for infs. The inf of the smaller set is bigger than or equal to the inf of the bigger set. All right? And this just follows immediately from the definition of sup and inf, so I'll just prove the sup statement. So since sup of B is an upper bound for B, and A is a subset of B, this implies that sup B is an upper bound for A. Sup B sits above everything and B. A's a subset of B, so it sits above everything for A. Now, the supremum of A is supposed to be the least upper bound, and therefore, if I take any upper bound for A, that has to be bigger than or equal to the sup of A. So since sup B is an upper bound for A, this implies that sup A is less than or equal to sup B-- and similarly with the infs, so I'm not going to write-- so similar for infs. OK, so let's go back to the proof of this theorem here, that if I have a bounded sequence, then the limsup and the liminf exist. And we show that by proving that these two sequences have these monotonicity properties. So proof now of the theorem we were-- started off proving-- OK. Since the set of xk's, with k bigger than or equal to n plus 1, is a subset of xk's for k bigger than or equal to n-- because now I'm-- I have a set where k's starting at n plus 1. Here's a set with k starting at n, so this is clearly contained in here. This implies that a sub n plus 1, which is the sup of the left hand side, by this little theorem that I stated here, is less than or equal to the sup of the bigger set. And this is just a sub n. So we've proven, for all n, a sub n plus 1 is less than or equal to a sub n. OK. Therefore, this sequence is monotone decreasing. OK? And so this uses the sup part of this previous theorem, but if we use the inf part, then we also get the statement for the b sub n. So rather than write out the details, I'll leave it to you just to flip around the inequalities for the inf in your notes. So similarly, for all N, a natural number, b sub n is bigger than or equal to b sub n plus 1-- or other way. OK? Now, so that shows these two sequences are monotone. Now we show they're bounded. And this follows simply from the fact that the x sub n's are bounded. Since there exist a b bigger than or equal to 0, such that all N natural numbers, x sub n in absolute value is less than or equal to b, which is the same as saying b is-- x sub n is bounded between minus B and capital B. So taken as elements of each of these sets, if you like this x of k, k bigger than or equal to n, that means minus B is always a lower bound for these sets and B is always an upper bound for these sets. So this implies that minus B is always less than or equal to x sub k, k bigger than or equal to n. And this always sits below the supremum. Since all of these are bounded above by B, the supremum has to be less than or equal to B, which I'll state in terms of the a sub n's and b sub n's-- means minus B is less than or equal to b sub n is less than or equal to a sub n is less than or equal to b. So these sequences are bounded. OK? So in fact, these two implies that-- all right, so we've shown that these two sequences that define-- that we use to define the limsup and the liminf are monotone and bounded, so therefore, the limit of these two sequences-- the limits, which define the limsup and liminf, actually exist. So limsup and liminf is always a well-defined object. Now, this proves one. To prove two, this follows immediately from what we've proven right here. So by this part, we have b sub n is less than or equal to a sub n. So for all n, I have these two sequences, one sitting below the other. And last time we proved that taking the limit respects inequality, so limit as n goes to infinity of b sub n-- which is the liminf-- sits below limit as n goes to infinity of a sub n, which is the limsup. And that completes the proof. So I've shown that these two objects we defined-- the limsup and the liminf-- exist, and are well-defined for every bounded sequence. So this is kind of-- can be a little bit of a daunting couple of objects to come across when you-- especially in your first analysis class. So the best thing to do is look at examples. Whenever you come across something that you just don't quite understand, start writing down some actual things. So for example, let's again look at our favorite example of a bounded sequence which does not converge, x sub n equals minus 1 to the n. Then, if I am looking at this set x sub n, n bigger than or equal to k, and writing-- instead of x sub n, let me just write what it is, minus 1 to the n. So what is this set? This is just a set consisting of two elements, 1 and minus 1. Yeah? And therefore, the sup of this set, which is the sup of this set, is just 1. Oops-- so if I take-- which implies that the limsup of minus 1 to the n, which is the limit as n goes to infinity of sup minus 1 to the n, n bigger than or equal to the k-- as we just saw, this is just the sup of this set consisting of two elements, 1 and minus 1. This is equal to the limit as n goes to infinity of 1 equals 1. So the limsup of minus 1 to the n is 1. Now, if I change all these sups to infs, then the inf of this set is going to be the inf of this set, which is just minus 1. And therefore, we also get-- OK? So the limsup is 1. The liminf is minus 1 for this set. That's just supposed to be a squiggly line, not necessarily looking like sigma. OK, so there's one sequence. How about our next favorite sequence, x sub n equals 1/n? So the set of elements 1/n, such that n is bigger than or equal-- so 1/k I should write. Oh, so I was using some-- this should have been a k. Hope didn't make that mistake throughout. No, I kept writing x sub k. So that should be x sub k. OK, all of that is written correctly. This should have been minus 1 to the k-- minus 1 to the k. All right, very good. So we're looking at now the set 1 over k, where n is-- where k is bigger than or equal to n. So this is just a set 1/n, 1/n plus 1, 1/n plus 2, 1/n plus 3, and so on. OK? So as I move to the next entry, things are getting smaller and smaller. And in fact, this sequence just here now, written as-- thinking of this as a new-- so this is not a sequence-- this is a set. Taking entries to be these guys, it's easy to see that converges to 0. So anyways, let's say I take the supremum of this set, which is what I need to compute the limsup. So I'm now taking the supremum of this set. 1/n plus 1-- that's always smaller than 1/n, and so is 1/n plus 2, and so on and so on. And 1/n is an element of the set that's bigger than or equal to everything else in the set. And I think there's an exercise in the homework that-- as you just show, that if you have a set that contains an element which is an upper bound for the set, then that has to be the supremum. So the supremum of this set is simply 1/n. OK? So this supremum is equal to 1/n. So as n goes to infinity, the limit of the supremum here, which equals the limit as n goes to infinity of 1/n, equals 0. OK? So the limsup of 1/n equals 0. OK? But now let's say I look at infs of this set. So I had to take sups to look at the limsup. Now let's say I take infs. OK? Now, the inf of this set-- these are elements that are getting closer and closer and closer to 0. So there's 0, 1/n, and then they just keep getting smaller and smaller and smaller and smaller, converging to 0. And you can, in fact, prove this rigorously if you'd like, but I think it's easy to at least convince yourself that infimum of this set is equal to 0. The smallest thing-- so first off, 0 is a lower bound for this set. And if I take anything bigger than 0, that thing cannot be an upper bound simply because of, if you like the Archimedean property, I can always find something from the set less than that real-- positive real number. So 0 must be the least upper bound. So in summary, the liminf of 1/n, which is the inf-- the limit as n goes to infinity of the inf of this set, is just the limit as n goes to infinity of 0 equals 0, all right. So the limsup of 1/n equals 0. The liminf equals 0 as well. OK. So let's take a look at these two examples a little bit, and let me just make a few remarks. First off, what's to notice about this sequence is the fact that the limsup is 1. Liminf is equal to minus 1. So the limsup does not equal the liminf. However, in this example, the limsup, which is 0, equals the liminf, which is 0. Now, what's the difference between these two sequences? What's the property that one holds, but the-- one has, but the other doesn't? This is a convergent sequence and this one is not. And we'll see this is a general fact, that if we have a convergent sequence, then the limsup and the liminf equal each other and are equal to the limit of the original sequence, because this sequence converges to 0. But that's not just a one-way street. It's a two-way street that, in fact, we'll prove, if the limsup equals the liminf, then the original sequence converges. So the sequence converges if and only if the limsup equals the liminf. And we saw here that in-- on display, that the limsup and the liminf don't equal each other, and the original sequence, which we know-- or shown last time-- doesn't converge. And so one other thing I'd like to point out is that-- so the limsup of this guy is 1. The liminf of this guy is a minus 1. Now, we can also find a subsequence which converges to 1, which is the limsup. We just take the entries that we choose to be the even numbered entries, and that just gives-- produces a sequence 1. And that sequence converges to 1. If we take the odd entries of the sequence, that subsequence is just minus 1 and converges to minus 1, which is the liminf. And that's also a general fact will prove, that for any bounded sequence, there exists some sequences converging to the limsup and the liminf. And that will give us the proof of this Bolzano-Weierstrass theorem. All right. And I think that's all of the remarks I want to say. And let me make one other important comment. So this sequence, which we use to define the limsup and also the liminf-- in three out of these four cases, they were actual subsequences. So the sup of this set equals 1, which I can consider as a subsequence of the original guy-- and the same thing for the liminf. The inf of this set was minus 1, which I can consider as a subsequence of minus 1 to the n. And for the sup of this set, I got 1/n, which I can-- this is just equal to the original sequence. Definitely, I can consider it as a subsequence of the original sequence. But if I take the infimum of this set, which I need to define the liminf, I got 0 for every entry, which is not a subsequence of this original sequence. OK? So all of that is to say that the sequences I get through this process to define the limsup and liminf are not necessarily subsequences of the original sequence. OK? What I just said a minute ago about there actually being some sequences which do converge to the limsup and liminf-- this is a non-trivial fact, which we're going to prove. OK. So theorem-- and this will give us the Bolzano-Weierstrass essentially immediately right after it-- let xn be a bounded sequence. Then there exists subsequences xn sub k and xm sub k-- so they don't necessarily have to be the same subsequence-- such that limit as k goes to infinity of x sub n sub k equals the limsup-- so it's a convergent subsequence-- and the limit as k goes to infinity of xm sub k produces the liminf. OK. And so before I prove this, this immediately gives the Bolzano-Weierstrass theorem, which is that every bounded sequence has a convergence subsequence. OK. So again, this follows immediately from the previous theorem, because if I take a bounded sequence, by the previous theorem, I can find a subsequence which converges to the limsup, which always exists for a bounded sequence. OK? And then that's [INAUDIBLE]. In fact, we have something stronger, in that we have at least two subsequences which converge to these two numbers, which may or may not be the same. OK? So the reason this is so powerful and so strong is that it-- to get your hands on something, it doesn't require you to show something as strong as showing there is a sequence converging to that. So quite often, you can think in terms of variational problems, where you want to show that a minimum of something always exists or a maximum of something always exists. Well, what you can try and do is take a sequence of guys that you stick into your-- so this is a general nonsense that you stick into your machine or function that spits out output. And these outputs are approaching the maximum or approaching the minimum. And what you'd like to say is that there does, in fact, exist an element that you can stick into your machine and produce the maximum amount of output. Now, maybe it's not clear how to do that. So first, take a sequence approaching-- so that the values are approaching that-- the outputs are approaching the maximum. Maybe you could show that the inputs converge to something, but that's typically really hard. But you don't have to work that hard is what this theorem says. It says that what you really need to do, and which is much more straightforward, or simpler, or impossible really, is to show that that sequence of inputs that you put into your machine to get the outputs is a bounded sequence. OK? Then you could pass to a subsequence, which actually does converge to something by this theorem, and proceed in that way by showing that you do have some minimum or maximum-- some input that produces a maximum output or minimum output. So that's a bit small bit of rambling about why this theorem is so useful is that, again, it's-- to get your hands on something that typically you want to study, it's very difficult to show convergence to that thing you want to study, or that thing exists, because there's a sequence that you come up with ad hoc actually converges to that thing. But this theorem says you don't need to work that hard or try to do the impossible. And typically, it's much easier just to proceed by showing your sequence of inputs is bounded. OK, so that's enough of that bit of rambling about why this theorem is so useful. It's also useful in the study of PDEs, which is what I study. So I have a soft spot for it. Actually, one of its generalizations-- all right, so let's prove this theorem that there exists some sequences which converge to the limsup and liminf. So as before, I'm going to use-- rather than keep writing the supremum of this set and-- actually, I'm not even going to do this statement. I'm going to leave this as an exercise. I'm going to do this statement. I'm going to use this notation a sub n, as before-- a sub n b sup of x sub k's, k bigger than or equal to n. So I want to show that there's something converging to the limit as n goes to infinity of the a sub n's. And so what I'm going to do is I'm going to try up a subsequence of elements between a sub n and a sub n minus something which is converging to 0, and not quite a sub n. So it'll be along a subsequence as well. So we know that there exists an n sub 1 bigger than or equal to 1 simply by how this is defined as a supremum, and by the exercise from assignment 3 that there exists an n sub 1 bigger than or equal to 1 such that a sub 1 minus 1 is less than or equal to is less than x sub 1 is less than or equal to a sub 1. OK? All right. a sub 1 minus 1 is not an upper bound for the set a sub 1, which is x sub k's, where k's bigger than or equal to 1. Therefore, I should be able to find an element from this set strictly bigger than that. And of course, it's always less than or equal to the supremum, which is a sub 1. OK. So now, since a sub n sub 1 plus 1-- so a sub n sub 1 plus 1, not 1 plus 1-- since this thing equals the supremum of xk, such-- k is bigger than or equal to of 1 plus 1, there exists an a sub n sub 2-- at least there exists an n sub 2 bigger than n sub 1-- in fact, it has to be bigger than or equal to n sub 1 plus 1-- such that a sub n plus 1 minus 1/2 is less than a sub n sub 2 is less than or equal to a sub n plus 1. OK? So why the n sub 1 plus 1 is because I wanted to obtain a new entry from the sequence that comes from farther than the index n sub 1. OK? I'm trying to build a subsequence. And like I said, the idea is I want to somehow sandwich-- this should be x sub n-- I want to build up a subsequence which is sandwiched between things converging to the limsup, and use the squeeze theorem. All right? OK, so since this is the supremum of this set that-- and because this thing is not an upper bound for this set, there exists something from the set-- so some element k bigger than or equal to n sub 1 plus 1, which I'm going to call n sub 2, so that x sub n sub 2 is bigger than a sub n plus 1 and a sub n sub 1 plus 1. And then I just keep doing this. Since a sub 2 plus 1 equals the supremum of the x of k, such that k is bigger than or equal to n sub 2 plus 1, there exists an n sub 3 bigger than n sub 2, such that a sub n sub 2 plus 1, minus now a third, is less than x sub n sub 3 is less than or equal to a sub n sub 2 plus 1. OK. But now we're essentially home free. We'll just continue in this manner. Let me write down. Now, strictly speaking, I need to state the construction of this sequence as an inductive argument, but for the purposes of this class, I'm not going to do that. I'm just going to say-- continuing in this manner. And that will be what I say for this part. So continuing in this manner, we obtain a sequence of integers, natural numbers-- n sub 1 less than n sub 2 less than n sub 3, so on-- such that what holds-- such that, for all k [? N, ?] natural number, a sub n sub k minus 1 plus 1 minus 1/k is less than or equal to x n sub k less than or equal to a sub n sub k minus 1 plus 1. OK? And here, if you like, we didn't define what n sub 0 is, so-- so really, we're only interested in the n sub 1, n sub 2. But for the sake of this whole thing making sense for all integers k, with n sub 0 being defined to be 0-- OK? That's just the first case that we dealt with. So we obtain this subsequence x sub n sub k that's sandwiched in between this subsequence of a's and minus 1 over k, and this sequence of a sub n sub k minus 1. So since n sub 1 is less than n sub 2 and so on, this implies that-- write it this way-- n plus 1 is less than [INAUDIBLE] 2 plus 1 is less than n sub 3 plus 1. So this is a subsequence-- a sub n sub k minus 1 plus 1 is a subsequence of a sub n's. Now, what do we know" We know the a sub n's converge to the limsup. And we proved last time that every subsequence of a convergent sequence converges to the same thing. That's the limit as k goes to infinity of a sub n sub k minus 1 plus 1 equals the limit of the original sequence, which is, by definition, the limsup. OK? So now I have this subsequence. So now I have this subsequence of x's sandwiched in between two sequences-- this guy on the left, this guy on the right. This guy on the right converges to the limsup. This guy on the left converges to the limsup minus 0, because 1/k converges to 0. So by the squeeze theorem, we get that the limit as k goes to infinity of x sub n sub k equals the limit as k goes to infinity of this and this whole thing, which is the limsup. OK. And again, so I'll leave it to you to do the liminf part. But the point is that now, basically, this 1/k gets moved to over here, and this becomes a sub n sub k minus 1 plus 1 plus 1/k. And now I just have this on sitting below this guy. All right? That's really the only change for the infs to get a subsequence converging to the liminf. OK? Now, we've shown that there exists a subsequence converging to the limsup and the liminf, which gives us the Bolzano-Weierstrass theorem. And now let me come back to the statement I made about these two sequences here-- namely, that the limsup equals the liminf if and only if the original sequence converges. So let's prove that now. Let x sub n be a bounded sequence. Then xn converges if and only if limsup equals the liminf. And there's one more part. Moreover, if xn converges, then all these limits agree-- equals the limsup, equals liminf. OK? So the sequence converges if and only if the limsup equals the liminf-- and in the case that we do have this limit of the sequences given by this common value of the limsup and liminf. OK, so let's go in one direction. To do this direction, we'll use the squeeze theorem. So suppose L equals limsup x sub n equals liminf of x sub n. We're assuming that these two things equal each other. They're given by a common value L. And what we're going to end up showing is that this sequence converges to L. So that actually gives the second part of the statement of this theorem here. So suppose L is this common number, limsup and liminf. Then, for all N, [? a ?] natural number, we have that the inf of the x sub k's, which is for k bigger than or equal to n-- so x sub n is in this set, so it's certainly bigger than or equal to this inf. And it's less than or equal to the sup of that set, again, because x sub n sits in this set. All right? But as n goes to infinity, this sequence of numbers converges to the liminf, which is L. As n goes to infinity, this sequence of numbers converges to the limsup, which is L. So by the squeeze theorem, the thing in between, which is x sub n, converges to L. So for the second part, it's-- follows from what we've proven-- what we proved to obtain the Bolzano-Weierstrass and what we know about convergent sequences and their subsequences. So this is for this direction-- namely, that convergence implies the limsup equals the liminf. Let L be the limit as n goes to infinity of x sub n. So now we're assuming that the sequence converges to something which we call L-- doesn't have to be the same L from-- so it's just L I'm just using for this limit. By previous theorem, there exists a subsequence x sub n sub k, such that the limit as k goes to infinity of x sub n sub k gives me limsup of x sub n. But this is a subsequence of a convergent sequence, and a subsequence of a convergent sequence is convergent, and converges to the same thing. So this thing on the left is just equal to L. OK? Similarly, there exists a subsequence x sub n sub k, such that the limit as k goes to infinity of x sub n sub k equals liminf of x sub n. And again, this is a subsequence of a convergent sequence, so it's convergent and convergence of the same thing, L. So that implies that this thing is equal to L. All right. And therefore, the limsup and the liminf equal each other, and they also equal the limit of the original sequence. So that is the end of the proof of this theorem that the limsup and liminf-- when they coincide and tell you that the original sequence converges. So that's another way of thinking about limsups and liminfs is that they also somehow measure just how divergent your sequence is, or at least the difference between them. If that difference is 0, then your original sequence is convergent. OK? All right, so I think we'll stop there.