Lec 32 MIT 18.085 Computational Science and Engineering I

the somehow seems pretty appropriate to finish uh this series with a little well a little bit of serious fun I guess you could say um and uh I'm going to pick because it's the one topic Within game theory that's manageable and beautiful two person games so there are two opponents no coalitions if we have seven a seveners game then one group can form a team and go against another group and and uh it becomes very complicated and also the game will be Zero Sum whatever the two players we might as well call them R for the person who's going to choose a row and C for the person who's going to choose a column so I have a matrix I start with the the payoff Matrix let me let me take an example example that's 2x two just to explain the rules so maybe the entries are 1 2 4 and 7 okay so R the person who all the this R chooses a row C chooses a column that identifies a number and that's the payoff to R so whatever uh so so if so R chooses row I C chooses in the best way possible column J column J and R gets that amount AI J that entry from C well this game is certain stacked against C right all all payoffs are positive in this case no reason if I subtracted a 100 from all the entries what would be different well it'd be stacked against R all if all the entries were negative but the strategy wouldn't change I mean if if it was 100 subtracted from all entries then it would be like the same game and then and then uh see would get 100 each time so it's the strategy that we're looking we're looking for strategies looking for for the best strategies best optimal strategies that means which row should R choose well that's clear in this case what's R's reasoning and which row does he choose row two row two and the reasoning is is whatever C does I'm ahead right if C chooses column one I'm ahead if C chooses column two I'm ahead okay and and now what about what's C's reasoning what does c choose C chooses column one with the same reasoning that whatever happens whatever R is doing um it's better to be in column one okay so that's um that's an easy case the best strategies are completely clear they're what I would call Pure strategies so this this is a case where the best strategies are pure now let me take let me just change a couple of numbers here put the four here and the two and the well let me change the numbers around a little more all right that's a more reasonable G game now what now now what think about the strategy so we know the rules of the game what should R do well not clear right not clear because if R so what will happen if r likes the idea of winning seven right so if R chooses this row what will C do c will notice we're thinking of this game as being played repeatedly independent plays every play is independent but the the players know what's happened from their the their decreasing of or increasing pile of uh income okay so yes I'm sorry I don't get it so our chooses saw no no they sorry I should have said no no no they choose at the same time they choose at the same time would C always choose column one oh did I not improve it yeah see what yeah so what all right let's follow this example then which you're right is is is a relatively easier one C would always choose column one on the basis that whatever R whatever R did two is better than four and one is better than seven good and then what would R do thinking this through R would say look C is he he he doesn't know what c is going to do but he he can think it through he can think the same thoughts that c thinks so he he realizes that c is going to choose column one so R is therefore going going to choose Row one all the time and we'll be there we'll be there yes right so there's another instance less obvious but oh yeah no actually that that one yes that that that's pretty clear okay now can you help me make it harder switch the two and the four all right switch the two and the four okay all right now let's put ourselves in the place of r r says well what what would you do if you were R let me just ask you if you were R facing this game what what row would you choose well say either one yes one say you would say choose Row one okay well if you kept choosing Row one over and over I mean if that was your strategy then C would choose because it look would look C would look only at that row C would choose column two over and over but then what would you do if you were R hey we got this guy nailed right he's choosing column two I'll just slip in uh row two and get the seven right the but that Happy Feeling won't last very long because as soon as he says wait C he says wait a minute now he switched to column two and I'm paying out seven I'll switch to row one so then he switches to row one all right and then our notices okay he's on Row one I'll go on column one C would switch to column one and not pay off seven then R would go back to row one and pick the four and then at some and then he would then C would go back to row to column two and be paying off two and then it would go on right so so here we're really at the critical moment in this lecture uh right away what what could be an optimal strategy or what of of I'm not asking for the optimal strategy precise answer but but you've got to have some be some randomization some randomization weighted dice yeah weighted dice that's exactly right each each player ends up in this as soon as we take a a case with the numbers a little little like this which would be the typical case then um the the best strategy is you you just you cannot tell that other person uh you can't give that person any information about a particular particular play of the game because if you pass out information about a particular play of the game then the your opponent will use it so the best you can do is and you can do it in front of your opponent is flip a coin weighted coin so that so that R will choose these in proportions X and 1 - x so our's Choice amounts to selecting the fraction X of the time of the number of plays that he that that he'll choose Row one and the remaining fraction row two but randomly and C similarly has to randomize or or is going to pay the price so c will choose some X and Y are between 0 and 1 so that you know this is some fraction like a third and 2/3 of the time I don't know if that's right this is some fraction like two fifths and three fifths of the time well that doesn't sound right that would come to seven quite a lot so anyway that that's the idea so the best strategies are what I would call mixed strategies mixed meaning you mix up the number of rows the the choices you mix your choices in a random way and of course I don't know I mean every poker player would let's just think about other games for the moment um does this in if we think of Poker does this fit the way you would do poker or blackjack or I don't know what's your favorite game would be in poker well let's just take poker well so there many many more strategies the trouble with the trouble with most realistic games is the number of pure strategies see this could have been a 3X3 Matrix if if the if if the or it could have been a 3x5 Matrix we could have given R three choices and C five choices and in poker of course uh the number of different strategies would be uh overwhelming because we have to have a strategy for each situation but generally speaking it poker is like this and that you might Bluff so Bluff would be uh yeah Bluff would be maybe in the case of C going this way uh you know and taking a chance that uh um that uh yeah anyway that taking a chance that for for some fraction of the time uh R will be uh making this choice and you'll only have to pay two so I I want to know X and Y and the average payoff so what's the what's the average payoff yeah maybe I call these I better call these X1 and X2 because there could have been seven of them and I'll call these y1 and Y2 so the point is of course that the sum of the X's is one that all the X's are of course greater or equal zero and they add to one they probabilities of choosing different rows and similarly the Y's are all non- negative and they add to one that's that's clear okay but now what is the payoff if if I've decided on playing these fractions the rows and C has decided on those fractions of the columns what's the well how often does a four get chosen how how often is the payoff four1 X1 y1 because these are independent decisions so the payoff the payoff would be four X1 y1 that would that would have account for the fraction of the time the fraction of the time that that that we have a choice of X1 and y1 times the four that you pay and then what are the other terms well there's a two from X1 right the payoff of two occurs in the fraction X1 Y2 and the payoff of one occurs in the fraction X2 y1 and the payoff of seven occurs in the fraction X2 Y2 and I'd like to have a nice linear nice Matrix form for that uh and I think it would be just uh y transpose ax can I can I write that y That's y1 Y2 multiplying the a 4217 oh no I've got to get the it's X transpose a y isn't it because I've got to get X choosing rows these have to multiply the rows and these have to multiply The Columns is that right that if I do that multiplication X1 will multiply the four and the y1 the X1 will multiply the two and the Y2 I'll get the four terms and that's the I'm just getting a nice notation here so what is the job of the job of R the person who's choosing the rows well he's the person who's getting the payoff so he's trying to maximize the payoff so maximize this payoff X transpose a y wait chw choose X chw CH let me say it in words or use more words here choose his he's he's choosing his his row fractions to maximize the payoff X transpose a y okay but now and of course he has the constraint that his his fractions should add to one that's clear but what's his what's really constraining him here what is it he's what is he actually he has to when he chooses es an X what will y do because after a year or two why is going to notice what those fractions are I mean now we have to think about the big picture again we we've decided that you have to decide on fractions and randomize right but after a, 2,000 a million plays that that X is going to be known to why he's going to see pretty closely what the fractions are so this will be known to why so what will if that once that's known to why what would y then do the C then do the choice of columns he would minimize yeah so why is going to minimize so so um he has to maximize this but but um but has to recognize that why is let me show you what's up here it one of them is trying to maximize over all the X's this is the minimum over the wise of the payoff and the other is trying to save money he's trying to minimize overall the yse but has to realize that whatever Choice he makes X is going to take the maximum to to to those are the two calculations this is the calculation that R has to make this is the calculation that c has to make so let's it's it's dual so we could just think about R so R has to think okay I'm going to take the best X but it has to be best against the worst y the worst from his point Sorry the worst from his point the worst from his point of view that's right the worst from his point of view okay now if I was strong enough we could work this out I don't know if let's see um can we can we try to see if we could do it this is this is the payoff written up in full Okay so what basically what does X have to do he has to present y with a with a he wants to present y with with with two columns where why why has no reason to go one way or the other I I I I think in this case if see what will y usually get when when when y yeah think about this way suppose y chooses column one what can he expect the payoff to be on on the average after zillions of plays 4x1 + 1x2 I think is that right if we're in y's position we we why the payoff to Y in choosing column one would be 4x1 plus 1 X2 that's the payoff from column 1 the payoff from column two is 2x1 and 7x2 and I think X wants to adjust those so that those are the same I think it seems right to me that if provided this gives uh non non non- negative X's so what does that tell us that tells us that 2x1 is 6x2 is that what we get out of this this leads to subtracting the 2x1 that gives us 2 x1's is 6 x2s it looks to me like this ought to be uh the bigger one three quars and this should only be one quarter I think that's the conclusion basically gives why no opportunity to change no there's no reason for why to change because if he if he didn't do it that way why will change so then the average payoff when X1 is a is 34 and X2 is one quar is 34 what do I get then for the for those numbers just one quarter and that's makes one no three 3/4 times that's three I think we get three and a quarter and I believe that's the value of the game you can say wait a minute I haven't figured out why strategy yet and that's true let's see so let's figure out why's strategy and the same reasoning okay so y strategy will be if x if R if if Row one is chosen the the average payoff is 4 y1 + 2 Y2 whereas if the second row is chosen the average payoff will be y1 + 7 Y2 and if he makes those equal then R will have no reason to to choose one row over another so what would that give us that'll give us three that leads to 3 y1 is 5 y2s so they're in the ratio of 5 to three so maybe 38 and 58 and 38 58 and 38 because then 3 * 58 does give 5 * 38 now I think we've got it so there's a particular game that's solved that's a particular game that solved um and and so the the first and most important point is is is the point we saw earlier that in in this game a mix strategy is the best a randomized mixed strategy if there's any pattern the opponent will take advantage here there's no no pattern except the the only pattern is this is 34s of the time and this is 1/4 of the time and then uh and then if you plug those numbers into the payoff you would get three and a quar I'm sure we could we yes um this business of being best to pick a point where y doesn't want to change yes is it is it clear why why that is is that because there's some quadratic somewhere that I guess there is somehow y this is that's probably it so this this is a Remar what the end result is is the fact that the best X the best x x can do is to guarantee himself this amount and the best why can do is to guarantee to not pay more than that amount cuz he's he's making that as small as he can against anything X can do and the beauty is this Minimax theorem that this Minimax equals the Maxim in that that's that's the duality of course uh that the two players are somehow locked in this dual picture and uh that um that that uh is um that that that holds now let me I I don't want to so this was a case where a mixed strategy was right so that the for both of them let's go back to the what what was an easy case at the start and and and just think it through again so again R and c and let me what did I have then so something like say 1247 and then what were the best strategies for this game this was the easy game remind me what what should R do in this game to row to all the time so his fractions should be zero and one and what should C do all the time choose column one so his fractions should be one and zero so this is y one should be one and X2 should be one so there's a case in which because we were bumping up against the limits the 01 limits we didn't we didn't find a mixed strategy this this was this is like the key with these inequality constraints either you're kind of in the middle in where you're in the middle and you could go either way a little more in column two a little more in column one when you're there it's like being in the middle of a function say like this it's like being like minimizing when you're in the middle it's you could go either way and a derivative has to be zero but if we were maximizing that function what o over this interval what would be the situation the maximum would be at an end and the derivative isn't zero there it's like you have an inequality that can only be a winner for the for the guy who's maximizing if uh the derivative is positive because then you wouldn't want to come you wouldn't want to come in which is what the only Direction you you would be allowed to move this this Max so we're talking about maximum minimum optimization with constraints that that if the constraints are active if you're at the edge then the constraint is an inequality constraint because there's only one way you can move where if you're at the center the constraint is an equality constraint some derivative has to be zero because you could move both ways so this was a case where you could move both ways so e equalities had to hold we had equations like this we made we we made it so that c had no reason to prefer one or the other because he could move both ways here we we're facing C with a choice um where he's losing four all the time and yet sort of we're at the edge uh if he he would like to to choose this even less but or even sort of well anyway we're we're at a one zero situation for both of them in fact so that do you see what I'm saying that that R is not presenting C with a equal choice if if R is keeps choosing this this row then C's choice is to lose four or seven so he always picks losing four there was a case there where there was an inequality if four was less than seven therefore all the marbles went on to column one this was a case where in either direction it didn't matter and we ended up with mixed strategies maybe that's the uh the theme of um of uh optimization with inequality constraints and maybe I express that now in a in a language that's not Game Theory but but uh typical problem of minimizing some cost function subject to some inequality constraints so now I'm really in these last minutes speaking about optimization as we really know it um we're minimizing something that's optimization there are constraints they can be nonlinear that can be nonlinear and this uh and they can be inequalities or equations so that's there is like a general optimization problem and how would you handle such a problem that's not an easy problem because you don't know if you knew you see if if you knew so the best X I'm looking for the best X xstar the winning X we're in any number of Dimensions here and we have any number of inequalities then if we knew that the best X if we knew where it was hitting the boundary if we knew in which components a of X star was equal to the corresponding B and in which components it was less we would we would be golden because then we would have equations do you see unfortunately we don't know that so so X is n n different numbers so we've got n numbers to choose and B is M different numbers so we've got M inequality constraints and the the uh my my point is if we knew which where the equalities were then those would give us some equations for xstar and then the and we knew the other ones were inequalities then we would know some derivatives in those components were zero because we would be in in those components we would be able to go left or right and therefore a derivative with respect to that would have to be zero we would have a set of equations we could solve but we don't know which which constraints are active and which are not that's what it comes to an active constraint is one where where uh where you're pushing at the where equality holds you're you're right up in my picture here there is an active constraint that the X can't go beyond that value that's forcing us to be there and if we KN once if we knew that then we could just look there if we knew it didn't hold we could look you know in the middle maybe whatever so it's but it's a tough problem because we don't know and we cannot check all possibilities the number of possibilities is too great so you have to introduce th this trick of introducing the gr multipliers is the brilliant idea so the gr multiplier you you uh the gr thing then you use you you bring in some multipliers why you do that by taking the cost and adding to it these y's times the constraints and the nice way to write that is y transpose B minus a of X so somehow yeah I'm I'm I'm not going to be able to describe like's multipliers in one uh in one go here uh but the the trick is to bring in these new variables that seem very mysterious and they have to be great or equal Z and you look for a maximum of or you look for a minimum with respect to X and a maximum with respect to Y so it actually is a is a nonlinear type of game theory so in the end you look for the the minimum over all the X's because that's the problem you finally wanted to solve but then there's a maximum over all the Y's and you get this uh this function to work with uh I I think probably what I need to say about optimization is what I've said already that it's a subtle type of mathematics the algorithms have to play back and forth between the reducing the cost and satisfying the constraints and uh I think you wouldn't want to write your own software to do this I mean this is this is for Professionals in a in that field of optimization to uh create uh software to uh solve problems maybe maybe you maybe it's worth saying that uh that a major activity in numerical analysis over the last say generation has been to create software that um that everybody that that could be used could be widely used that's reliable efficient um stable algorithms and produce it I mean so the the packages that so uh uh linpac would have been a a typical package of software that was for solving linear equations and then you would have a special code if the Matrix was symmetric you would have a special even more special code if the Matrix was positive definite you'd have a special code if it was tridiagonal you would just have a it's a suite of linear solvers to handle different cases and ice pack which actually came first uh is for the igen value problem again dealing with symmetric or un symmetric matrices all these cases and now in this problem in this case there will be a package called minpack and you can see the human invention uh so everybody who uh if if you want to do conformal mapping well look for a package that uh that does it if you want to do uh if you want to use parallel uh this this is now replaced by La pack as a upgrade and then replaced further by a parallel package if if you had uh or if you were wanted a software that did it in parallel and maybe I'm saying what you all know just so mat lab would mat lab would use that particular package so mat lab ma the math Works people are not the create the original creators of these packages and and the really nice thing about this area of numerical linear algebra is that these packages have been created by people and put out for free H so meth works is free to use to incorporate that package in Matlab then of course Matlab is not free far from free um um the the the one source of free ones is there's a there's a email uh probably a web page also net that's sort of net Library often library is the word for a whole whole collection of packages net libe was maintained I I think it's uh um Oakridge nationall la.gov you you might just you and there would be a web page now which where originally it would have been you would ask you would send a message like send uh uh send index or something would be the subject of a message to to netlive and you'd get a list of of you know organized list of of some of the things that are available so that would be one source and then there are commercial sources also uh but there are these I would always look first at these free sources um so I have haven't come close to listing all the packages like everything I I mean OD pack right what's that going to do it's going to solve ordinary differential equations and many many many more um it's a interesting world of these packages uh what um so so lab of course has has these packages in its system uh uh there say for the finite element method there'd be a fem pack I'm sure uh actually that reminds me uh a new company which uh some people in Sweden wrote a wrote a finite element code for solving partial differential equations called fem lab and they decided okay we'll try to uh because that's a big job to write PD packages uh that's a multi-person multi-year task and and uh very expensive uh so they've actually uh uh established a company here in Burlington uh which I understand is making a go of it the it's the Swedish company was commo Computer Solutions so that's the company corresponding to math works and then this is the package they sell for solving uh partial differential equations by finite elements and uh mat has and matlb has a PD package sure I don't I hate to answer that say which is better on a permanent let's say they feel it's better right right and and uh and other people too uh yeah I mean they've focused on that and and uh and uh of course for you're you're seeing here as you've seen in the lecture the range of problems um if we're solving a linear system that's a pretty well-defined problem of course we have these cases within it uh of um banded Matrix or positive definite Matrix so we need special sub routines to solve partial differential equations those come in such variety that there the job is to organize it what what are you going to allow the user to ask for and I mean what are you going to supply well you pretty much have to allow the user to tell you the region somehow in which they want a solution and to tell you what the equation is and to tell you what the boundary conditions are you you you have to allow the user to tell you the necessary information but then you want to take the decisions on the mesh you the program should do that you want to you want to relieve the user of that maybe the user specifies some accuracy that they're willing to that they insist on and then that will control how many mesh points you have to take but it's it's a it's a so that's become a central part of the world of uh of U numerical uh numerical analysis scientific Computing is uh as these uh problems get get um kind of into a into a the way a book can be written when a when a subject is kind of getting there so software can be written yeah uh so that's a and then of course there are the other problems that haven't reached a stage where we don't uh where we don't know uh people are finding better ways all the time so that investing in creating a permanent package the time isn't ready yet yeah so that's uh that's part of the subject this is the part of the subject thinking about my own subject now where you're trying to create packages without knowing the problem right the it's the user who knows the problem you're trying to create packages that will solve many types of problems I I I I distinguish that from the other part of the field which is gaining the name computational science where where the numerical person is right in there with the biologist or the chemist or the chemical engineer or the the uh signal Theory person right in the problem and knows what the problem is uh so that's a that's a in a way a harder thing for the the computational person to get right in there and and and and sort of have a mixture so this is is computational science it's it's some mixture of of uh you need to know the science that's the point plus you need to know Computing the so computer science really you need to know what you can get the machine to do efficiently and you need to know uh the numerical you need to know about numerical algorithms you have have to have a feeling for which way are which are stable you know what what's a what's a good approach to to this and so this computational science is becoming a a sort of very important uh theme say within sayam the society I'm most involved with uh is more and more people for example in biology uh it's not good enough in biology ology just to sit back and and offer La pack you have to be um in there in the formulation and of the of the of the problem into into a optimization problem say I mean protein folding is a is an example that's uh very much in the in the air now uh so proteins you you you know what you're starting with but they fold themselves to get into a position of minimum energy with some constraints of course that atoms can't and molecules can't come too close and uh that's turning that's turning out to be a very very large optimization problem these numbers m&n are enormous and uh if you don't know something about the biology of proteins you can't write the best code so you really need to be uh expert in involved with the science with the with the the with the algorith the stability and Analysis of the algorithms and with the fast execution of them so that's really maybe uh where Applied Mathematics has come that uh classical Applied Mathematics was the theory of of different equations vessel equations and all the others maybe let vessel stand as an example of what of a whole subject and many people would still that's the center of their work but the LA recent years have brought other um parts of Applied Mathematics and they're in a kind of uneasy equilibrium now uh uh journals of Applied Mathematics some journals will emphasize the classical issues others will be uh uh purely numerical about about giving a job how would you do it and then others about what's the job so that's somehow a um a summary of uh what I hope these lectures uh brought out uh uh so with my own background uh those were not equally balanced I tried to identify mostly I I think to in order to achieve something within the hour uh try to identify a problem and then ask uh how to tackle it so I didn't um partly because of uh because I wouldn't know how to do it t uh go way way back to the origin of the problem that's more than I could do in these uh in these lectures but I want to emphasize I wish I want to emphasize the importance of that the modeling of the problem given the physical problem creating the model creating the equations is a very uh crucial and non-trivial step and uh highly valuable and uh that's U that's part of our subject too well this is maybe the right time to uh thank you guys for uh making these lectures possible uh I enjoyed it all and it's as you've seen um suggested problems and also it's suggested writing that's going to affect the uh the courses and uh I hope uh a new a new textbook so thanks right thank you