Lec 19 MIT 18.085 Computational Science and Engineering I

two big themes in in in lectures that are coming uh well there's several lectures on Signal processing and in particular on wavelet developments so that's a half dozen lectures that that are sort of a unit in the course uh the other major theme is maybe the right word would be optimization how do you how do you identify and compute uh an Optimum uh of course in freshman calculus you have a function f ofx you take its derivative and set it to zero now uh we got to move up from there to functions of several variables so we got partial derivatives and I'll start with that today and then functions of functions so where the unknown that you're looking for is not just uh a point but a function a point in function space we could say so continuous variables and then and then constraints is optimization when there are uh constraints on the allowed on the admissible uh choices that that's a big subject and uh it's partly a world of its own optimization it's funny math meetings then there's linear people who do linear programming operations research uh numerical optimization and and that's a different world from pdes and uh and uh uh physical problems uh but anyway maybe it shouldn't be so such a different world and we can connect them okay so today uh uh let me start with a problem that I'll I'll just copy it up here again uh minimize a quadratic so start with quadratic functions I I guess we're I mean here's one General theme is uh quadratic uh minimum problems and they ought to lead to linear equations and the and and probably that's the form that you solve them in and many times that's the form that you create them in you formulate but there are other problems that are formulated as minimum problems like this one minimize uh let me call it P of view for potential energy or or total total energy really uh there's often we need this quadratic term U transpose KU this is our stiffness Matrix positive definite symmetric and then uh Source term which will be linear so you see this is this is a a a quadratic function of you of all of of the vector U of the N components so th this is here's the quadratic part here's the linear part and a natural question is where's the minimum what what value of U gives the minimum um I don't I'm going to just jump to the answer jump to the equation for the minimum there um and uh uh because because we've touched on this topic before uh and it's easy to check that it's the minimum but can I first ask you I use the words positive definite about K about that Matrix K in the middle why did I why what's the connection of positive definite with the with with what you see on the board here uh what is it that positive definiteness of this Matrix uh tells us tells you it's a minimum that's right exactly it tells you we we're dealing with that when we find if we set the derivatives to zero so and that's one I mean that's one way to the calculus way would be just set the derivatives to zero find a Stationary point then that's a minimum point because that's positive definite right that that's it exactly okay and what is that point what's the equation for that Point well the equation is kuu equal F if I well you can see that when I take the derivatives of this thing with respect to U the the partial derivatives I'm going to get the components of F and when I take the derivatives of this quadratic with respect to U with respect to each of these U's I'll get uh the result comes out to be KU so that's our that's our winner that's that that's our winner okay uh and and that so that's going back to uh uh early in the in the applied math book okay I want to move to a constrained problem because that's more subtle and uh let me take a minimum again so I'm going to use a quadratic and my unknown now is going to be it's just natural as you'll see for us to pick the other variable that this would be like the Potentials in a in a network problem and this would be like the currents so you'll see a w transpose our C inverse W that's the energy the heat generated by those resistors uh or or the potential energy in the springs but of course the Min what's the minimum of that just just so we see where the constraint comes in so what what w would minimize that expression there zero would right right but uh somewhere we ought to have we want to I want to connect this problem to this one and this problem had a source term namely F and here the source term enters in the constraints and and the constraint is so we're we're you see that we're on the in our in our famous diagram of U's and the vector a that takes us to E's that was that was up here now maybe I do dotted line C Inver C is the connection to W's and then a transpose and the equation a transpose W equal F the balance equation we we have had bunch of examples of that and I could multiply those examples so this problem you didn't see any W in this problem you don't see any u in uh but I want to let let me just ask now this question how do I find the minimum when I have to obey some linear equations how do I find the minimum when I have to obey some linear equations and let me give a specific example of this type of problem and then and then we can think how to solve it so the example will be uh distance to a line okay so I I'm I'm going to be in the there's some line I'm in w space W1 W2 and I have some line like well I don't know what that line looks like it has a slope of one so maybe maybe w 1 - W2 = 2 does that same a conceivable equation for what I've drawn there uh or should it be a plus guess plus is that minus was it you see a holiday in France and you don't know plus from minus uh is that right yeah minus look better and was it reasonable when W2 is zero W1 is two that's reasonable and when W1 is zero yeah thanks that's fine okay so what are what's our problem then distance to a line we're minimizing so what quantity am I minimizing and where's the winner let's let's see how this problem goes where uh so I'm minimizing distance so I'm taking any point on the line here's a typical point on the line and distance to a line from from the origin from 0 0 right right thanks thanks so all right now let state so there there is one point but not the closest point certainly but that distance then is we're minimizing W1 2 + W2 squ right distance squared we could say we're minimizing distance squared subject to the condition that we're on the line okay how to do that problem and you see that that's an example of this and it's an example with C what what's the Matrix C in this case adjust the identity and what's here I have uh W let's let's remember if we follow our convention should have M components and we're imposing n equations n constraints let me call them constraints right so over here m is two there are two W's and N is one there's one constraint so that now I have here an example of that and we could ask let's stay with this example for a moment and then come back to the to the um the general case with with the sort of the case with letters here's the case with numbers okay first of all tell me where where's the winning Point what uh what where is the W geometrically oh you've got I see I want is that oh all right I uh and and and and so what is special about one minus one I'll put it in it has a right angle exactly it's the so it's the point that's uh where that that's at 90° and that's the winter should we call it w star for the winner and I think you must be right it's one minus one this is working out even better than I wanted it to okay okay so you could see that that wins but how to approach problems like this how how to so now I ask you the question how do I deal with a constraint how do I deal with a constraint I could think of here I could think of well open to suggestion I could just substitute I could just say Okay this constraint tells me that W1 is W is 2 + W2 so I could I could change this I could solve this so I could say W1 is 2 + W2 and then I could say okay I'll minimize then W1 2 is 2 + W2 2 and the W2 2 okay so I'm back to a unconstrained problem I I I got the constraints out of the picture by solving for W1 now now that certainly seemed like a pretty good way to do it but I and and the but it's not the way I want to recommend and so the way I'm going to recommend is associated with these lrange multipliers so I'm going to I'm going to build the constraint I'm going what I'm actually going to do is is bring in a new unknown which seems kind of uh expensive or I mean inefficient to here we've got two unknowns this reduced it to a one unknown problem that was certainly certainly looked great and and instead I'm going to go the other way and I'm going to say I'll minimize I'm going to keep this quantity that I'm minimizing and then I'm going to subtract off this this constraint W1 minus W2 let me put the two on the same side and multiply this by some mysterious Shadow multiplier that I'll call you so this is this is the and Lambda it's often called Lambda but I for a reason you'll see I wanted to call it you so this is the famous ground multiplier so it multiplies the constraint now I have a problem with three variables W1 W2 and U and uh I want to take their partial derivatives with respect to those three variables that'll give me three equations when I set them to zero and then I'll solve for W1 W2 and U and will they be linear equations yes cuz I've got second degree stuff here when I take derivatives so let's just do that take the derivative with respect to First the derivative with respect to uh W1 what do I get I get two w1s minus a u equal 0 is is that right I set the derivative to zero now the derivative respect to W2 I'll get two W2s the Der spected W2 this is a plus u is zero and finally the derivative with respect to U is what it's the it's the constraint pops back so that you could say well uh certainly safe in that respect that the constraint popped back W1 minus W2 - 2 = 0 and let me move the two back onto the onto the right hand side so there was a d by dw1 here was a d by dw2 and that was the D by du three equations I guess I should ask now there's something not quite right on that word at the moment and what's not quite right is that those letters m this they're right here it's the minimum with of W's but now when I bring in this you guy this lrange multiplier it's not you see I'm not it's not positive definite if I if I wanted to make this small I I would I would make this guy Min i' sort of make this guy far off and I could well so what I'm trying to say is uh this is really uh I I don't know whether to cross out Min because I I shouldn't it's it's not a minimum over the it's really a minimum over the two the W still but it's probably a maximum over U I think yeah yes uh oh yes let's see uh I think it's okay but I did play quick with that minus sign so if I take the derivative respector U yeah let me do it right and then I think it'll then I think if I take the deriv with respect to U there really is a minus sign that's true minus the W1 minus W2 minus 2 should be zero that's that's the D BYU written right written correctly now I ignored that sign and I moved the two over there yeah yeah sure no I I did it too quick uh just to say what would happen here let can we before I go any further with that example can I do the same thing here so this minimum problem I'm going to introduce maybe we better give it a name like L for lron this so this is the lran this is so I I won't put Max or Min I'll just Define it as L as this is the has the W's in it and has the u in it and and uh it's the key quantity before I forget I I I I have to justify in some way uh doing this making it more making the problem more complicated where here we had a way to make it so simple so that that was so simple and why would we so this is the simple way but there has to be a butt and I'm trying to think how to express the butt Part somehow what what did we have to do to get there well we had to be we had to express some of the unknowns in terms of other ones we had to solve the constraints and that's what I want to say is not so easy in general the the this simple way but you have to but need to solve the con straint equation find the solutions you could say it's and and it's not there there there are problems where this is a reasonable way to go and it's called the null space method or something because you got to find you got to deal with this a transpose W equal F equation which means you you remember what what's going on with this equation here this constru equation uh it's n equations with M unknowns and M is bigger than n of course so we've got uh we've got some degrees of freedom here and and the the the problem of identifying exactly what are those degrees of freedom which doubl W should be solved in terms of which other W's is not uh easy not easy to make to do it systematically and if we had nonlinear equations like with log WS or e to the Ws or sin WS or something then we couldn't solve for some W's in terms of others so lrange has really got a better way so the lonian takes the thing that you're minimizing and then subtracts off introduces this multiplier U times the constraint and says okay build in the constraint into your function that that's what L Grange has done and if you if you've seen and used the gr multipliers and I'm you're you're I appreciate your patience CU this is uh most uh basic um context for those now there's one thing I have to fix here how many constraints have I got n how many lr's multipliers have I got I need n each each each of those equations deserves its so the first equation should get multiplied by U1 the first legr multiplier the second by U2 and they're they're n different legr multiplier I'm going to end up with L this guy L is a function of all the W's and all the U's there's so there's M of these guys and N of the this is how many unknowns we've got now M plus n m original W's and n multipliers because we've got n equations here so now can you tell me how to fix up what I've got there transpose I really I want the inner product between the B all all n of the U's and all n of these of that column right now it's right okay so is that does that copy for the general quadratic linear case this example for the specific uh distance to a line problem now do you want should we should I go back to that distance to a line problem and and and because the beauty of L Grange is he his method produces these equations we didn't have to work we didn't have to untangle these constraints at the start we're we're getting that system okay what's the solution to that set of equations now I'm I'm hoping that it's the right answer uh so what are what are W1 W2 it's three linear equations in three unknowns we it's produced in a systematic way and and we can solve it in a systematic way but actually in this example since you already started me uh we can solve it by sight what what's the answer W1 is what W2 is What U is what so W1 the correct W1 is one thanks and the correct W2 is -1 and that agrees with what you identified as the winning point now we get some more information which is what the the U and what does the U come out to be two all right so we got some more information except we don't know what it means right uh uh but but it has a meaning very important meaning actually this this too has a very very significant meaning so to to this is where the gron is a lot of subtle stuff in there and uh so at one level the gr is just telling you how to deal with a constraint what he says is build it into the function but at another level he's got uh his his that that's not just some arbitrary when we find its values in this case to they have an interpretation maybe I mention in in economics uh these would be called Shadow prices uh and I I won't try to U digress and and uh explain that but those are prices uh um that are kind of in the background uh let let me well so what do I want to say can I tell you what that U does actually represent at least have a shot at it I'm going to I'll take two or two or three different comments about the lrange multiplier and here it's just a single number because n was one in that example uh let me see I've almost solved the problem what I guess I haven't done is actually I written down the you you found the best W but what was the minimum then what what what was the minimum for W1 2 + W2 2 the thing that we were trying to make small turned out oh turns out to be what 1 SAR and Min - 1^ s two all right now please that's not you that's not you I'm sorry not not the same as you not the same as you in general I mean it it of course two is two but it's not wasn't meant to be all right now now I'm going to say what this U represents the U represents it's sort of the derivative of this answer with respect to a change in the constraint gosh can I try to put that down in words here what one one meaning for this lron multiplier is it's the rate of change of the minimum value with respect to changes in the data and the data here is this is the is the F the two d f in other words I think that if I change this to to to to well if I if I change it to 2 plus Epsilon what will happen can I suppose I change that two by a little bit then what's different about my problem I I'm not going to change this part of the problem I'm just changing the constraint a little if I change the constraint a little what happens to my picture that line moves the line moves right so here's the line here's the 2 plus Epsilon line right and then the winning W wouldn't be the same anymore the winning W would be on the new line of course has to be on the new line and the the minimum value would be a bit more and my claim is that it would be about U * Epsilon more and maybe actually we could even see that here I've I've never carried it as far as this but but but uh shall we have a shot at it um yes yeah right right I think that if I move that line a little then the winner what would the winning what would the winning point be 1 plus Epsilon over 2 and a minus one plus Epsilon over 2 or minus Epsilon over 2 okay and then I would I'm I'm hesitant this is going further in this lecture than I want to want to get and I want to get to the continuous calculus of variations so can I can I chicken out uh and just leave you with this wonderful fact that the the gr multiplier has a meaning and this is one one of its meanings okay I I've got to follow through in this General case what we did in the in the n in with the numbers I got to do now with letters okay tell me the equations I get when I take derivatives set to zero what what equations do I get when I take the W derivatives and set them to zero here so what equation will I get when I take the W derivative and and set it to zero uh you remember how this one went the the derivative of that gave me a KU so what's the derivative of this guy going to give me just C inverse W right I mean it it it just if this was scalar one by one this would just be2 w^2 over C and the derivative would be W over C right and now what about the W what's the W derivative from here Au good Au perfect perfect should equal zero and now what about the U derivative that's the the other rule here the other the other uh so this is this is setting one derivative to zero now tell me what the U what so this is this is M equations now I need in more equations and they come from the U derivatives being zero and what do I get when I said this is the easy question I get that constraint back a transpose W minus F and again I'll put the F onto the other side huh okay this is like supposed to be the the uh golden moment here uh in the that this problem which looked different from this one it did use the same stuff it used the same stuff because the K in this problem was was a transpose CA we we that we didn't say it but we understood it so the data up here is these matrices C and A and the vector F and the data down here is the matrices C and A and the vector f and my point is that I got to this system of equations this is exactly the letters case for this being the numbers case right and uh and now now what now I've got to solve those equations so just start me out how do I how how shall I solve this system of equations I'm hoping this looks familiar even though it's been a month solve for yes solve for w good so how do I I solve this first equation for w and what does that tell me have I got just I think I've just got space here you see I haven't lost The Knack of of uh of uh using everything all right so I solve that equation for W and it tells me that W is what cou good which we remember right W look you see this this framework what this is doing now what I hope is going to happen is we're going to see that it's the same problem we're going to see that this problem which sort of was associated with this part of this this this side of the framework gives the same set of equations as this problem that was associated with this part of the framework and what's more we going to identify what the lron multiplier U is physically turning out to be it's going to turn it right it it's going to be this guy that this LR multiplier that that came out of the blue is actually the potentials for the in the network example now now let me just finish here if W is CAU that was great and now I plug that into this right and I get a transpose now W is C Au U equal F and sure enough that's the same as that right so I I I I I you know like I could present lon's multipliers just as a device but that would be to miss uh the beauty of that idea it it is a device to build the constraint in and lead to equations and you see that if the constraint had been nonlinear that wouldn't have baffled us we still would have had its the ground multipliers we still would have taken derivatives we would have got nonlinear equations but we'd have got they'd have been ready to solve so lon's idea is really very general and you'll see near the end of these this series of lectures that we could even deal with constraints now this is this is going to be I'm going to make this constraint nonlinear not by taking logarithms or signs or exponentials but by that device by making it an inequality now I have that's a nonlinear constraint it's sort of like pretty close to linear but it's not linear uh and um and yet lrange can deal with that and will deal with it so linear programming quadratic programming that whole world of of uh of inequality constru the gr is is is up to where uh our simple method would just have uh wiped out okay yes matter no it doesn't and I'm and I for a moment I was wondering what did I do in the book and I wouldn't be surprised if I did it with a plus in the book and and we remember because and that would have produced a plus there which was which is what we would have uh been uh we've seen it with a plus yeah we've seen it with a plus right right yes so yeah so that con that's that's really just a convention whether to build in with a minus sign or with a plus sign both equally good and I shift from one to the other um yeah so can I just recap my my point is that there are whenever you have an optimization problem you have two problems that that's a a way to say it and that other Shadow problem you might not see and you might you know if it's just a straight minimization you probably don't want to see it just minimize this thing go for it but there is that problem back there and here in this form you got to see it we're minimizing something with a constraint we have to deal with that constraint the gr deals with it by building it in and increasing the number of variables increasing the number of equations but it connects back to the other so there's a what I'm going to what I'm mean is that there there's a a problem called Primal which is probably naturally call that one the Primal and there's another problem around hiding in the shadows it's the Dual problem optimization problems come in pairs can I even try to do describe a a a problem that that's sort of dual to this to this one let me let me get back to my original uh question here um let me move up a dimension let me move this into three dimensions to just to describe geometrically a dual problem because they're quite fun this this relationship between Primal equals dual and the there's a theorem called the Minimax theorem that the that that gives the connection that that's Duality I I'm I'm putting words on the board that I'm going to elaborate on in future lectures sure to this original one well because so I I can give a partial answer why why does the constraint have to be true let me get the Epsilon out of it because it showed up so by building it in this lrange way when I take the derivative respect to U that's bound to throw me back throw out the con throw the constraint forward you're that you that's that's uh excellent question so I I was answering the easy question that that the derivative of U will bring back the constraint that's that's just we built it in that way we we we multiplied the constraint by U so when I take the U derivative out will come the constraint that's easy the real question that you're asking is not easy is why does this work yeah yeah yeah and I I don't think I'm up to answering that today uh it's it's uh yeah it yeah you can you know anything you say I'll nod my head but guarantees that you don't add when you satisfy that yeah well that's right but somehow this you magically you know that we built it in in a way that it showed up here and that's that's really question I mean you know where the heck did it come from and so on which which makes everything perfect which is the winner yeah that's right you're you're thanks for that question question I mean you might say you might say well why not answer it but and and I haven't I it's it's it's a more subtle question than I'm than I I'm prepared to answer uh that's true that's right if I wanted to yeah let me that suggest a picture that I could draw for this for this problem which might ultimately give us some insight to to answer your question what would be the picture that I would draw well one picture is this picture but let me draw a picture in uh let me draw a surface so so what's what have I got there uh it's a one of my favorite bowls right circular Bowl because we've got the identity Matrix so if I graph W1 2 plus W2 s it's one of these guys right and what's the minimum zero okay but now impose the constraint so what does the constraint do in this in this in this way of looking at the problem so here is W1 W2 and the and I'm graphing W12 + W2 2 this thing that I'd like to minimize but now what is this constraint W1 minus W2 equal two what is so I've got this nice beautiful Chinese Bowl uh oh I'll have to bring you my what I brought back from China EXC you'll allow this digression two two interesting things on the plane uh coming back from China uh one was Henry Kissinger uh well thing I excuse me Henry but uh he was in seat onea so I wasn't quite across from him but uh I had a word with him at the uh he collected his own luggage at the carousel in San Francisco so anyway it it had been 30 years since his first you know that historical visit to China uh so that was one thing the other thing was something I brought on the plane I'll bring it in to show you if you're willing um it's it's a Chinese teapot I I won't spoil it now I just I'll I'll nor will I use up more of the lecture to tell you about it but um I I will bring it with me next time okay you'll you'll smile when you see it okay now where am I I want to build this I want to say okay in this picture of the problem where I'm looking at the minimum point on the surface but now I'm imposing that constraint what is that constraint do exactly good and and and that Bob's question brought it out this is a vertical plane W1 minus W2 equal 2 it's it's there's our line this is the same line is in that picture and it there's a slice coming down so we're really slicing through the so we're slicing out a kind of Parabola and the winner is the point there's our winning point one minus one and the winning height was whatever we decided it was too I guess so the I mean that's really nice to see that to see that bowl with the vertical slice coming through it you can that's something in 3D that our eyes can visualize where we would have the the beauty is like gr gives this mechanical approach where we don't really have to think uh in high Dimensions we just do what he says and outcome equations but you're totally right to since I'm emphasizing that it's not purely mechanical and that his multipliers have a meaning then your question is totally right that what is that meaning what's going on so maybe it's good that this isn't the last lecture on on optimization we'll see the gr multipliers we'll see inequality constraints and linear programming all all these things we'll see Duality uh maybe maybe if if if I've achieved two things one is to what's the system what's the mechanical system and the second to sort of wet your appetite with what's going on what does it what does you really mean then that that's I'm happy with that outcome of of here's the way to introduce the ground multipliers and then I gave some kind of a meaning to them the change in the answer with respect to change in the input data it's fantastic and uh that's what L gr has has achieved so there's a but to see the to see the wise I haven't uh I haven't tackled that in this lecture okay I have a little time left in that in this lecture for the continuous time the function equivalent are you take a deep breath I'll do the same can I and I'll make a little space can I try to give the corresponding problem in continuous time okay so there you you can Reon let me put a yellow thing around the two problems one problem was minimizing this right and we'll be pretty happy if we can turn that into continuous time the second the Dual problem was this one and we may not have time to discuss its continuous analog so it's those it's these problems in discreet for n variables and M variables that I now want to say what happens if the unknown is not a point but a function okay and of course and and our framework is going to work what will a be in our framework now this now U is going to be a function I'll let's go to the most basic example you remember the what was the most basic example a became d by DX remember remember that and a transpose became minus d by DX so we had differential equations what was the differential equation uh what was this equation let's just remember what it is we're doing this equation for when a was D by DX was do you remember that was so the a transpose was minus d by DX then we had the C ofx and then the a which was D by DX of U = FX that's the equation the linear equation that's the that's this okay now all I want to ask is what's the minimum problem that it comes from what's the what's the continuous analog of this uh minimum problem so this this is going to be our winning equation this is going to be the equation for the winner and and I'm working up here now and not not the constrained case just the easier unconstrained case this picture is still relevant but now I've got to think what's P of you so what's P of U so I remove this example and make space for p of U in the continuous case Okay okay so now I'll put up here in the yellow letters continuous variable the unknown is unknown is is a function U of X and what's the P what's the thing that we're minimizing we're minimizing some P of U can you can you convert that with K as a transpose CA so let me let me make K into a transpose CA so I want to I want to convert that to the continuous case what's the continuous equivalent of that problem that's that's okay what do I do what are these dot products going to be let's let's just start off easily what's the continuous version of that now f is a function f ofx u is a function U of X what's the dotproduct it's an interval right so this term is going to be exactly minus the integral over some some range let's make it 0 to one again of U of x f ofx DX good that's the that's the uh linear term that's the linear term okay what about the quadratic term so it's it's a there's a so I'm look at this this way look at this as Au transpose C Au is that all right so what what have I got here then I've got the 1/2 now I I've got a DOT product there right so I'm going to have an integral and what is it the dotproduct of it's the dot product of au and and I'm going to focus on this example a is d by DX is it's DX the dotproduct of Dux that's Au and now what's this well there's a c of X and and Au is another Dux so it's really DX squared sure DX well it's the energy I actually since they're both integrals I could I could simplify my life with just one integral sign and one DX yeah that that would be a lot better so minus U of x f ofx DX yeah I'll just move this DX out a little bit so you know so this is it this is our this is our Au transpose Au transpose C Au and this is our U transpose F yeah that's right what sorry what about the half oh it can't put the half on the outside right the half only goes with the quadratic terms can I squeeze it in I can't squeeze it in there that's for sure uh let me put it there yes thank you absolutely right absolutely right okay all right so that's what have we done we've operated by analogy I mean and that's what part of this series of lectures is to develop that instinct that analogy between the discret and the continuous case right so so dot products became integrals matrices became derivatives and the unknown changed from a vector to a function but how do I minimize when my unknown is a function how do I find the function U of X that makes this quantity a minimum do you see that it's a it's it's a quantity that depends on U of x you you've got to I've got to I'm looking for a whole function now I'm not just looking for what point in space I'm looking for a a point in function space H how to minimize such a thing let me ask you what the equation you what you think the equation is for the minimizer somehow do you see what I if you give me a function U like sinx I can plug it in I get a number then you give me another function like e to the X I plug it in I get a different number and I'm looking for the function that produces the smallest number but obviously what I've described is just randomly trying functions is not a recommended method uh yeah good exactly I have to take I need exactly I need the idea need need the idea of of the derivative what let me use a different Delta of the fun of this p with respect to U that's what I need and of course I'll set it to zer I mean it'll be calculus but it'll be calculus of well the real word is calculus of variations so calcul of variations all that means is move the problem up to up to here where the unknown is a function okay so I need that idea I and I don't know that in 30 seconds I can produce it but the main point is you see what's needed you see we need well how would you do it let's not even do it in detail but just actually my my second lecture today can can can can uh build a little further on this so let let me just ask you for a thought about how to create this idea yeah how would you how would you look for the minimum or look for an equation for the minimum what what what would be a property of the minimum I mean so I'm really asking you okay be Newton all over again invent calculus uh no constraints at this point you you're allowed any function U of X this is this is given this is given this is the data this is the data and you're allowed any fun fun U of X and I want to know which function U of X makes that integral as small as possible and I well I I I want an equation for that function and so I want to reinvent calculus for functions so any thoughts on that yeah yes I'm expecting a linear right yes right so I'm expecting a linear equation because I've got a quad something quadratic in you right complete a square is that's an interesting idea but but now how did how did calculus how did calculus come up with this this this idea that the minimum value is that point where the derivative is zero what what was underlying that that idea for for calculus right any change increases it exactly if I'm here's my winning point and if I move over a little it goes up now so what's a change here you is the winner means what you if you is the winner it means that this value is smaller than what I will get if I change you a little change d by DX of a little can I change it by v^ S C of x/ 2 minus I'm changing U by V any v f ofx DX did I did you see that idea because that's the central idea this is got to be true for any V any V ofx any V ofx you recognize V ofx is a function now it's it's so in other words U plus v is a is a candidate somehow I'm I'm taking the derivative of the spect of V right well well I'm no I'm taking I'm taking it's I I think you should think of it I'm at a point in ordinary calculus I'm at a point x and I'm trying to minimize some function f and and what's my rule my rule is that if x is the winner then f ofx is smaller than what I get if I move a little bit that's that's what led to that's what led to calculus and this Delta X could be to the right or to the left and this will lead me to the conclusion in the calculus conclusion will be that DF DX has to be zero and if I if I want to know that it's a real minimum then there's something about the second derivative being positive and will this sort of square here is what's going to deal with that but I'm I'm at the first derivative level finding the equivalent so this was the this was the calculus idea you could say and and what is that calculus idea the calculus idea said well what is f ofx what's the connection between this line and this one well calculus says okay if you just moved a little bit this is approximately f ofx plus what uh Delta X fpre right if if my movement was just small then there's corrections but they'll be extremely small so so now I have f ofx is smaller than this what's the reasoning now there would be a second order term yes but but I I if i' I'd like to if I can just get it out of this first order cuz I'm only looking for the the equation this is DF DX of course what what's the reasoning why does that have to be zero that's that's what my question is cuz if it's not zero you could move a little bit what if it was negative move in the other direction great that's it that's it that's why the derivative has to be zero because this Delta X you could move one way or the other way so that if this derivative wasn't zero then some way it's going to drop and this would wouldn't be the winner so that's what we'll do here that's what we would do here we would say okay we can this can be any V it can be positive somewhere negative somewhere and so out of all this will'll will'll identify okay the line the term that's linear in V the the the the this the linearized correction I I've carry this beyond the hour but and I'll stop here and pick it up at at at uh in the second lecture uh and then go on to the finite element method which uh illustrates this in a nice way um so uh but do you see where I am we we're looking for the equation for the winning U and by the way what is that equation going to be it's going to be this that's going to be the equation for the winning U and it's named after or oer or actually often named after it's often called the oiler lrange equation so somehow the equation for the winning U when I follow the calculus ideas will will turn out to be that equation and then the analogy will be perfect has a that has a u in it here yes and this is the winning you yeah I I yeah so but but you're raising a good point I I'm not making a very good notational distinction here between this is p of anyu any is allowed for the moment and then I'm looking for the winning one maybe I should give a star to the winner you you somehow indicate the function that that makes this a minimum so that's the point that if if this guy is the winner then wherever I move brings me up and that should that should uh lead to an equation for ustar okay that let's let me pause there if you get a chance to think a little bit about this during the 20 minute break and pick up at 3:30 thanks