L17.7 LLMS with Multiple Observations
We have presented the complete solution to the liner least mean squares estimation problem, when we want to estimate a certain unknown random variable on the basis of a different random variable X that we get to observe. But what if we have multiple observations? What would be the analogous formulation of the problem? Here's the idea. Once more, we restrict ourselves to estimators that are linear functions of the data, linear functions of the observations that we have. And then we pose the problem of finding the best choices of these coefficients a1 up to a n and b. What does it mean to find the best choices? It means that if we fix certain choices, we obtain an estimator, we look at the difference between the estimator and the quantity we're trying to estimate, take the square, and then take the expectation. So once more, we're looking at the mean squared error of our estimator and we try to make it as small as possible. So this is a well-defined optimization...