3.4.4 R3. Election Forecasting - Video 3 A Sophisticated Baseline Method

Now, we're ready to actually start building models. So as usual, the first thing we're going to do is split our data into a training and a testing set. And for this problem, we're actually going to train on data from the 2004 and 2008 elections, and we're going to test on data from the 2012 presidential election. So to do that, we'll create a data frame called Train, using the subset function that breaks down the original polling data frame and only stores the observations when either the Year was 2004 or when the Year was 2008. And to obtain the testing set, we're going to use subset to create a data frame called Test that saves the observations in polling where the year was 2012. So now that we've broken it down into a training and a testing set, we want to understand the prediction of our baseline model against which we want to compare a later logistic regression model. So to do that, we'll look at the breakdown of the dependent variable in the training set using the table function. What we can see here is that in 47 of the 100 training observations, the Democrat won the state, and in 53 of the observations, the Republican won the state. So our simple baseline model is always going to predict the more common outcome, which is that the Republican is going to win the state. And we see that the simple baseline model will have accuracy of 53% on the training set. Now, unfortunately, this is a pretty weak model. It always predicts Republican, even for a very landslide Democratic state, where the Democrat was polling by 15% or 20% ahead of the Republican. So nobody would really consider this to be a credible model. So we need to think of a smarter baseline model against which we can compare our logistic regression models that we're going to develop later. So a reasonable smart baseline would be to just take one of the polls-- in our case, we'll take Rasmussen-- and make a prediction based on who poll said was winning in the state. So for instance, if the Republican is polling ahead, the Rasmussen smart baseline would just pick the Republican to be the winner. If the Democrat was ahead, it would pick the Democrat. And if they were tied, the model would not know which one to select. So to compute this smart baseline, we're going to use a new function called the sign function. And what this function does is, if it's passed a positive number, it returns the value 1. If it's passed a negative number, it returns negative 1. And if it's passed 0, it returns 0. So if we passed the Rasmussen variable into sign, whenever the Republican was winning the state, meaning Rasmussen is positive, it's going to return a 1. So for instance, if the value 20 is passed, meaning the Republican is polling 20 ahead, it returns 1. So 1 signifies that the Republican is predicted to win. If the Democrat is leading in the Rasmussen poll, it'll take on a negative value. So if we took for instance the sign of -10, we get -1. So -1 means this smart baseline is predicting that the Democrat won the state. And finally, if we took the sign of 0, meaning that the Rasmussen poll had a tie, it returns 0, saying that the model is inconclusive about who's going to win the state. So now, we're ready to actually compute this prediction for all of our training set. And we can take a look at the breakdown of that using the table function applied to the sign of the training set's Rasmussen variable. And what we can see is that in 56 of the 100 training set observations, the smart baseline predicted that the Republican was going to win. In 42 instances, it predicted the Democrat. And in two instances, it was inconclusive. So what we really want to do is to see the breakdown of how the smart baseline model does, compared to the actual result -- who actually won the state. So we want to again use the table function, but this time, we want to compare the training set's outcome against the sign of the polling data. So in this table, the rows are the true outcome -- 1 is for Republican, 0 is for Democrat -- and the columns are the smart baseline predictions, -1, 0, or 1. What we can see is in the top left corner over here, we have 42 observations where the Rasmussen smart baseline predicted the Democrat would win, and the Democrat actually did win. There were 52 observations where the smart baseline predicted the Republican would win, and the Republican actually did win. Again, there were those two inconclusive observations. And finally, there were four mistakes. There were four times where the smart baseline model predicted that the Republican would win, but actually the Democrat won the state. So as we can see, this model, with four mistakes and two inconclusive results out of the 100 training set observations is doing much, much better than the naive baseline, which simply was always predicting the Republican would win and made 47 mistakes on the same data. So we see that this is a much more reasonable baseline model to carry forward, against which we can compare our logistic regression-based approach.