5.2.6 An Introduction to Text Analytics - Video 4 Bag of Words

In this lecture, we'll use a technique called Bag of Words to build text analytics models. Fully understanding text is difficult, but Bag of Words provides a very simple approach. It just counts the number of times each word appears in the text and uses these counts as the independent variables. For example, in the sentence, "This course is great. I would recommend this course my friends," the word this is seen twice, the word course is seen twice, the word great is seen once, et cetera. In Bag of Words, there's one feature for each word. This is a very simple approach, but is often very effective, too. It's used as a baseline in text analytics projects and for natural language processing. This isn't the whole story, though. Preprocessing the text can dramatically improve the performance of the Bag of Words method. One part of preprocessing the text is to clean up irregularities. Text data often as many inconsistencies that will cause algorithms trouble. Computers are very literal by default. Apple with just an uppercase A, APPLE all in uppercase letters, or ApPLe with a mixture of uppercase and lowercase letters will all be counted separately. We want to change the text so that all three versions of Apple here will be counted as the same word, by either changing all words to uppercase or to lower case. We'll typically change all the letters to lowercase, so these three versions of Apple will all become Apple with lower case letters and will be counted as the same word. Punctuation can also cause problems. The basic approach is to deal with this is to remove everything that isn't a standard number or letter. However, sometimes punctuation is meaningful. In the case of Twitter, @Apple denotes a message to Apple, and #Apple is a message about Apple. For web addresses, the punctuation often defines the web address. For these reasons, the removal of punctuation should be tailored to the specific problem. In our case, we will remove all punctuation, so @Apple, Apple with an exclamation point, Apple with dashes will all count as just Apple. Another preprocessing task we want to do is to remove unhelpful terms. Many words are frequently used but are only meaningful in a sentence. These are called stop words. Examples are the, is, at, and which. It's unlikely that these words will improve the machine learning prediction quality, so we want to remove them to reduce the size of the data. There are some potential problems with this approach. Sometimes, two stop words taken together have a very important meaning. For example, "The Who"-- which is a combination of two stop words-- is actually the name of the band we see on the right here. By removing the stop words, we remove both of these words, but The Who might actually have a significant meaning for our prediction task. Another example is the phrase, "Take That". If we remove the stop words, we'll remove the word "that," so the phrase would just say, "take." It no longer has the same meaning as before. So while removing stop words sometimes is not helpful, it generally is a very helpful preprocessing step. Lastly, an important preprocessing step is called stemming. This step is motivated by the desire to represent words with different endings as the same word. We probably do not need to draw a distinction between argue, argued, argues, and arguing. They could all be represented by a common stem, argue. The algorithmic process of performing this reduction is called stemming. There are many ways to approach the problem. One approach is to build a database of words and their stems. A pro is that this approach handles exceptions very nicely, since we have to find all of the stems. However, it won't handle new words at all, since they are not in the database. This is especially bad for problems where we're using data from the internet, since we have no idea what words will be used. A different approach is to write a rule-based algorithm. In this approach, if a word ends in things like ed, ing, or ly, we would remove the ending. A pro of this approach is that it handles new or unknown words well. However, there are many exceptions, and this approach would miss all of these. Words like child and children would be considered different, but it would get other plurals, like dog and dogs. This second approach is widely popular and is called the Porter Stemmer, designed by Martin Porter in 1980, and it's still used today. Stemmers like this one have been written for many languages. Other options for stemming include machine learning, where algorithms are trained to recognize the roots of words and combinations of the approaches explained here. As a real example from our data set, the phrase "by far the best customer care service I have ever received" has three words that would be stemmed-- customer, service, and received. The "er" would be removed in customer, the "e" would be removed in service, and the "ed" would be removed in received. In the next video, we'll see how to run these preprocessing steps in R.