In this lecture we'll introduce the idea of classification, which is something we'll be studying for the next several lectures. And in particular, we'll compare classification versus regression models we've seen so far. And we'll address the kinds of problems that can be solved using classification versus regression techniques. Okay, so far we've studied regression, which has allowed us to make predictions of the following form, where we have some label we're trying to predict. And we model the prediction with an inner product between some parameter vector theta and feature vector x for each data point. So in particular the assumption this kind of model is making, is we have real value when numerical output is y that we're trying to predict. So things like rating data or heights or weights or things like that. Okay, so the classification, our goals is going to be to adopt these kinds of models, predict binary or categorical variables. So we still trying to cut off the function from data or some features of the data through some labels. But now, outputs should no longer be real value predictions but they should be things like binary prediction 01, true or false or categorical predictions, an element of a set from one to N. So what would that look like in practice? Or what are some example that we use classification rather than regression? So for example you wanted to make a predictor which says will I purchase this product? Which is a bit different from say estimating what rate I would give to a product. This is going to be a binary prediction. So given any user and product combination, I'd like to come with the binary prediction true or false, yes or no, it would say would this user purchase this product? Similarly, would a user click on an ad is also an instance of a binary prediction problem. Also, estimating a quantity like what animal appears in this image? This is an image of a mandarin duck. You could imagine if you wanted to solve image classification problems, they could be some very large collection of possible objects you're interested in identifying in images. If you'd like to make those predictions, it's not going to make sense to do so using real value outputs. Rather you'd like to predict individual categories of objects it might appear in the image or maybe a combination of different categories that appear in the image, Similarly if you want estimate the categories let's say a book belong to an Amazon or a restaurant belong to on Yelp. Maybe we want to make these kinds of categorical predictions where an individual instance could belong to multiple categories simultaneously. Again, not something we'd be able to address using this real value predictions have seen so far. But we need some more complex type of predictor which is capable of predicting category or combinations of categories. So in summary the concept we'll be looking at in the next few lectures is linear classification and we're going to try to build classifiers that make decisions according to rules in the following form. Okay, so we still have this predictor which takes the following in a product we're going to feature vector Xi and a parameter vector theta. However, rather than treating it in a product, as our prediction, instead, we make this comparison. We say, is it in a part greater than 0 or is it less than or equal to 0? If it's greater than 0, we predict 1, or yes, or true. If it's less than or equal to 0, we'll predict 0, or no, or false. So we're taking this idea of linear regression in the part between features and parameters. And we're saying, can we pass that idea through this thresholding function which says, if our prediction is greater than 0 we'll return true, otherwise we'll return false? So this is called linear classification because it's still using the same type of idea where we're making decisions according to a linear function. In other words, we can write it down the decision function as an inner product create a parameter back to add a feature vector. So there are several types of classifiers we'll look at. First, we look at nearest neighbors, which is a very simple but even in an instance of learning has no parameters in this model. It's a very simple classification scheme. Essentially just to introduce notation to use when dealing with classifiers. Later on we'll look at some learning base solutions such as this regression, which is basically a generalization of the techniques we've already seen with the regular linear regression. We'll briefly discuss Support Vector Machines. And finally, we'll see some Python libraries that can be used to do classification. So we'll introduce the mathematical concepts behind these different classifiers, but really we'll be focused on the practical aspects of how you can actually use these classifiers in Python, using libraries. Okay, so just to summarize, in this lecture we've introduced the idea of classification. Next time we'll be studying for the next several lectures. And we've compared and contrasted classification both with regression and seeing what types of problems each can solve.
