In this section, I'll cover running experiments, using SageMaker Autopilot. And, an experiment just refers to an Autopilot job. In this scenario, you're gonna use Autopilot to create a classification model, that will predict one of three classes, based on product review input text. In this case, you want to predict the sentiment, which includes a label of either -1 for negative, 0 for neutral, and 1 for positive, for each review on input. You're gonna let Autopilot find the right combination of feature transformations, algorithm, and hyperparameters that generate the best-performing model. As I go through Autopilot, and as you work with it, an important thing to note is that you can interact with Autopilot in multiple ways: including one, programmatically, going through the SageMaker API and using the AWS Command Line Interface, the AWS SDKs, or the SageMaker Python SDK; or you can also interact with Autopilot through SageMaker Studio, which is an integrated workbench for end-to-end machine learning workflow activities. Whether you're interacting with Autopilot programmatically or through Studio, you're still calling the same set of APIs, and you'll get to see both options in this session, but you'll focus on working with Autopilot programmatically in the lab. To get started, I'll show you how to configure your Autopilot job programmatically. First, you'll configure the Autopilot job completion criteria, which is really defining how long a job is allowed to run and how many candidates a job is allowed to produce. Some key configuration items you're able to define here include max_runtime_per_training_job_in_seconds, which allows you to set a maximum time in seconds that a training job is allowed to run. Second, max_candidates, which indicates the maximum number of times you want to allow a training job to run. And then finally, max AutoML job runtime in seconds, which indicates the maximum time in seconds that an AutoML job is allowed to run. So once you've configured the Autopilot job completion criteria, you want to also define your input and output data. Here, you'll specify your input data. So, this is the location of your S3 bucket that contains your dataset. You'll also need to specify your target attribute. Again, this is the attribute that you're trying to predict, which is sentiment in this use case. You also need to specify an S3 bucket that you want to use for your output artifacts, which I'll talk a little bit more about in the next video. But this would include items such as your generated notebooks, as well as your model artifacts. An optional configuration that you don't see here is generate candidate definitions only, which indicates the capability that I discussed earlier, where you can direct Autopilot to only generate the data exploration and candidate notebooks, so that you can review or modify them instead of running them in complete autopilot mode. Once you've configured your job, you now want to launch your Autopilot job. To launch your Autopilot job, you'll specify a name for your job and refer to the configurations that I just walked through. You'll also use a role that is defined in AWS Identity and Access Management, or IAM. The rule must be set up to allow permissions required for the job to issue the API calls that are needed, to complete your Autopilot tasks. I just walked you through the steps to programatically configure and launch your Autopilot job. The same steps can also be performed in more of a click-through experience, directly inside Amazon SageMaker Studio as well. Once your job is submitted, you can then monitor the progress of your job, either through DescribeAutoMLJob API call, or through your studio environment. Based on the product review data, including the target label for sentiment that you provided on input, Autopilot is going to automatically generate the feature engineering code required to take that data and transform it into the format expected by the algorithm selected by Autopilot. The product review data contains text, and there's a high variance in that text. So Autopilot is going to convert those text, or categorical, values into numeric values, that the selected algorithm can understand. There are several different ways to perform feature engineering and transformations for text data. Autopilot uses the SageMaker Python module, which is an extension built on top of scikit-learn. The module is available on GitHub, but it contains several submodules for various transformations, that are used by Autopilot, for the automatic generation of feature engineering code. In this example, you'll see a submodule that is focused on feature extraction of texts data, called MultiColumnTfidfVectorizer. MultiColumnTfidfVectorizer converts collections of raw documents to a matrix of Term Frequency Inverse Document Frequency, or Tfidf features. At a high level with Tfidf, you're evaluating how relevant a word is to a document, or a collection of documents. This is done by calculating how often the words appear and the inverse document frequency of those words across a set of documents. With the MultiColumn TfidfVectorizer, it converts collections of raw documents to a matrix of Tfidf features. Let's dive a little bit deeper here, so that you understand what's happening in the background. For processing texts with machine learning tools, you need to transform the document into a vector of features. A typical approach is to put all text in a bag and then sample from it without replacement. This way you can tally how many times a given word appears and produce a vector of counts. This approach is called Bag-of-Words. Once you produce a vector using bag-of-words, you want to quantify how important word is to a document, relative to others in the corpus. What you're aiming for here is a concise and reliable statistical description of a word's weight. This weight has to increase proportionately to the number of times that it happens in a given text, but it should decrease if the word is very frequent in that corpus because it will lack specificity. The Tfidf is a concise measure that will achieve exactly that. So, let's do a divide and conquer approach and start with the formula for term frequency, or TF: T refers to a term or a word, lowercase d to a document, and uppercase D to the corpus. The TF is the relative frequency of a word in a document, against all other words in that document. So because every document is different in length, it is possible that a term would appear many more times in long documents than in shorter ones. More concisely, it's the ratio of the word frequency in a document, over the same of the frequencies of all words in that specific document. Inverse document frequency, or IDF, measures how important a term is by looking at the whole corpus. The numerator is just the total number of documents in a corpus. The denominator tallies up all the instances of a term in the corpus. So as you will see, very common words like is, if, or the, may appear many times, but they have very little importance. Thus, you need to scale down the weight of frequent terms, while you scale up the weight of rare ones. This is the whole point of IDF, which puts this quotient on a log scale. The only caveat with this formulation is that it might lead to division by zero. Let's do an example. In a document containing 200 words, where the word food appears ten times, the term frequency, or the TF, for food is then 10 divided by 200, which equals 0.05. If the corpus has 1 million documents and the word food appears in 1000 of these documents, the inverse document frequency, or IDF, is log 1 million divided by 1000, or 3. The Tfidf weight is the product of these quantities, 0.05 times 3, which equals 0.15. In this section, I covered how you can configure and launch your Autopilot job, using your input dataset and defining your target labels. I also covered the approach that Autopilot takes for text transformations, including some of the theory behind those transformations. In the next section, I'll cover the resources and artifacts generated by Autopilot once your job is complete.
