Hello, hello, everybody. Welcome to Module 2. In this module, we're going to talk about composite hypotheses, Type II errors, and p-values. In this first video, we're going to get started with Type II errors. I defined Type I and Type II errors in the first module and yet, we've only talked about Type I errors in our examples. So it's time we get to Type II errors. The setup is going to be our same beginner setup. We're going to assume we have a random sample of size n from the normal distribution with unknown mean mu and known variance Sigma squared, which again, is just a simplification for now. We're eventually going to take that assumption out. I know I did say we'd be talking about composite hypotheses, but for my first example, let's consider the simple versus simple case that I have up here. Mu_0 and mu_1 are fixed unknown. Those are actual numbers that you get to see. One is larger than the other, and we need to know which is which. We're just going to assume for this example that mu_0 is the smaller one. We did this hypothesis test already, and so I won't take you through, I won't drag you through the steps again. But in the end, the test that we did based on the sample mean, X-bar was to reject the null hypothesis in favor of the alternative if X-bar was large. Large meant that X-bar was greater than mu_0 plus the critical value z_Alpha times Sigma over the square root of n. The given value of Alpha comes in right there in that critical value, z_Alpha. The Alpha was our probability of a Type I error. In this video, again, we want to start talking about Type II errors. What is the Type II error for this test? I would love to set it just like we set the Type I error. But we're going to see in this example that we're stuck and our Type II error is locked in. Recall really quickly that we have two universes: the null hypothesis is true or the null hypothesis is false. In the case that it is true, we can fail to reject it or reject it. If it's true and we reject it, that is wrong and that's a Type 1 error. Then the other type of error, the Type II error is that the null hypothesis is false, so we should have rejected it, but we didn't. We let Alpha be our probability of making a Type I error and now, we're going to let Beta be the probability of making a Type II error. In the test, we just talked about, which is summarized up here, the Beta is actually locked in and let's see why. The Type II error means the null hypothesis is false. This is going to be the probability you fail to reject the null hypothesis. The when false means that the true mean is actually mu_1, the value given in the alternate hypothesis. False turned into when mu equals mu_1. We rejected when X-bar was greater than mu_0 plus z_Alpha times Sigma over the square root of n. We fail to reject if X-bar is less than or equal to that value. The last line here is just notation that I would like to use for the rest of the course rather than keep writing out things like when mu is equal to mu_1. We're going to use a semicolon and just put a mu_1 in there to denote the current value of mu that we're using to compute this probability. I was saying that Beta is locked in. Beta is the probability of the Type II error on the previous slide. We translated that to the probability of this happening. Now, I want to use the fact that I started with a normal distribution. The sample mean has a normal distribution because it's a linear combination of normals. We are under the assumption that mu_1 is the true value of mu. That is to say that the alternate hypothesis is true. This means that X1 through X n where IID from the normal distribution with mean mu_1 and variance Sigma squared. The sample mean is there for normal with mean mu_1 and variance Sigma squared over n. I want to subtract the mean, mu_1 and divide by the standard deviation to turn this into a standard normal. When we do that, on the left, this thing over here can be now called Z. We're done with this information to the right of a semicolon. This all boils down to the probability that Z is less than or equal to this big mess. But look at the big mass. Mu naught is in there and that is fixed and known. Mu_1 is in there and that is fixed and known. Z sub Alpha is in there. That is something you can look up once you have a value of Alpha and that is fixed and known. Sigma squared was assumed to be known. Sigma is the square root of Sigma squared is known. Unless specified otherwise in this course, we are assuming that we have a fixed sample size. All of that said, everything over here just blows down to a constant. We can look up the probability that a standard normal is less than or equal to a constant. That probability will be whatever it is. It will not necessarily be a particular Beta that we wanted to see. This is what I mean when I say, given we set the Alpha that determined the test for us, that actually determined the value of c for us. At that point, the Beta probability, the Type II error probability is locked in. We're going to talk briefly about how to unlock it. But just for the record, you could come at all of hypothesis testing from a Beta centric point of view, no one ever does this. It's usually the Type I error and then the Type II error is more of an afterthought like we just saw here. But we could have defined everything, starting with a Beta and using that to define our c. We're not going to do that. The other thing I wanted to say is if you would like to set your Beta, then you need this number not to be fixed because once it's fixed, this probability is fixed. The way you do that is by freeing up the sample size. If someone gives you an Alpha and Beta, then you say I'll find you a test, meaning I will take the statistic you want to use, the sample mean. I will give you a direction, I will give you the cutoff C, and I will tell you what sample size you'd need to use in order to set both Alpha and Beta to the desired probabilities. Again, our favorite two by two chart showing us Type I and Type II errors. I just wanted to point out that Beta is not in general going to equal 1 minus Alpha because the Type I and Type II errors are not complimentary events. Because you are either in the universe where the null hypothesis is true or the null hypothesis is false, you are in one of those. If it's true, you reject or fail to reject. Those are complementary events. In this box over here, the probability of making this Type I error is Alpha. The probability of making the conclusion fail to reject the null hypothesis will then have probability 1 minus Alpha. Those are complementary events. Same thing in the bottom row, the probability of making a Type II error, we're calling that beta. Therefore the probability that we reject the null hypothesis when it's false is the rest of that event, 1 minus Beta. Beta is not 1 minus Alpha because these correspond to errors that happen in different universes. But we will actually see that they are equal in a lot of cases, but they're not guaranteed to be equal. We'll eventually see cases where they're not. I said we get to deal with composite versus composite in this video. Let's go. Suppose I have the same setup. I have a fixed sample size unless otherwise specified. That's always the assumption. I've X_1 through X_n, IID or a random sample from a normal distribution with mean Mu and variance Sigma squared, where Sigma squared is known. Given a fixed number Mu naught, suppose I want to test the null hypothesis that Mu is less than or equal to Mu naught versus the alternative that Mu is greater than Mu naught. What are we going to do? Step 1. Step 1 is to choose an estimator for Mu or at least a statistic that we can work with. Because Mu is the mean for the population or distribution, let's take the analog for the sample, the sample mean. Step 2 is to determine the form of the test. This is always determined by looking at the statistic you chose and the alternate hypothesis and say, if that alternate hypothesis were true, how would that be represented in the statistic? The alternative hypothesis here says that Mu is greater than Mu naught and in general, just greater than it is when the null hypothesis is true. If the mean of the normal distribution is greater under the alternate hypothesis, then when you sample from that distribution, your x-bar will also be a larger value. This is why the form of the test is going to be just like before, to reject the null hypothesis in favor of the alternative if the sample mean, x-bar, is greater than some c to be determined. So far nothing has changed, but we're on to Step 3. Now in Step 3 we find c and we find c by using the definition of Alpha and the given set value of Alpha. Remember Alpha was the probability of making a Type I error. A Type I error means the null hypothesis is true, but you rejected it and so rejection for us means that the sample mean x-bar is greater than c and the null hypothesis being true means that Mu is less than or equal to Mu naught. The next step is always to standardize x-bar and turn it into a z. But you'll notice a problem. To standardize x-bar, we need to know the actual mean, not that the mean is in this whole interval of values less than or equal to Mu naught. All I know is that mu is less than or equal to Mu naught. I don't know exactly what it is, so I don't know what to subtract off when standardizing this. My point is that Alpha does not really make sense in this composite case. I've got a new definition for you and it turns out that definition will continue to hold back in our simple cases. There won't really be two definitions. They're only be one in the end. The level of significance or size of the test, we're still going to denote it by Alpha. It used to be the probability of making a Type I error. Now it's the maximum probability of making a Type I error. You want to control the maximum probability of making this error. The Type I error is still the case where the null hypothesis is true, but you rejected it. To make a Type I error, this is the probability that you reject the null hypothesis. To say, when the null hypothesis is true, we're going to put a generic view over here and then we're going to maximize this probability over all the Mu's contained in the null hypothesis. One way to write that is with this kind of epsilon thing. It can be read, is an element of. I'm saying here we're going to maximize over all the muse contained in the null hypothesis. But another way we might write this is exactly like the hypothesis is written. For the example we're in, we would talk about Alpha being the maximum over all Mu less than or equal to Mu naught of the probability we reject the null hypothesis when the parameter is Mu. Before going on to the Beta, I just want to quickly look at these simple null hypothesis using this definition and we'll realize that this definition will hold for simple or composite. In the simple case, the null hypothesis, if it was a simple hypothesis, is something like Mu equals a fixed number Mu naught. If you use this maximizing definition, then you would be maximizing the probability of rejecting the null hypothesis when the parameter is Mu and you'd be looking at the whole set of Mu's given by the null hypothesis, which consists of one point. Imagine trying to maximize a function over an interval, or even better over a single point. That's the only point the rest of it is erased. That is your maximum, that is your minimum, that is everything. Maximizing a function over a single point means plugging that single point in and so when you do this Mu turns into a Mu naught and the maximum goes away, and we're back to our original definition for the simple case. We're ready for Beta. Beta was the probability of making a Type II error. The Type II error method null hypothesis was false and we fail to reject it. In this case of a composite alternative hypothesis, this is going to be the probability we fail to reject the null hypothesis when we should and when we should means the parameter is a Mu, but we're looking at all the Mu's in the alternate hypothesis. Again, I wrote this as Mu in the alternate hypothesis. But in our particular problem, we can write this as Mu greater than Mu naught. Before concluding this video, I have one more definition for you, and that is what is known as the power of the test. Our Alpha is called level of significance. Our Beta doesn't really have a name, but 1 minus Beta is known as the power of the test and what I have written on the slide is a little confusing because I'm doing one minus a maximum, which is turning into a minimum which is turning into a, so let's forget the max and the min and just talk through this. The power of the test is 1 minus Beta. Beta is the probability of making a Type II error, really the maximum. This means the null hypothesis is false and we're failing to reject it. One minus the probability we fail to reject the null hypothesis is it's a compliment, and that is the probability we do reject the null hypothesis. But we are in the case where the null hypothesis is false, so we should reject it. That means we want 1 minus Beta to be large. Beta was a particular error, so we certainly want that to be small. Therefore, 1 minus Beta will be large. This quantity, 1 minus Beta is known as the power of the test. We'll talk about tests having high power. High power is good and we'll eventually be talking about these high power tests. But in the next video, I want to actually carry out a composite versus composite hypothesis tests now that we know how to define Alpha and Beta in this case. I will see you in the next one.
