Hello, everyone. Welcome to this week of the foundations of Sports Analytics Course. In the previous weeks, we have learned to use descriptive and summary analysis, data visualization, correlation analysis, and regression analysis to perform sports and in a task. In the last week of this course, we will talk about an interesting and a hot topic in sports, The Hot Hand. We'll demonstrate to you how we can use analytic methods to test The Hot Hand. First of all, what is Hot Hand? Hot Hand in sports refers to the idea of strict shooting. That the success rates of scoring will follow a pattern rather than being random. This means that there tend to be series when the athletes attempt to score are successful or series when the athletes attempt are unsuccessful. In other words, if a player experience a successful outcome, he will have a greater chance of success in future attempts. The idea of a Hot Hand originated from basketball. It is suspected that a shooter is more likely to score if he's previous attempt was successful. One common explanation for this suspicion is that previous success at a task can change the psychological attitude of the player and therefore change subsequent success rate of a player. In sports, the success rates in shooting goals should not be random. Some players have higher success rates than others due to their skews, their experience, and their forms. On the other hand, we need to distinguish that given a success rates, the sequence of success and failures should be random. A random sequence is the alternative of Hot Hand. For example, if a player has an average shooting success rate of 50 percent, the first scenario here appears that there is a mix between success and failure and this sequence seems to be random. In the second scenario, the player scored the first five goals consecutively, while missed the last five. This sequence does not appear to be entirely random, and similarly for the third scenario. Is the Hot Hand a phenomenon or just a fallacy? What has the academic research found? The first academic paper that studied the Hot Hand is by three psychologists in the 1980s. In the paper published in 1985 in cognitive psychology, the three offers first survey 100 basketball fans from Cornell University and Stanford University on their beliefs regarding sequential dependence among shots. From the survey, 91 percent of the fans believed that a player has a better chance of making a shot after having just make his last two or three shots, and 68 percent expressed similar believe for free throws that a player has a better chance of making his second shots after making his first shot. In this paper, they use field goal records of home games after Philadelphia 76ers players in the 1980-1981 season that has the probability of scoring conditioned on the players recent histories of hit and misses and found no positive correlation between the outcomes of successive shots. They argue that the players may change strategies in response to the results of their previous shot. Additionally, after a successful shot, a player may become more confident and attend more difficult shots, while in the meantime, the opposing team may shed from the defense of pressure on the player. On the other hand, after missing a shot, a player may become more conservative and only take high percentage shots. They also use free throw data from Boston Celtics during the 1980-1981 season, as well as the 1981-1982 seasons. This data allow them to award impacts from players on choice of shot and the defense of pressure from the opponents following the players previous successful shots. The authors examine the conditional probabilities of hitting a goal given that the previous attempt was successful. They also test for serial correlation on success rates of shots. Again, they do not find any evidence that the outcome of the second shot is influenced by the outcome of the first shot. Lastly, they also conduct a controlled experiment with Cornell's men's and women's worst team players. Each player would take 100 shot at a location, where his or her shooting percentage was roughly 50 percent. The players will be rewarded and the amounts of the reward was determined by how accurately they shot and how accurately they predicted their hits and misses. Again, the outcomes of field goal and free throw attempt are independent of each other. This pioneer paper concludes that previous success does not predict future hits or miss, both in field goal and free throws. The Hot Hand is attributed to misconception of chance. Then why do people believe that there's The Hot Hand? The authors provide two explanations. First, they argue that people could be biased towards looking for streaks. Long sequences of hit or misses are more memorable than alternating sequences. Second, people may expect the essential characteristics of a chance process should be represented not only globally in the entire sequence, but also locally in each of it's parts. For example, we know that when we flip a coin, the chance to get a head or a tail is 50 percent. People may expect that even with a small number of attempts, we should get roughly 50 percent heads and 50 percent tails. This conception of chance has been described as a belief in the law of small numbers. That is, people believe that the law of large numbers applies to small samples as well. This induces a believed that the probability of heads is greater after a long sequence of tails than after a long sequence of heads. He also leads people to reject the randomness of sequences that contain the expected number of runs. For example, people would tend to reject the randomness when they get four consecutive heads out of 20 tosses. A more recent paper published in the Journal of Sports and Exercise Psychology in 2003 also fails to find evidence to support The Hot Hand. Is the Hot Hand just a fallacy then? More recent researchers suggest a slightly different story. A paper published in the Journal of Quantitative Analysis in Sports in 2010, find that players have 2-3 percentage points higher probability in scoring the second free throw if they score the first one. Another paper in 2011, found that the probability of hitting the second shot significantly improves following a hit. A paper in the MIT Sport Analytic Conference in 2014 incorporates both the shooters location as well as the defenders location and find evidence to support The Hot Hand. Most recently, a paper published in econometrical suggests that the reason that the previous studies found no evidence to support a Hot Hand is due to a bias selection process. In the studies of Hot Hand, the sequence of shot is a primitive outcome rather than the individual shot. With all these studies he suggest that The Hot Hand may not necessarily be a fallacy. Now, let's look at our data and see what evidence we will find. To test The Hot Hand, we'll use a basketball Shot Log Data for the 2016-2017 season collected from the nba.com. Please open your Jupyter Notebook, Understanding and Cleaning the NBA Shot Log Data.
