Topic 9: Foundations for Inference (Spreadsheets)

About

This activity provides an introduction to the Central Limit Theorem. You’ll encounter the Central Limit Theorem in action through several simulations which construct approximations of the sampling distribution. You’ll have opportunities to change parameters of the population distribution as well as the size of the samples being drawn. The goal is to discover connections between a population distribution, sample size, and the resulting sampling distribution.

Note. The button above resets multiple choice and checkbox questions. Currently, resetting code cells must be done manually via hitting the Start Over button on each individual interactive cell.

Using a Spreadsheet Program

There are times throughout this activity where you’ll be asked to do work inside of your favorite spreadsheet program.

A Note on the Simulations in This Activity

This activity makes heavy use of interactive simulation code blocks. Simulations such as these genuinely require a computing environment and there is no direct, convenient analog to spreadsheets without add-on functionality. When you encounter an interactive code block, engage with it – change the parameters as indicated by the comments provided.

You do not need to understand the code, though you are welcome to inspect it if you’d like. You cannot break anything beyond repair – simply hit the Start Over button on the code cell to reset that code cell to its initial state.

Foundations for Inference

In this activity we’ll begin investigating the true power of statistics — using sample data to make accurate claims about a population, even when we don’t have access to the entire population. We start here by exploring the connection between a population distribution and the distribution of sample means, often called the sampling distribution. We’ll do this through a series of interactive code blocks which you will run and use to answer questions.

Exploring the Connection Between Population and Sampling Distributions

Start by viewing the following video from the New York Times.

The video claimed that the sampling distribution can help us answer questions about the population. This is really important because, as we mentioned in our first activity, collecting a true census is almost always impossible. In the following sections, you’ll use the provided code blocks to explore the connection between the population and the sampling distribution for various different populations.

The Sampling Distribution of the Sample Mean

The sampling distribution of the sample mean is a theoretical distribution consisting of the means of all possible samples of a fixed size (\(n\) elements) drawn from a population.

The Sampling Distribution of the Sample Mean will be our central object of study throughout this activity. Our goal will be to understand how this distribution is connected to:

  1. the population distribution, and
  2. the sample size.
How to Use The Code Blocks

You’ll encounter several large code blocks in this activity. As mentioned earlier, you are not expected to examine or understand the code in each block. Instead, focus on the plots that result each time you run the code. You are invited to change the first few lines of each block (the parameters), and you should do so! You are not expected to modify the remaining code though.

Each time you run a code block, two plots will be displayed.

  • Both plots show the resulting approximation of the sampling distribution via a histogram.
  • The plot on the left overlays a normal distribution whose mean and standard deviation match the assumed population parameters.
  • The plot on the right overlays a normal distribution whose mean matches the population mean, but whose standard deviation is adjusted according to the Central Limit Theorem.

A Normally Distributed Population

Work with the following code block to explore the connection between the population distribution and sampling distribution when the population follows a normal distribution. Use your explorations to answer the questions that follow.

Hint 1

Try changing the following parameters one at a time and re-running the code — these are the mean, the standard deviation (sd), the sample size, and the number of samples (numSamps). What changes do you notice about the population distribution? What changes do you notice about the sampling distribution?

Hint 2

Pay particular attention to what happens to the shape and spread of the sampling distribution as you change the sample size. What do you notice?

Check Your Understanding: Normal Population I

Which of the following regarding the population and sampling distributions is true?

Check Your Understanding: Normal Population II

Try various different values for sample size. What is true about the mean of the sampling distribution?

Check Your Understanding: Normal Population III

Try various values of the mean, standard deviation, and sample size. What can be said about the sampling distribution?

Check Your Understanding: Normal Population IV

Now that you’ve tried various values for the parameters, what can be said about the connection between sample size and the spread of the sampling distribution?

Recap: Normally Distributed Populations

Okay, so if a population distribution is approximately normal, then the sampling distribution is also nearly normal – what’s the big deal?

The real takeaway here is that, if it’s reasonable to assume that our data comes from an approximately normal population, then our sample mean is a reliable estimate of the true population mean — and the larger our sample, the more reliable that estimate becomes.

This might seem obvious, but it’s actually the foundation of everything that follows in our course. We almost never have access to an entire population, so we rely on samples. Knowing that our sample mean is a trustworthy stand-in for the population mean — and being able to quantify how trustworthy it is based on sample size — is what makes statistical inference possible.

Okay, so what if we can’t assume that the population we are interested in follows a “nearly”-normal distribution? Do you believe in “magic”? (Disclaimer: There is actually no magic, or trickery, involved in what you are about to see – just rock-solid mathematics. Get ready to have your mind blown!)

A Uniformly Distributed Population

A population is said to be uniformly distributed between some minimum value A and a maximum value B if all values between A and B are equally likely to be observed.

Work with the following code block to explore the connection between the population distribution and sampling distribution when the population follows a uniform distribution. Note that in the initial plots, assuming a normal distribution for the either the population or sampling distribution is an extremely poor choice. Use your explorations to answer the questions that follow.

Hint 1

Again, try changing the minimum value, maximum value, sample size, and number of samples (numSamps) one at a time and re-running the code. What changes do you notice about the population distribution? What changes do you notice about the sampling distribution?

Hint 2

Pay particular attention to what happens to the shape of the sampling distribution even for very small sample sizes like 2 or 3. How does this compare to what you observed with the normally distributed population?

Check Your Understanding: Uniform Population I

Which of the following regarding the sampling distributions is true?

Check Your Understanding: Uniform Population II

Try various different values for sample size. What is true about the mean of the sampling distribution?

Check Your Understanding: Uniform Population III

Try various values for the parameters. What can be said about the connection between sample size and the spread of the sampling distribution?

Recap: Uniformly Distributed Populations

Given a uniformly distributed population, the sampling distribution is approximately normal for samples of size 2 or greater.

  • The mean of the sampling distribution is the population mean.
  • The spread of the sampling distribution is related to the sample size – larger samples result in more narrow distributions.

The Impact of Skew on the Sampling Distribution

We’ve seen that the sampling distribution is nearly normal for any sample size when the population is nearly normally distributed, and nearly normal for samples of size at least two when the population is uniformly distributed. What if we move in a very different direction and consider a population that is extremely skewed?

We’ve encountered skew in our course already, and we know that it describes the effect of the mean being pulled away from the center of our distribution by outliers. This witchcraft (read: mathematics) certainly cannot apply in the face of strongly skewed distributions, can it?

Work with the following code block to explore the connection between the population distribution and sampling distribution when the population follows a strongly skewed distribution. Use your explorations to answer the questions that follow.

Hint 1

Try changing the shape and rate parameters one at a time — can you figure out what each one controls by watching how the first plot changes? Then try increasing the sample size gradually. What do you observe?

Hint 2

Try sample sizes of 5, 10, 20, 30, 50, and 100 in sequence. How does the shape of the sampling distribution change as the sample size grows? How does this compare to what you observed with a uniform population?

Check Your Understanding: Skewed Population I

Try running the code with samples of size 5, 10, and 20. Are the resulting sampling distributions nearly normal?

Check Your Understanding: Skewed Population II

Try samples of size 30, 50, and 100. What can be said about the resulting sampling distributions?

Check Your Understanding: Skewed Population III

Try various different values for the other parameters. What is true about the mean of the sampling distribution?

Check Your Understanding: Skewed Population IV

Since you’ve tried various values for the parameters, what can be said about the connection between sample size and the spread of the sampling distribution?

Good work through the previous sets of questions. Think about your answers and use them to answer the following questions about the connection between population distributions and sampling distributions in general.

Check Your Understanding: General I

Consider a generic population distribution. What can be said about the shape of the sampling distribution?

Check Your Understanding: General II

For a generic population distribution, what can be said about the mean of the sampling distribution?

Check Your Understanding: General III

For a generic population distribution, what can be said about the connection between sample size and the spread of the sampling distribution?

Recap: Putting It All Together

In all of these cases, the mean of the sampling distribution falls close to the mean of the population distribution — but the more important observation has to do with the spread of the sampling distribution.

We’ve been working backwards here — we assumed we know the population distribution and took many thousands of samples to construct the sampling distributions. This is exactly opposite of the real statistical scenario: we don’t know the population distribution, we can’t collect thousands of samples, and we generally have just one sample of a fixed size.

Luckily, from the experiments we’ve run (and from mathematics which has been proven to work), we know that our sample mean falls “near” the population mean, and we can describe what “near” means since it depends on the size of our sample. This is how I was able to draw those normal distributions in all the plots on the right, and it is what makes classical statistics work!

The Central Limit Theorem

What you just discovered through the simulations and questions above is what statisticians call the Central Limit Theorem — the result discussed in the CreatureCast video at the beginning of this activity. This is one of the most important theorems in all of statistics, and it is what will allow us to make and test claims about populations for the remainder of our course.

The Central Limit Theorem (Simplified)

Regardless of the shape of a population distribution, for sample sizes large enough to overcome skew, the distribution of sample means (the sampling distribution) is approximately normal. Furthermore:

  • The mean of the sampling distribution equals the population mean (\(\mu\)).
  • The spread of the sampling distribution is described by the standard error (\(S_E\)), which depends on the population standard deviation and the sample size.

For a population with mean \(\mu\) and standard deviation \(\sigma\), the sampling distribution of sample means from samples of size \(n\) is:

\[\bar{X}_n \sim N\left(\mu,~S_E = \frac{\sigma}{\sqrt{n}}\right)\]

Sample Problem: Suppose you have a 46 square foot wall which you want to cover with spray paint. The brand of spray paint you plan to use is known to have coverage which is approximately normal, with an average coverage of 10 square feet per can and a standard deviation of 1.5 square feet. Use your spreadsheet to answer the questions that follow.

Check Your Understanding: Spray Paint I

How many square feet would each can need to cover if you wanted to use only four cans to cover the entire 46 square foot wall?

Take the 46 square feet and divide it up into coverage by each of the four cans.

Use an empty cell in your spreadsheet and calculate 46 divided by 4.

Type = 46/4 in an empty cell and hit Enter.

Check Your Understanding: Spray Paint II

What is the probability that a single can of spray paint covers at least 11.5 square feet? Use your spreadsheet to compute this.

We want the probability that a single can covers at least 11.5 square feet. Draw a picture.

The per-can coverage is approximately normally distributed. Your picture should have the right tail (upper tail) of the distribution shaded. How do we find the probability in the upper tail?

Use = 1 - NORM.DIST(___, ___, ___, TRUE) in your spreadsheet.

Use = 1 - NORM.DIST(___, ___, ___, TRUE) in your spreadsheet. Fill the first blank with the boundary value (11.5), the second with the mean (10), and the third with the standard deviation (1.5).

Check Your Understanding: Spray Paint III

What is the probability that a random sample of four cans of spray paint covers an average of at least 11.5 square feet? Use your spreadsheet to compute this.

What makes this one different than the previous question?

The Central Limit Theorem here guarantees that the sampling distribution will be approximately normal (because the population distribution was). The sampling distribution won’t be identical to the population distribution though – what is different?

The mean of the sampling distribution will be the same as the mean of the population distribution (10), but the spread of the sampling distribution will be measured by the standard error: \(S_E = \sigma/\sqrt{n}\)

We’ll still use = 1 - NORM.DIST(11.5, 10, ___, TRUE), but that third blank will be filled in by 1.5/SQRT(4) because the spread of the sampling distribution is narrower than the spread of the population distribution.

Check Your Understanding: Spray Paint IV

Should we expect to cover the entire wall with only four cans of spray paint, or should we plan to buy five?

There is a chance that the spraypaint cans will cover the 46 square feet, but is it a good one?

We generally set our threshold for an outcome being “unlikely” or “unusual” at below 5%.

Since the probability of the average coverage of the four cans being at least 11.5 square feet is so low, we should just plan on buying five cans of spraypaint.

Submit

If you are part of a course with an instructor who is grading your work on these activities, please copy and submit the hash below using the method your instructor has requested.

Question Hash

The hash below encodes your responses to the multiple choice and checkbox questions in this activity.

Exercise Hash

Since there were no code cell exercises in this activity, there is no exercise hash to generate.

Summary

In this activity you discovered one of the most important results in all of statistics through hands-on simulation. Here are the key takeaways and a look at what’s ahead.

Main Takeaways
  • The sampling distribution describes the behavior of sample means. When we take many samples of size \(n\) from a population and compute the mean of each, the resulting distribution of those means is called the sampling distribution.
  • The mean of the sampling distribution equals the population mean. Regardless of the population shape or sample size, the center of the sampling distribution is always close to the true population mean \(\mu\).
  • Larger samples produce narrower sampling distributions. The spread of the sampling distribution — measured by the standard error \(S_E = \sigma / \sqrt{n}\) — shrinks as sample size grows. Larger samples give us more precise estimates of the population mean.
  • This is the Central Limit Theorem (CLT): For large enough sample sizes, the sampling distribution is approximately normal, regardless of the shape of the population distribution. For normally distributed populations, any sample size works. For skewed populations, larger sample sizes (30 or more for moderate skew) are needed.
  • Skew increases the required sample size. The more extreme the skew in the population distribution, the larger the sample size must be before the sampling distribution becomes approximately normal.
  • The CLT is what makes statistical inference possible. Because we can predict the shape and spread of the sampling distribution, we can make reliable probability statements about how close a single sample mean is likely to be to the true population mean — even when we only have one sample.
  • = NORM.DIST(q, mean, sd, TRUE) computes probabilities from any normal distribution. When applying the CLT, we use the standard error to describe the spread of the distribution. In the case of a mean with known population standard deviation, the standard error is sd/SQRT(n).
Looking Ahead

Now that you understand why sample means behave predictably, the next activities will put this to work. We’ll use the Central Limit Theorem as the foundation for confidence intervals — a way of expressing how precisely a sample mean estimates the population mean — and hypothesis tests — a formal framework for deciding whether observed data is consistent with a specific claim about a population. The intuition you’ve built through this activity will be essential going forward.