Who do you ask?
How many?
Each survey you run adds a dot below, showing the percentage it found.
Things to try
- Before showing the true answer, run a random sample of 25 a few times. How confident would you be reporting any one of those results?
- Clear the chart. Run a random sample of 25 fifty times, then a random sample of 1,600 fifty times. Compare how wide the two clouds of dots are.
- Going from 100 to 400 people is four times the work. How much narrower does the cloud get? What about 400 to 1,600?
- Now ask 1,600 students in the library, fifty times. Show the true answer. What went wrong, and would asking 5,000 fix it?
What you’re seeing
Every sample is a little off, by chance. That’s sampling error. It isn’t a mistake; it’s what happens whenever you ask some people instead of all of them. With random samples, the dots pile up around the true answer, and most land close to it.
Bigger samples help, with shrinking returns. To cut the typical error in half, you need four times as many people. That’s why national polls usually ask around 1,000 to 1,500 people: the margin of error is about ±3 points, and asking thousands more would only shave off a little.
Who you ask matters more than how many. Students in the library on a Friday night are less likely to be working long hours, since many working students are on shift. A sample like that is biased, and making it bigger just gives you a more precise wrong answer. Researchers call a sample you pick for ease a convenience sample.
This has happened for real. In 1936, the Literary Digest magazine mailed out about 10 million ballots, got roughly 2.4 million back, and predicted that Alf Landon would beat Franklin Roosevelt easily. Roosevelt won in a landslide. The magazine’s list came from telephone directories and car registrations, which skewed toward wealthier people during the Depression, and the people who mailed ballots back differed from those who didn’t. George Gallup called the election correctly with about 50,000 people.
Random doesn’t mean haphazard. Stopping people “at random” on the quad is still a convenience sample. A random sample needs a list of everyone and a way of picking that gives each person the same chance.
Talk about it
- A poll of 1,000 people says 52% support something, with a margin of error of ±3 points. What can you conclude? What can’t you?
- An online poll gets 200,000 responses. Why might a careful survey of 1,000 be more trustworthy?
- Who’s missing from surveys done by phone? By email? At a mall? How could each group’s absence change the result?
- Even a perfect random sample has a problem if many people refuse to answer. Why?
About this page
The campus and its numbers are invented for this simulation. Each survey is a fresh set of simulated students drawn by your browser, so your dots won’t match anyone else’s. The margin of error shown is the standard one pollsters report: the range that 95 out of 100 random samples of that size should fall within, calculated for a 50/50 split.
Source for the 1936 poll: Squire, Peverill. 1988. “Why the 1936 Literary Digest Poll Failed.” Public Opinion Quarterly 52(1): 125–133.
How to cite this page
Sharp, Gwen. 2026. “How Many Do You Need to Ask?” https://gsharpokie.github.io/samplingsim/
Spot an error? Suggest a correction (needs a free GitHub account).