The Oxford Maths Festival returned this year and it was tons of fun, at least for this volunteer! I failed to take pictures, but a few opiglets were involved: Flo and company took their VR work for the Ashmolean Dimensions exhibit and demonstrated it at Templars Square, and Conor did a spectacular job pretending to be a police constable for the maths escape room.
Last year Mark blogged about how we demonstrated the German Tank Problem at the festival. I thought this time round I’d share another of the Mathematical Mayhem activities: a game illustrating biased sampling.
The problem
Complete, error-free data is the dream, but in reality we do sampling all the time. More often than not we happily convince ourselves that the samples we work with are representative of the whole unobserved population, but that’s not always the case.
Suppose we want to know something, e.g. how much exercise the average opiglet does. Rather than stalking each and every group member (ill-advised, and also kind of illegal), we can just ask a bunch of them about their exercise habits and base our estimate on the responses. There is a slight issue, however. If you’re like one of the sportier members of our group, you might be quite forthcoming about your daily routine. And if you’re a lazy so-and-so like myself, your responses might be a little more vague, or altogether absent. People who are keen to talk about exercise also tend to do more exercise, thereby making our sample biased.
If we want to estimate how much exercise the average opiglet does, we should probably go a little under our observed sample average. Or a lot under if our data is collected exclusively from Blopig posts by university rowers.
The activity
Sample bias can arise due to a range of reasons, including self-selection (i.e. who answers our questions) and inspection (i.e. who we choose to question). It’s inspection bias that we looked at during the Maths Festival. We presented each participant with a non-transparent cloth pouch with twenty-five marbles in it. There were two types of marbles in the bag: the majority were of your standard small glass variety, but a few were significantly larger and heavier. The aim was to guess their total weight.
Participants were allowed to draw five marbles out of the bag and weigh them. They could take a number of such samples before making their guess. Since at any one point we weigh a fifth of the marbles, the obvious thing to do is to estimate the total weight as five times the observed sample weight (or five times the average of a few observations). If the samples were fair, that would certainly be a good tactic.
However, when people tried this they often came up with numbers that were far too high to be even vaguely plausible—sometimes as much as three times the actual weight of the bag. Did they get their multiplication wrong? Were the scales broken? What inevitably happened—even to me, and I knew the ratio of large to small marbles in the bag—was that people took out more than their fair share of large marbles, resulting in uncharacteristically heavy samples.
The take-home message
So how do we make more accurate guesses? There’s two pieces of information we need: firstly, the weight of each type of marble and secondly, the ratio of small to large marbles in the bag. Resampling can help with both of these.
It only takes two different samples to figure our the individual marble weights, which most of our participants immediately set out to do. Guessing the ratio is a little trickier, and relates to population size estimation.
Suppose that you draw three or four samples, and you pay attention to the large marbles only. You might make a note of what their colours are, or you might cheat a little and mark them with a sharpie the first time you draw them. If you draw the same two or three large marbles again and again, then there probably aren’t all that many large marbles in the bag. However, if each of your samples contains totally different large marbles, you’d guess their overall number is higher. This line of thinking is formalised by a method called “mark and recapture”. It’s employed by ecologists, who tag animals in order to estimate and track population sizes. If you want to read more about it, you could go to Wikipedia, or you could check out the much more amusing Do you feel lucky? explanation. Once you’ve guessed how many big marbles there are in the bag, estimating the total weight is easy. You just need a calculator or a patient, numerically literate friend.
Final comments
In real life bags are often of unknown size, marbles may have totally different weights to each other, and the source of bias might not be so obvious as in the example above. This is an incredibly common, yet at the same time often unintuitive, problem we face in data analysis. I liked the marble exercise because it showcased biased sampling in a clear, tangible way, and let people come up with their own workarounds.
The Maths Festival, as I mentioned at the start, involved a wide range of other activities. If you’re curious, you can check out the website, or look them up on your favourite social media platform (provided you’re not too hipster and your favourite is either Facebook or Twitter). I look forward to seeing what the festival looks like next year!
