Once the class gets the idea, you play the sneaky form of the game. You give them the instances of your category, say George Washington, Abraham Lincoln, and George Bush. They then guess presidents, like Thomas Jefferson, and you say yes. Then they all guess that the category is "presidents". And no, that isn't the category. (Actually, since the teacher was there and not paying close attention, I had her guess the category and be wrong. And I explained that everyone guesses presidents.)

The Confirmation Bias

The "confirmation bias" is the tendency to guess only things that fit the category, and then to assume that you know the category just because all of your guesses fit the category. Apparently it is very robust -- without suspicion, people naturally fall for it.

It isn't so unreasonable. If the members of the category Americans were chosen randomly, it is very unlikely that they would all be presidents. If I am trying to communicate the category of Americans, I should not choose three presidents. So, in normal life and normal communication, it is very reasonable to think that the category is presidents.

Of course, it is still unwise to guess just members of the suspected categories.

Thinking Mathematically

Someone guessed "Franklin". I think they meant Benjamin Franklin; they probably thought Franklin had been president. "Franklin" was a yes. The students ignored this. Maybe they thought there was an obscure president named Franklin. Maybe they thought the yes referred to Franklin Piece or Franklin Roosevelt.

I think the human brain was built to find patterns in noise. To the students, the yes for Franklin was noise, completely swamped by the overwhelming pattern they were perceiving -- that almost all of the members of the category were presidents.

In real life, it pays to ignore noise. As noted above, maybe there was a president named Franklin. Maybe they just heard wrong. Maybe I made a mistake in my answer -- maybe I heard wrong, maybe I answered too quickly without thinking, etc.

But in mathematics, a single exception to a rule means the rule is wrong. A single exception is not treated as noise, it is treated as important information.

In real life, you would grow crazy (or be very cynical) if you rejected rules and patterns because they seemed to have occasional exceptions. But ignoring exceptions has its dangers too, and it is useful to think about exceptions.

What Next?

I pointed out that Benjamin Franklin was a yes, so the category couldn't be presidents. I think that didn't help -- I think no one got it or learned anything from that.

The class was now stumped. So I asked for crazy guesses (to which I would answer yes or no depending on whether they were members of the category). This arguably is part of brainstorming.

The students then fixated on colonial times. I think this was because, when they guessed presidents, they mostly guessed presidents from colonial times. Essentially, they repeated their mistake with a broader category -- most of the students guessed people from colonial times, and then they thought that confirmed their idea that the category was people from colonial times.

However, there were several noncolonial yes's, such as Albert Einstein. And Bush was on the blackboard as one of the original category members. So the students were doing some serious ignoring of contradictory data.

And there were enough noncolonial yes's that some students could see the potential problem. I think I let them guess people from colonial times, which was a no. Eventually they found my category -- Americans.

Next, I started them out with apple, orange, and banana. People made guesses. Many guessed fruits and got yes's. Others however, we now on track and gueessed nonfruits, often getting yes's. (My category was plants or parts of plants.)

After they had asked their yes-no questions, I then solicited their guesses and wrote them on the board. Those guesses included fruits, fruits and vegetables, plants, food, things that you grow, and edible things.

(I was going to clump edible things with food, but then I realized that there are edible things that we don't eat. I don't know if that was relevant or not to what came next.)

Then I asked what guesses we could make to discriminate between the possible categories on the board. That exercise worked better than anything else I did that day. Only some students could do it, but those were getting it.

The final two contenders were plants and things that you grow. "Turkey" was a "no", but the majority vote was that turkeys are raised, not grown. So that did not discriminate the two possibilities. Then one student suggested those plastic things that you buy, put in water, and they grow. They were a no, eliminating "things that grow".

Teaching Goals

I suppose there is also a lesson about avoiding the confirmation bias -- realizing that a category could be broader than the examples, and testing if it is. To me, this is not particularly important.

Finally, and perhaps most importantly to me, the actual process of playing the game involves tinkering. You start with a hypothesis, but you can improve on it as you collect more data. The game points out how different hypothesese are possible, and how to choose between them.

Exercise: Defining Tinkering

Exercise: Testing

Tinkering Intro