Tuesday, March 17, 2015

The Post Hoc Fallacy in Playtesting

This morning, my cats would not stop meowing when I was trying to feed them. (Even if you’re reading this on a different day than I published it, I guarantee that my cats meowed me to just about  the point of madness this morning.) Someone once told me that they do that because they learned it from meowing at their adorable kitty mommies when trying to get food as kittens, and the behavior just stuck.  I don’t know if that’s true; it makes me want to feed them less.

Every. Damn. Morning.

My cat, without knowing it (because cats don’t understand logical fallacies), is engaging in what philosophers and debate kids would call the post hoc ergo propter hoc fallacy. Wikipedia tells us that fancy Latin phrase literally means “After this; therefore, because of this”. It’s any time we claim that an event caused a reaction simply because it came before it.  I carried this rock around, and then I wasn’t attacked by tigers. Ergo, the rock must protect me from tigers!  (This logical fallacy is also called a "false cause" fallacy or "affirming the consequent".)

That said, it doesn’t mean that nothing that comes before can be said to cause what comes next. “Yes, I first stuck my face in boiling oil. Yes, now my face is gone. Connection? Naaah!” What the fallacy tells us is that we can’t automatically assume a causal relationship because one thing comes first. It makes sense that putting boiling oil on my face would lead to severe burns because hot things burn skin. I learned that at the age of four because my favorite color was red and I liked to grab things.

Actions do have consequences.

In game design
On the surface, the fallacy doesn’t seem like it applies to games. In games, things happen after other things because those things caused it to happen. When I roll a 7, I move the robber because rolling a 7 causes the robber to move. There’s no post hoc fallacy there; based on the rules of the game, one event causes another event.

Where I’ve seen this, though, is in playtesting. The purpose of games is to entertain people (however you define game and however you define entertain). So, when I’m playtesting games, that’s my question for myself: Is this game fun in the ways that I want it to be fun?

People who like games have fun playing them. It’s easy to entertain board gamers if you’re a reasonably competent designer. But just because people have fun while playing your game doesn’t mean that it’s complete. The question I think designers should ask themselves more often is, “Are these people having fun because the game is fun – or are they just having fun?”

Over the summer, I was playtesting games for a contest I ran, and not all of the submissions were great. (In their defense, it was a 24 hour contest.)  To help me test them, two friends came over and played some games that ranged from almost there to almost unplayable. We were laughing and having a good time, but we didn’t have the most fun with the best game. In fact, in one of the games I liked least, one of the characters reminded us of our kung fu instructor, so we spent the game reading all the text in his voice. Also, we had a bit to drink. Had the designer been there, he might have thought that playtest was going well. It wasn’t…but we had a lot of fun.

Monopoly: Fun on its own since never.

So, while whether or not people enjoyed themselves while playing the game is pretty important, we need to be able to point to causal evidence that would suggest that people enjoyed the game because of the game, not because of the company, the drinks, or because some of the card art looks like the players’ Sifu.

Testing for Causation
Most playtesters will be honest about this. If the designer had been there and asked us what we thought, we would have told him. And most experienced playtesters will do that. But inexperienced ones might not. When working on Gothic Doctor, one player told me that the game was a little bit slow, but that was okay because he still really liked it since looking at the art kept him entertained. That’s not okay: the novelty of the art will wear off quickly on repeated plays.

So, we need to be clear about why people liked the game if they liked it – just as we need to know why they didn’t like it if they didn’t. People are good at asking what someone didn’t like about a game – but we need to get better at drilling down into what really makes the game good. The theme was fun? That wears off. It was cool because it took them a little while to figure out their strategy? That’s great, but was it still fun once they did?

If you get good answers to those questions, then think that people had fun because of your game – not just that they played your game and also happened to have fun.

The Inverse Is Also Not True
In logic, there’s the principle that if an if/then statement is true, then its inverse must be true. For example, if the statement, “If I jumped out of an airplane without a parachute more than five minutes ago, then I will be dead” is true, then “If I am not dead, then I did not jump out of an airplane without a parachute more than five minutes ago,” must also be true. The same holds true if the if/then statement is false; the inverse of the if/then statement must then also be false.

Applied here, that means that we also can’t say, “If someone did not have a good time, then the game is not fun.” There are all sorts of reasons some people might not have a good time. I watched a game of Gothic Doctor take an hour and a half for three people because one of the players spent more than five minutes every turn agonizing over the cards in his hand before making any decisions.  Consequently, the other two players – especially the one who wasn’t his friend – didn’t have a good time. At all.  Really, that’s not a problem with the game, since it’s not an AP-prone game. And it definitely isn’t the case that he didn’t have fun because the game wasn’t fun.

"You roll. Then you move. It's not complicated, Dad."

Plus, sometimes people are just having crappy days and don’t really want to play a game. I’ve seen that happen with the best games out there. The first time I played Gil Hova’s The Networks, I was miserable – since it was 1 in the morning and I knew I should go to bed, but goddamnit I wanted to play one more game. I just wasn’t prepared for the thinkiness or time commitment of it at that hour – but I’ve played it since and it’s awesome. I was just too tired to have fun.

As Louis CK once said, “I hear people say their phone sucks. Your phone doesn’t suck. The [crappiest] phone in the world is a miracle. Your life sucks. Around your phone.”  It’s entirely possible that your game doesn’t suck – it might just be that your game wasn’t amazing enough to make someone’s [crappy] day better, so their day sucked around your game.

Still, your game should make people happy to play it (unless it’s just not a kind of game that they like). If people are consistently not having fun playing your game – especially if they sit down smiling and leave after texting for twenty minutes straight. I’d suggest that if you can’t think of a very, very good reason they didn’t like your game, then assume it’s a problem with the game. The worst that happens there is that you’re too hard on your game and try to make it even more fun, which you should be doing anyway.

So, remember two things from this very long article. First, just because someone plays your game and happens to have fun doesn’t mean the game’s perfect. And vice versa. Second, if anyone finds the mute button on cats, I will make a board game for you and about you to celebrate your achievement.

Doug Levandowski is a game designer who co-created Gothic Doctor. He has other designs in the works, too - because that's what designers do. When Doug's not designing, interviewing, writing articles, or sleeping, he's teaching English to a bunch of amazing high schoolers. They're working on Macbeth right now, which is his favorite play to teach. You can find him on Twitter at @levzilla.