Thursday, December 3, 2015

Objective/Subjective Note Taking

In the past, one of the game-design-related things that I wasn’t good at was everything. But, two of the specific parts of everything that were particularly problematic were keeping the two halves of my playtesting notes (the subjective and the objective) separate and recording enough detail.

Definitionally, by objective, I mean the facts about the test. What was the score at the end? How long did the game take? Who went first? What did players say during the game or during feedback? How many times did players check their phones? In short, things that, if multiple people watched a video of the test that captured everything, they would necessarily agree about having happened.

By subjective, I mean everything else. Do I think that Joe didn’t like the scoring because he lost due to his bad decisions? Do I think Becky didn’t enjoy herself even though she said she did? What ideas does this playtest give me about the game, either in the moment or upon reflection?

The two are quite different – and in ways that might not be immediately apparent, though that’s a larger issue than this article could delineate. Rather, my plan is to outline how I realized I needed a better method for separating objective and subjective commentary, what I started using, and how good the results have been so far.

The Problem
About a month ago, I playtested a game that my buddy Ben Brown and I are working on called (for now) Framed, and it’s the most complex one either of us have done yet.
Whoa, whoa... It's more complex than UnPub!?
Much of what makes it more complex is the number of moving parts in it. Whereas You're Fired has just a few core mechanics that didn’t change too much over the course of playtesting, Framed has quite a few – and they’re often changing. So, whereas for You’re Fired, I could jot down, “Bob liked the Manager,” for Framed, I’ve been having trouble remembering what the exact rules I played with were during much earlier games – and my old notes weren’t much help.

But, the moment that made me realize I needed to change my system was when one of my playtesters said, after a very short game, “Wow. That game was shorter than the explanation of the rules.” I immediately said that I didn’t think it was – and then realized that, in truth, I didn’t know. In my notes, I wrote down what he said and after it, “I think he’s wrong.” But in reading it over, I just don’t know – and my subjective response is a bit too blended with what he objectively said.

Also, I have a comment in that day of playtesting that just says, mid-game, “Well, that doesn’t work.” I have NO idea what I was talking about there – much less am I able to explain anything to my co-designer. Those just aren’t helpful notes.

It’s more than writing down when the game starts and saying what isn’t working, though. The issue was that I wasn’t taking notes that were nearly detailed enough – and I wasn’t separating what happened and what I thought about what happened carefully enough.

"But I can still change!" the old Dougie-Poo said. "It can be like that moment the Grinch stopped the sled!"
The Solution
When I was learning to teach, my advisor taught me the best way to take classroom observation notes I’ve ever come across, and I’m a bit embarrassed that I didn’t think of applying it to playtesting sooner.

The system is deceptively simple: take a piece of paper, fold it to make two columns. The left column is the “objective” column, where you write down everything that happened exactly as it happened. This includes what other people say, even if those are their subjective opinions – because it’s a fact that they said it.

The right column is the “subjective” column, where you record any opinion-based responses that you have, either in the moment or upon reflection. Because the page is folded, you’re able to connect the objective element with the subjective response quite clearly, starting them on the same line. Simple as that.

It looks like this. No jokes here. Move along.
The Benefits
I’ve been using this system for a few weeks, including during the three-and-a-half day playtestfest that is Metatopia, my favorite con of the year. Here are four benefits this system has brought me as a designer – and two drawbacks.

1. The objective and the subjective are clearly separate.
Not surprisingly, I’ve had a much, much easier time keeping what happens and my opinions of what happened separated, and I’ve had a much easier time connecting the two because of both the level of detail and the fact that the subjective reflection starts on the same line as the objective source of that reflection.

While it might not seem surprising that it does well the main thing that I’m started using it in order to do well, I’ve taught high school and designed games long enough to know that intent doesn’t always equal results. But this one seems to.

2. I’ve started recording more detail.
Part of moving into the mindset of recording the objective has led me to record much, much more than I used to. Simply knowing that I should be writing down what’s happening objectively has caused me to record more. A lot more.

As I learned in student teaching, you never know what’s going to be important. The same is true in playtesting. You won’t know if something small is a trend unless you’re keeping careful notes.

3. I’ve been more engaged in the playtests – even when I’m not playing.
Because I’m recording so much more, I now know what to do while people are playtesting my games. Before, I would either be a little bit bored just watching – or be that guy with his phone out during his own game.

I call it managing my social media presence, which makes me even more obnoxious.
Now, I’ve got something to occupy me: insanely detailed notes. I’m engaged in my playtests because it isn’t an okay-I’ll-be-on-Facebook-if-you-need-me thing. It’s a I-need-to-make-sure-I-get-everything-written-down thing.

4. It helps me see the forest AND the trees.

Easy. Trees on the left, forest on the right.
In my other notes, I realized that I was often seeing the proverbial forest but missing those equally proverbial trees. Overall impressions, even in games with fewer moving pieces, only get a sense of the overall. The smaller pieces are important, too, and without careful notes, it’s easy to miss the minutiae.

Now, the left column asks me to look minutely, while the right column asks me to synthesize and look broadly. In short, to the extent this is possible, I’m proverbially looking at each tree and then putting them in the context of the forest.

The Drawbacks
While I’ve found that the benefits outweigh the drawbacks of this system, I’d be lying if I claimed that it’s unmitigated perfection. Most simply, it is a lot of work, and it uses up a lot of paper. No way around those two issues – and I knew those would be factors when I started. But, there were a few problems that popped up that surprised me a bit.

1. I sometimes have to pause the game to take notes.
Some of this is just getting used to recording more and faster. Like any skill, this will get easier with time. But when I’m trying to write things down, I sometimes have to slow some moments of gameplay – especially games with end-of-game scoring – in order to get all of the details.

This isn’t so bad at the end of the game, but in a trick-taking game I’m working on where scoring happens each round, I’ve gotten the sense that some players want to move on to the next round rather than have me write down all of the conditions at the end of the round that gave them the points they had.

Besides, I think, since the purpose of playtesting is to improve the game, it’s okay to slow down gameplay to get better information. While the goal is to improve the game and make sure it’s fun, I’m okay with taking away from that fun temporarily to get the facts right – and ultimately make it more fun in the future.

2. It doesn’t work if I’m playing.
The games I design these days don’t have enough downtime for me to take these extensive notes when I’m playing – so I’ve had to choose between playing and taking these kinds of notes. It’s not a bad choice to make, and I’m starting to come to the belief that designers should be watching others play their games rather than playing them yourself, especially in later stages of playtesting.

That, for me, works, though. I think these notes are best for late-stage playtesting when you’re trying to fine tune rather than make broad changes that are obvious even from superficial observation.

I’m not saying this way is the only way to take notes – but it is one way. I’ve found it very, very helpful, and I’m planning to continue with it. If new issues arise – or new benefits pop up – I’ll update the article. If you have any questions, as always, ask on Twitter, on Facebook, or in the comments – though I don’t get notifications about comments, so that’s probably not the best way. And, of course, if you start using this system or one based on it, I’d love to know about it! (FinallyIhaveaKickstarterrightnowpleasebackitthankssomuchitsagreatgamesorryfortheshamelessselfpromotion.)

Doug Levandowski is a game designer who co-created Gothic Doctor, UnPub: The UnPublished Card Game, and created You're Fired (on Kickstarter at the time of publication). He has other designs in the works, too - because that's what designers do. When Doug's not designing, writing articles, sleeping, or playing Star Realms on his phone, he's teaching English to a bunch of amazing high schoolers. They're working on The Scarlet Letter and All Quiet on the Western Front at the time of publication. You can find him on Twitter at @levzilla and on Star Realms as DougLev, where he'd love to lose to you.