The Definitive Guide to Playtest Questions

 insights

The goal of post-playtest ques­tions is to get unadul­ter­at­ed feel­ings from the per­son. You do not want to lead them in any way, but you want to find out what they thought. Try these questions: 

  1. What was the most frus­trat­ing moment or aspect of what you just played? 
  2. What was your favorite moment or aspect of what you just played? 
  3. Was there any­thing you want­ed to do that you couldn’t?
  4. If you had a mag­ic wand to wave, and you could change, add, or remove any­thing from the expe­ri­ence, what would it be? 
  5. What were you doing in the experience? 
  6. How would you describe this game to your friends and family? 

The easy to remem­ber (though ridicu­lous) mnemon­ic is ffwwdd:

Frus­trat­ing
Favorite
Wanted
Wand
Doing
Describe

The order is on pur­pose. And yes, you may get some of the same respons­es, but often you don’t, and those sub­tle dif­fer­ences can be telling. Some­one might say X was the most frus­trat­ing but then what they want­ed has noth­ing to do with it, or their wand changes some­thing else. All are good ways of get­ting input, which then you will have to decide what to do with the infor­ma­tion- but at least you’ll have it. 

And num­ber 6, describe’, is very inter­est­ing. If they describe your game as a puz­zle game and you thought you were mak­ing a sto­ry game- where is the disconnect? 

Always keep in mind that your goal is to ana­lyze the results and deter­mine what actions to take. Not all prob­lems need a fix, but if you note obscure issues and then you start to see them, or vari­ants, more often, you’ll be able to piece togeth­er trends that may need addressing.

When in doubt, dur­ing or after a playtest, if they ask a ques­tion, you can always just ask it back to get them thinking: 

Tester: What should I do with this laser?”
Me: What do you think you should do with that laser?”

Below are some answers to playtest relat­ed ques­tions I have giv­en over the years:

1. What form(s) of feed­back do you col­lect from playtesters? (e.g. sur­vey, focus group, 1-on-1 inter­view, playtesters voice thoughts while play­ing, etc.) If you col­lect mul­ti­ple forms, which form do you think gives you the most use­ful information?

I have used all those forms in the past. Some­times the effec­tive­ness depends on the stage of the project and the feed­back you need the most.

  • Sur­veys are good for base infor­ma­tion and com­par­ing a bunch of answers to sim­pler questions. 
  • Focus groups are good at ear­ly stages when you want broad opin­ions on your project. 
  • 1-on-1 inter­views are the best (and, ide­al­ly, required) after a playthrough of the experience. 
  • Voice thoughts and (even bet­ter) video while play­ing are super use­ful for team mem­bers who can’t be there or for review­ing after a sur­vey or a 1-on-1 has dug up a few outliers. 

The most use­ful (in my opin­ion) is the 1-on-1 interview.

2. If you have playtesters fill out sur­veys, do you pre­fer open-end­ed ques­tions or close-end­ed (e.g. lik­ert scale)? How come?

Again, it depends on the data you want to get. If you’re com­par­ing dif­fer­ent parts of a game, rat­ings may be fine. How­ev­er, we most­ly use open-end­ed questions.

3. If you have playtesters fill out sur­veys, do you have them fill out any ques­tions before playtest­ing so that you can assess changes in vari­ables like mood, ener­gy lev­el, etc.?

Main­ly the pre-playtest ques­tions are get­ting used to the process and being com­fort­able. Gener­ic things like age, cur­rent play habits, cur­rent games, etc. are captured. 

4. If you have playtesters par­tic­i­pate in a focus group, do you have any strate­gies for reduc­ing the degree to which an individual’s response bias­es the respons­es of the oth­er focus group participants?

If we do a focus group it’s always at the end after any playthroughs, sur­veys, or 1-on-1 interviews.

5. If you have playtesters par­tic­i­pate in a focus group or 1-on-1 inter­views, what are, in your opin­ion, the most impor­tant ques­tions to ask?

See FFWWDD above. :-)

6. If you have playtesters voice their thoughts while play­ing, how do you ensure that their play­ing expe­ri­ence doesn’t feel too arti­fi­cial or con­trived since usu­al­ly they wouldn’t be nar­rat­ing their own experience?

I encour­age them to speak stream of con­scious style. Most peter off and just focus on play­ing. That in itself is telling of course ( ooh, your game is inter­est­ing enough!” ) Some keep it up the whole time, but either way I just let them play.

7. What specif­i­cal­ly are you look­ing for dur­ing playtest sessions?

Always go into a playtest ses­sion with a pri­ma­ry goal. 

  • Does this tuto­r­i­al teach what it needs to? 
  • Is this weapon overpowered? 
  • Did they find their way through the level? 

But always be ready and will­ing to take notes about what­ev­er comes up. You nev­er know what you’ll learn!

8. In your opin­ion, what are a few com­mon do’s and don’ts’ that peo­ple who run playtest­ing ses­sions should be aware of?

  1. Do: Put the playtester at ease. Let the play­er know that they are help­ing you. If some­thing breaks or is con­fus­ing, it’s your fault, not theirs. 
  2. Don’t: Help them through tough parts, or if they get stuck and ask. Tell them ahead of time that ask­ing ques­tions is great! But you prob­a­bly won’t answer them; you’re inter­est­ed in see­ing how they respond and play. You’re not try­ing to be rude, you just want to see how they play unaid­ed. If they do get real­ly stuck and it’s because of some­thing you know you’ll fix or change, note where it hap­pens and exact­ly what you said as a hint or prod­ding. Keep that phras­ing and tim­ing through­out those playtests so your data is use­ful and so you have a good idea of how to fix it for the next build. 
  3. Do: Always make it clear at the begin­ning that they can play for as long or as short as they’d like. (If they want to quit out, that’s great data! Keep a record of where and when it happened!) 
  4. Don’t: Ask lead­ing or too-spe­cif­ic ques­tions! You want to see what they thought or remembered:

Exam­ple 1- The Puzzle

Bad: Did you not under­stand the tree hint?”
Bet­ter: Was there any­thing spe­cial about the tree puz­zle?”
Best: What could have helped you solve the tree puzzle?”

Exam­ple 2- A Trap

Bad: You died a lot at the 3rd trap, what’s up with that?”
Bet­ter: Was the 3rd trap too hard?”
Best: What did you think of the 3rd trap?”

Exam­ple 3- Pow­er­ing Up

Bad: What gave you speed boost?”
Bet­ter: What did the red pow­er-up do?”
Best: What pow­er-ups do you remem­ber? What did they do?”

Keep On Playtesting!

Fol­low­ing FFWWDD will help you and your team design post-playtest ques­tions that will get unbi­ased feel­ings and thoughts from the playtester.

Need more? Down­load this free playtest­ing check­list.