Category Archives: Survey Results

Question of the Month and survey results

Average Team Purchases by City

Recently the Puzzled Pint Game Control groups across several cities have had behind-the-scenes discussions about bar booking strategies. As the number of attendees increases, it becomes increasingly more difficult to find friendly and compatible bars. A few cities have dealt with this by constantly using the same bar or cycling through two or three “home bars.” A few others, such as London and Portland, have split Puzzled Pint into two arbitrary groups.

Because most cities try to find new bars each month, we’re always contacting somone who is new to Puzzled Pint. We find a lot of bars misunderstand Puzzled Pint initially. They’ve never heard of it and want GC to reserve a private room or put down a several hundred dollar deposit to guarantee that the players purchase a minimum amount for the evening. We instead try to frame Puzzled Pint more like a trivia night without an MC. It’s a bunch of people eating, drinking, socializing, and solving puzzles. It looks like a regular night to them, albeit a busy one — so it would be helpful to have a few extra folks on staff, but we need no other special treatment.

We thought it would be helpful to share some purchasing stats from around the world. In March, the Question of the Month we presented on the answer sheet asked how much your team spent that evening on food and drink. Not all teams responded, but I’ve noted the count of teams that did.

City Teams Reporting Average Median
Austin 15 (of 21) $43.13 $45.00
Bay Area — Peninsula 11 (of 26) $76.73 $80.00
Bay Area — San Francisco 4 (of 13) $20.25 $20.50
Brooklyn 2 (of 5) $28.00 $28.00
Chicago 16 (of 24) $22.19 $21.25
Phoenix 2 (of 5) $45.00 $45.00
Pittsburgh 1 (of 9) $65.00 $65.00
Portland 20 (of 24) $56.80 $50.00
Seattle – City 18 (of 20) $80.28 $72.50
Seattle – East 26 (of 32) $61.92 $60.00
Montreal 4 (of 4) $81.25 $77.50
Toronto 4 (of 10) $61.25 $52.50
London 10 (of 21) £56.70 £50.00

That the average and median of each city are so close together shows that the average is pretty accurate for that city. There aren’t outliers skewing the average. If you’re a numbers nerd and want to do other sorts of analysis, you can find a read-only version of the source spreadsheet on Google Docs.

City Game Control: Feel free to point bar managers to these statistics.

Players: Don’t forget to generously tip the waitstaff.

Puzzled Pint & Social Interactions

We bill Puzzled Pint as a friendly, social, puzzle-solving event that’s geared toward newbies (as well as experienced puzzlers, especially if they’re happy to handicap themselves with a few pints). The truth is that Puzzled Pint is slightly less social than we make it out to be. Attendees are social within their team and when interacting with Game Control, but teams rarely talk to one another. Over the years, we’ve employed a few “icebreaker” puzzles. They’ve been few and far between and happened back when it was just Portland and a single Seattle site.

This past month, during Wil Zambole’s A-Team set, we had a set of puzzles that required cross-team data gathering and sharing. Wil’s idea gave each team a dossier on one of four A-Team members. Each of the puzzles had a few clues that could only be solved by looking up a personal fact from one of the dossiers. Hannibal’s favorite author. Face’s favorite TV show. Murdoch’s favorite band. B.A.’s favorite flower. Teams had to send a runner out to collect the relevant data.

In the week leading up to the event, the social and interaction angle of these puzzles became quite polarizing behind the scenes. GC in different cities seemed to either love or hate the idea. We ended up allowing regional options — the GC table had a full set of cards, cards hidden around the bar, that sort of thing. Some of the major complaints included:

  • Unfair to early/late teams because there may not be another team present with the correct card.
  • Interruptions during critical solve moments.
  • Accidentally overhearing solve mechanisms from other teams.
  • Artificial bottlenecks that unfairly affect solve times and standings.

On the other hand, many in GC thought that a few minor interactions would bring in something new and novel to the night, possibly bringing us back to our more social roots.

What GC thinks is one thing, but how players feel about the event is what matters most. We added a Question of the Month to help capture player sentiment: How would you rate the interaction puzzles from 1..10 plus room for comments.

At the time of writing, we have data from 13 of 16 locations (no QotM records from Boston & Pittsburgh, no data from DC). I tabulated the data in two different ways. First, we have the raw scores. Only 40% of people wrote in a numeric score. Many more filled in a comment. Because of the disproportionate responses between values and comments, I attempted to rate the comments as positive, negative, or neutral. They broke out like this:

  • Positive
    • If the team gave a score greater than 5
    • If the team gave no score, but a positive comment
    • If the team gave a low score but specifically indicated in the comment that they liked the concept but not the execution
  • Neutral
    • If the team gave a score equal to 5
    • If the team gave no score and no comment or an irrelevant comment (“I like turtles”)
  • Negative
    • If the team gave a score < 5
    • If the team gave an entirely negative comment

Globally, we had 236 responses. For the numeric 1..10 answers we had an average of 4.7 and a median of 5. This indicates that the positives and negatives were split pretty evenly. There weren’t outliers skewing the average out of proportion. The results looked like this:

distribution

When you graph the positive/neutral/negative range of the write-in comments, it shows that a lot of people didn’t mind and that there was a fairly even split between the likes and dislikes.

positive_neutral_negative

And if we break it down by city, the numbers look like this:

Average Plusses Neutrals Minuses
Austin 5.7 10 4 5
Bay Area, Peninsula 5.2 13 7 7
Bay Area, SF 3.7 9 5 10
Brooklyn 7 3 7 0
Chicago 5 3 20 3
London, Bubble 4 10 1
London, Squeak 3 6 2
Montreal 3.8 1 2 3
Phoenix 6.1 2 1 2
Portland 5.2 3 17 2
Seattle, City 4 7 15 7
Seattle, Eastside 3.8 5 6 13
Toronto 5 2 12 1

I’m not sure what city-by-city conclusions to draw from that, but thought it would be fun for each city to see how their responses compared to others.

Feedback

What do you think of puzzles requiring social interaction? Have you run into any at a previous event, such as DASH or an earlier Puzzled Pint, that you particularly enjoyed? Although we want to aim for no more than one or two “icebreaker” months a year at Puzzled Pint, how might we improve the player experience for this style of puzzle?

Portland Survey, 2015

In addition to the Question of the Month on your answer sheets, cities occasionally use local questionnaires to learn about attendees. It’s been a year or two since we had one in Portland so we thought it was time to collect a little more data. We handed out double-sided quarter-sheet surveys, one to each attendee, that looked like this:

survey

Let’s just go through the answers in order, shall we? Starting with the first, we’ll look at the experience of our players.

experience

As you can see, we’re mainly veterans and newbies. This has always been one of our biggest challenges, with regard to both attendance numbers and difficulty. Of course, we want to encourage new members to join the community — that’s one of the core tenets Puzzled Pint was founded upon. On the other hand, the influx of new people have pushed our numbers past 100, making bar scouring extremely difficult. We’ve since tempered this by splitting into Bridgetown and Stumptown sub-groups, each with their own location.

The mix also means we have to be careful with puzzle difficulty. We don’t want to scare away the newbies with puzzles above their difficulty level, yet we also don’t want the long-time regulars to get bored. We’ve skirted around this by managing expectations — maybe not going so far as saying “if you think this is too easy, go start your own intermediate-to-advanced puzzle event” but at least by imposing this as a newbie-friendly event. Other events, such as DASH, can produce different sets of puzzles for teams with varying difficulty levels. This is difficult for Puzzled Pint since we print everything ahead of time. We’ve kicked around the idea of puzzles that can be altered post-printing (such as marking flavor text with a highlighter), but so far the idea hasn’t gotten much traction.

Portland is a small “big city” and our travel distances and methods reflect that. The vast majority of people traveled less than 5 miles and less than half came by car. That bikes and public transit were big contributors wasn’t much of a surprise. I was surprised that almost a quarter of respondents walked.

distance

transit

The next question, “how likely are you to recommend Puzzled Pint to a friend” is a traditional NPS survey question. Marketing and support folks may know and understand this number, most folks don’t. Suffice it to say that most organizations do poorly. Beloved companies like Apple, Trader Joe’s, and Amazon do well. Many companies get close to zero, or even negative numbers. (The scale goes from –100 to 100.) We got a quite reasonable 69. Admittedly, there may be some selection bias here. Only people that took time out of their Tuesday night to come solve puzzles were able to respond, so the folks that don’t like us enough to not show up (and would likely also not tell their friends) would not be represented.

NPS_Result

And someone had difficulty letting go of friends that ascended into Game Control:

abandonment_issues

Next up is the neighborhood question. This helps us focus on where to look when scouting bars. With 80–100 people, there are far fewer choices. Now that we’ve split up into two locations, more options have opened. I particularly enjoyed the written-in “LOL” next to one person’s Beaverton answer. Although a non-zero percentage of folks said they’d go to Beaverton for Puzzled Pint, none of the current GC are interested due to distance and transit. (That being said, if you’re in Beaverton and want to start up a chapter out there, please don’t hesitate to email us.)

Beaverton-LOL

portland_survey_map

The vast majority of responses wanted downtown or northwest, which makes sense. It’s central to everyone. For reasons mentioned above, and despite the numbers, I think we are going to ignore the St. John’s and Beaverton suggestions. Both numbers are much higher than I expected, but our team isn’t terribly interested in the awkward travel. It looks like we probably won’t be going past 50th. The NE and SE quadrants got a lot of hits, though I personally didn’t expect the NE distances to be quite that far out.

On the difficulty front, we got average-to-intermediate, which is roughly the range we’re trying to hit. A few of the responses singled out some months as particularly difficult in margin notes.

difficulty

We’ve collated the bar suggestions into a spreadsheet and are actively investigating them during our scouting runs. So thank you for those responses! You’ll see some of those suggestions being used over the next few months.

And finally… kittens or puppies. About a third of you just couldn’t commit. You checked both and scribbled over the “or” with an “and.” Alas, as inclusive as you’d love to me, that answer just doesn’t help one side or the other. We had to pull you out of the tallies. A couple of you had strong affinities for puppy breeds:

corgis

huskies

A couple of you are full of negativity toward cats:

kick_them

no_no_die

Ultimately, puppies won out over kittens by two votes:

kittens_vs_puppies

I’d really expected Portland to be a kitten town. Thanks for letting me down, Portland!

That’s it for now. These are the simple, isolated stats. I’d like to work out some correlation across questions soon. Does your longevity in Puzzled Pint influence what you said for puzzle difficulty levels? Are the folks with a higher travel distance also the Beaverton and St. John’s voters? Are walkers/bikers predominantly dog people? If you’d like to run your own numbers, you can download a CSV of the survey results and go to town.