How Much Feedback Do You Need Before Launch?

"How many testers do we need?" is the first budgeting question in playtesting and the honest answer is annoying: it depends on what you're asking. The same game might need five testers for one question and sixty for another question. Once you see why, the numbers stop feeling arbitrary and start looking like a plan.

The question determines the number

Feedback questions fall into three rough families and each one has its own reason.

Usability questions: can people figure out the controls, the menus, the tutorial, where to go next. You're looking for problems and the problems that matter tend to be common.

Retention and pacing questions: do people keep playing, where do they stop, is the loop compelling across hours, is the progression system good. You're measuring behavior across many players, and individual variation is enormous.

Bug hunting: how many defects exist and how likely each one is to appear. You're sampling from an inventory you can't fully see.

Why five testers can find most usability problems

There's a classic finding from usability research, old enough to be industry folklore now: a handful of users will surface the majority of usability problems in a given flow. The logic is simple. Big usability problems aren't rare. If the tutorial confuses people, it doesn't confuse one tester in twenty. It confuses most of them. Your first five testers hit the same wall and the testers from six through ten mostly confirm what you already heard.

That's why small usability rounds work. Five to eight testers on a specific flow (the first session, a menu, one boss) produce a fix list longer than you can clear in a single iteration anyway. Returns drop off fast after that. Once the obvious problems are fixed, you need fresh eyes on the new build, not more of the same eyes on the old one.

Two caveats. Five works per flow, not per game: testing the whole game with five people tests nothing thoroughly. And it assumes you're observing the sessions or reading detailed reports. Five unobserved sessions with no notes teach you nothing, no matter how many testers you add.

Why metrics need thirty or more

"Did players keep playing past the tutorial?" is not a usability question. It's a measurement, and player behavior is wildly variable. Your sample includes people who play three hours a week and enthusiasts who play forty. With five testers, one person having a bad day swings your numbers by twenty percent. The average is noise.

You don't need statistical perfection. You need the shape of the curve to stop changing as you add sessions. For most decisions at indie scale, that means thinking in dozens: somewhere around thirty sessions before you trust a funnel, more if you're slicing the data by player type, platform, or difficulty setting.

If thirty sessions isn't realistic for you, and for most small studios it isn't, the practical move is to change the question. Instead of "what's our retention rate", ask "at what point did each tester stop, and what happened right before?" That's qualitative, five to ten testers can answer it, and it usually points at the same fix the metric would have.

Sessions, testers, and what you're actually counting

One clarification that saves studios from bad math: the numbers above count sessions, not unique humans. The same five testers returning for three build iterations are three usability rounds and zero new information after the first, because they've learned your game. Fresh sessions need fresh eyes, which is why a small studio burns through its social circle so fast.

On the other hand, for metrics questions, repeat sessions from the same players can be legitimate data (do the same people keep returning across a week?), but only if you planned for that question in advance. What never works is mixing the two without deciding: treating returning volunteers as if they were a growing sample. They aren't. They're an increasingly biased one, made of people who liked the game enough to come back.

Bug hunting is a different curve

Bugs follow a diminishing discovery pattern: each additional tester finds fewer new defects than the last. There's no magic number, only a confidence level you're buying. For a demo headed to a festival audience, test until recent rounds surface only minor issues. For a paid launch, add the platforms and configurations you actually ship on, because a bug that only appears on a Steam Deck at a small window size is invisible to your desktop test group.

What genre changes

Nothing fundamental, but emphasis shifts. Puzzle and narrative games live on usability and pacing, so modest numbers per act serve them well. Games built on deep systems (4X, survival, roguelikes) turn balance into a metrics question: you need many sessions to know whether a strategy is genuinely overpowered rather than overpowered in three people's hands. Multiplayer adds networking and matchmaking, which are rare failure events, and rare events demand volume plus scheduled sessions so players actually collide in your lobby.

A concrete example of the difference. A city builder studio suspects the early game loses people around the first winter. As a metrics question, that needs enough sessions to see where the curve dips, and even then the curve says where, never why. As a usability style question, it becomes: ask eight testers what they were trying to accomplish right before winter hit, and read ten minutes of reports. Usually the second path finds the fix, and the first path confirms it worked.

Signals you've tested enough

Saturation is easy to recognize once you've seen it:

• New reports repeat things you've already heard, almost word for word.
• Testers complete flows that used to block people, without hesitation.
• A round's fix list is all polish, no structure.
• You can predict the next report before reading it.

When those show up, more testers on the same build won't help. Ship the fixes, change something material, then bring in fresh eyes if the change was big.

A realistic shape for an indie studio

Four focused rounds beat one giant one. Small usability rounds early (five to eight testers). A pacing round in mid development (ten to twenty). A metrics pass in beta if you can muster thirty sessions, and a fresh eyes check just before launch. Different questions, different sizes, same game.

If assembling a new crowd for every round sounds like a second job, that's the actual problem platforms solve.

You can submit your game at PlaytestQuest and pull exactly the number of testers each round needs, instead of maintaining a community all year just to have somewhere to send a build.