Ask Angry: How to Playtest an Entire System on Three Hours a Week

Print Friendly, PDF & Email

February 21, 2020

Ask Angry is a weekly column wherein The Angry GM answers one reader-submitted question with no-nonsense advice, painful truths, and plenty of abuse. If you’re a masochist and want The Angry GM to answer your gaming question, e-mail ask.angry@angry.games. Keep it brief and make sure you tell The Angry GM exactly what to call you.

Ash asks…

I am creating a homebrew TTRPG system meant to be a D&D that focuses more on exploration and mystery rather than combat. I’ve almost finished my second draft (after some rewrites suggested from my playgroup), and now all I have left to do is playtest it. My problem is that we only play once a week for around 3 hours. How do I efficiently playtest my system so I can utilize my limited time well?

Thanks for the question, Ash. And thanks for giving me a nice, normal name to use. I’m so grateful that I even won’t snark about your name at all. And I won’t make any Pokemon jokes either. I’m sure you’ve heard them all anyway.

Let me start by saying that I’m not actually a professional game designer. I have one small credit in one product. I barely qualify as a freelancer. So I’m unproven when it comes to actually getting a successful game to market. So far. Of course, I do a lot of personal design and I publish most of it here on this site. And I do test that s$&% as much as I can. And I do have a game in design myself. It’s a big one. And it’s going to need testing. So I’ve talked to a lot of people in the industry who actually know what they are talking about.

Point is, even though I’ve been publishing homebrew game content here for free and I’m betting my future on my ability to pull off this game design s$&%, I’m still just a well-informed non-professional just starting out. Treat my advice as such. And if you’re serious about pursuing publication, you might want to talk to someone who’s actually succeeded.

That said, here’s my advice.

First, you want to be clear on what you’re designing and what you’re going to do with it. Are you designing a game so you and your friends can have a better D&D to play? Are you going to share it around for free on your website or in your own community? Are you looking to sell it? Why? Just because you wrote it and you think you can make a couple of bucks selling pay-what-you-want PDFs on DriveThru RPG? Or are you looking to give DriveThru an exorbitant cut of your sales so people can get print-on-demand copies? Are you looking to make and sell a product yourself? Are you going to crowdfund a print run? Going to take out a loan and clean out your savings to publish it yourself?

The reason to be clear about your plans is that your plans determine how polished your product has to be and therefore establishes how much testing you need. Which, in turn, determines how much work you’ve got to put in. Because there’s a lot more work involved than you seem to think.

When you say, “all I need to do now is playtest it,” that tells me you think you’re almost done. You wrote it and now it’s time to make sure it’s as great as you think it is, tweak a couple of numbers, and then slap a UPC and some art on it and start collecting sales. And I’ve got some bad news for you: getting to the playtesting stage is like getting to the bell gargoyles. It’s where the real design starts.

The first draft of anything is s$&%. Every successful professional will tell you that. The only people who think they write publishable drafts are people who have never published anything. Now, I know you said you’re on your second draft, but you’re not. Unless I’m misreading what you wrote, no one actually played your first draft. You wrote it, got your friends to read it and provide some comments, and then you revised the thing. The truth is that it’s impossible to judge a game without actually playing the f$&%ing thing. Which is why you don’t ask for feedback on written rules. You ask for feedback on the play experience.

There’s this concept in game design – in all design – called “fail faster.” Designers know their first designs are deeply flawed and need a lot of work. They might be promising, and they might have good ideas buried inside, but as a whole they’re crap. The key to finding the good and ditching the crap is breaking the thing as quickly as possible. You take your designs, cobble them together into an ugly, rough, playable state, and you play it. And then, when it sucks, you at least have some solid idea of which parts suck and which ones actually work. Then, you take the good parts back to the lab and you solder them back together into a playable state and you take that back to the table and break it again.

Design is an active, iterative process. You can see that in the homebrew hacks I fart out on this website. How many times have you read about a new, tweaked version of that stupid Time Pool thing I introduced two years ago? How many times has it changed? The first version was okay – and that version had been broken a few times and rebuilt already – but once people started playing with it, they found clunky, clumsy bits. So did I. The reason I haven’t been iterating on it more and it hasn’t moved faster is because it was a side project. Sort of. But I can’t talk about that.

The point is, if you’re serious about designing a game, playtesting isn’t something you do at the end. It’s something you do at the beginning. And then you keep doing it. Over and over. It’s a long way to the Indigo Plateau and playtesting starts in Brock’s gym, not Giovani’s.

Sorry about that.

Hopefully, I haven’t scared you off. And hopefully, you’re not flipping me off and saying, “well, maybe YOU can’t design a solid game on paper and have to go through a thousand sucky versions to get something good, but MY system is different and it’s REALLY GOOD.”

If you’re still here, then you’re ready to decide how important this really is to you and how much work you want to sink into it. Because designing an entire RPG system is hard, time-consuming work and you haven’t really started yet.

The good news is that three hours a week is enough time to test an RPG system if that’s the only gaming time you’ve got. The bad news is that it’s going to take a long time if that’s all you’ve got. And the worse news is that for the next six months or the next year or the next two years or whatever, you’re going to be spending all your gaming time playtesting your system.

Playtesting isn’t fun. It’s not like running a campaign. It involves doing a lot of things over and over again. It involves sometimes throwing huge chunks of your game out and starting again. It’s work. Your game won’t be about fun for a long time. Not for you. Will there be fun moments? Sure. You’re still gaming with your friends. But there will also be moments when everything breaks and the game is ruined for the night. And there will be moments when you’re stuck on a design problem and your back is against the wall and your friends are coming over in three hours and you’ve got nothing and you’re frustrated and depressed and you just want to give up on the whole damned thing and go back to when gaming was something that you did for fun.

I ain’t trying to break your spirit or discourage you. I’m just being honest. You need to make an informed decision about how important this is and how much you want to put into it.

Which brings me back to what your intentions are. If you’re just looking to make a fun game for you and your friends to play, then start playing whatever you’ve got and have fun. Laugh through the mistakes, fix it in your spare time, and just enjoy your game. Don’t put any pressure on yourself because the game doesn’t have to work for anyone but you and your friends. And you and your friends can smooth out the rough patches as they come up. But if you plan to take even one thin dime for your game from anyone else, you have to put in the hard work. You can have that dime, but you have to earn the hell out of it.

Here’s the thing, though: if you ARE planning to ask people to pay for your system, your playtesting setup isn’t going to work. It’s not that you don’t have enough hours, it’s that you can’t run your own playtest. You’re a terrible playtester. You and your friends. But don’t take that personally. All game designers are terrible playtesters. At least when it comes to testing their own game systems.

When you’re doing the early, iterative design work – breaking and soldering and breaking again – it’s fine for you to run the game yourself. But, in the long run, you can’t run your playtest sessions. Not all of them. You need other people running your game.

As the game’s designer, you tend to fix things as you run the game. You adjust numbers, tweak the balance, change the wording, and clarify stuff for your players. You try your damndest to make sure you don’t do that too much and that you take really good notes whenever you do, but you can’t. Because you do a lot of that stuff unconsciously. You don’t even notice you’re doing it. That means you think your game is running well when, really, there are lots of problems you’re fixing as you go that you didn’t even notice.

As an RPG designer, when you run your own playtest games, you end up playtesting the game you have in your head. Not the one you actually wrote down.

And that’s another problem. The game is in your head. You know the mechanics. Intuitively. The game is easy for you to understand and easy for you to remember. You created it. So you won’t realize how difficult something is for an outsider to understand or how hard it is to remember when to use a specific mechanic or how easy it is to overlook a certain rule.

As a designer, you also only have one perspective on the game. Your own. Everything in the game works for you because you wouldn’t have put it in the game if it didn’t work for you. Your game is designed to pander to your specific perspective on what makes a game fun. Maybe you don’t mind tracking encumbrance or using a spreadsheet to track statistics for every village in the game world. But that would bore the s$&% out of most people. So, you either need to streamline the system or accept that you will only sell your game to a very specific niche market of people who relax every night with a bottle of wine and a copy of the latest patch notes for Microsoft Excel.

Relatedly, you also only have one set of ideas about the game. Your own. And you can’t think of any ideas that you can’t think of. I know most people are stupid and terrible and couldn’t design a working table with a kit from Ikea and a set of instructions. But even the dumbest gamer can still accidentally blurt out a good idea that could change your whole design.

Finally, you’re biased as hell. Even if you think you’re not. Especially if you think you’re not. Every designer is either their own best critic or their own worst critic. But no designer is honest and objective about their own creative work.

For those reasons, you want other people running your game system. And, ideally, you want other people playing the game aside from the friends who helped you write the thing. You want other GMs running your game for other players. Ideally, you want to be there too. Watching and listening. Silently. Without interrupting. Without jumping in or helping or explaining. Just watching and listening and scribbling notes furiously.

Now, that’s time-consuming. And not everyone has access to a group of players and GMs they can watch creepily from the corner for hours every week. So, the next best thing is to get other GMs to run your system for other players and provide you feedback. Feedback in the form of both unstructured prose and structured surveys designed to assess specific parts of the system.

This is where your plans can help you. If you’re planning to distribute your system to an online community or crowdfund the project, you’ve got access to a bunch of playtesters. And if you’re serious about publishing, it’s a good idea to build up a community first. A real, meatspace community as well as an online community. Start getting your name out there. Visit local stores and make friends. Make friends online. Set up a Discord channel. Of course, that’s also a time-consuming process. But if you want to make money at something, you’re going to have to earn it.

To sum up, playtesting is an active part of the design process, not part of the final quality assurance test. And playtesting is all about volume. You want as many people as possible to put as many hours as possible into playing your game so you can spend hours combing through the feedback. The fewer hours you have, the longer it’ll take to finish the game. The fewer people you have, the more likely problems are going to slip through and the less broad appeal your game is likely to have.

And that’s why you need to know what your plan is. Are you going to sell your game? Do you need to start selling it in six months? Then you need lots of people playing and running lots of games. Do you plan to keep it to yourself or give it away for free to like-minded buddies only? Are you okay with the game being in a perpetual state of design, break, redesign, patch, tweak, and solder? Then you and your friends and some interested buddies online can handle all the playtesting by just playing a few hours a week.

Whatever you decide, Ash, I hope your game turns out to be the very best. Like no game ever was.

Sorry.


Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

21 thoughts on “Ask Angry: How to Playtest an Entire System on Three Hours a Week

  1. My only comment is a request for Angry to stop apologizing for his Pokemon references. Otherwise, solid advice. Good luck Ash, Professor Angry gave you your starter, time to get out there in the tall grass!

  2. As someone who is working on their rough draft for a game and is looking to start banging out the major dents with my own play group but isn’t sure exactly how far I want to try to take it, this is well timed.

  3. I now understand why I’ve written the rules for two games but have not published . I could not playtest them!

    When you have a lot of questions standing between you and your goal, there is one question that needs answering: How bad do you want it?

    It’s a sign of wisdom when you can honestly let something go because the answer was, “Not bad enough.”

  4. Since playtesting a whole new system seems like so much work, is it better to put that effort into playtesting a new mechnic or hack to an already stablished syestem?

    • Now you really have to define “better”. You sound like your goal is to design something gaming related and that is a very vague goal.

    • Ash said he wants “a D&D that focuses more on exploration and mystery rather than combat.” Depending on what he means by that, there may not be a system out there that really meets his needs without being completely torn apart and rebuilt anyway.

  5. Couple thoughts, from a professional non-game product tester.
    Only test as much as you need to learn what the problems are. After that point, it’s not “test” any more- it’s wasting time you could spend fixing known problems. If you know what your test is going to tell you then it’s not really a test.
    This means up front, you’ll learn things every test and your 3h long test-game a week is fine.
    Then it means whatever is wrong needs to be fixed, or your testers will spend their test time focused on that and probably not notice the next layer of problems, instead re-telling you what you already knew. This might mean fixing things you consider unimportant- if it blocks test it’s got to be fixed.
    Finally, it means the farther you go, the more test time you need relative to design time. Eventually, you’ll be happy- 6 sessions between complaints? Probably a product.

  6. “the next best thing is to get other GMs to run your system for other players and provide you feedback”

    So….if one happened to be designing a new RPG, they may need to find people to help run playtest sessions. Where would those interested in being involved in the playtesting sign up?

  7. This couldn’t have been better timed for me. I’ve just finally decided to put proverbial pen to paper on an RPG. No clue where I’m gonna take it, still assembling the thing, but it’s nice to see the different places I could be going once I’m done with the basic untested rules.

    • I just ran a one-shot last night, being the DM for the first time in 17ish years. My advice is don’t sweat it. If the players have knowingly signed up for a one-shot, they’re probably just looking to have some fun throwing ridiculous character ideas at a one-time-only adventure.

      Sometimes, the level 11 Cleric will succeed on a Divine Intervention and blow up the Prismatic Wall dome, and the Hasted Wood Elf Monk 9/Rogue 2 will double-dash 300′ and blow up the Sphere of Annihilation before they even SEE the Behir final boss, let alone kill it!

      • Ah I meant for publishing, sorry I should’ve been clearer.

        I already ran the adventure a few weeks back and everyone loved it.

        • Oh, lol. My mistake 😛

          I think Angry’s advice about having someone else run the thing you’ve written is the best way to test how publication-ready your adventure is. Like he said, you have a lot of the info you need to run the adventure in your head, but a different DM doesn’t and can show you where those gaps are.

          If you really do want to publish your adventure, then good luck and I hope it works out well for you!

Leave a F$&%ing Comment (Limit: 2,500 Characters)

This site uses Akismet to reduce spam. Learn how your comment data is processed.