I work in QA. Yes, I’ve done video games – 4 years for Sony/Playstation, 1 year for Microsoft Game Studio, but I do android/iOS/website now. In any case (haha that’s a pun), I do testing every day, so I figured I’d write up a quick and dirty guide for useful tips and tactics for testing. Every company is different, obviously, so some of this stuff won’t be relevant elsewhere.

Types of testing
There are many types of testing, all with specific goals. I’m going to outline a few, core types here to help make it easier to figure out what you should be doing at a specific stage in a test cycle.

1. Exploratory
This type of testing is basically the run-around-and-try-everything testing. This is when you go through all the commands, trying out weird as heck stuff to break whatever you can. Start with the “good path” (ie, basic user functionality) and then radiate outwards. Advance to common mistakes (typos, inverted command syntax, being in the wrong state) and then finally push the boundaries. Pretend you are drunk, or a 5 year old, or a total newb, and do unexpected stuff.

2. Test casing/Regression testing
This type of testing follows a template of expected results. You are basically ensuring things work when the necessary steps are executed. For example, a test case might include a test to verify that using x command produces y result. You would then verify that passes or fails or is blocked by some other issue.

“Smoke tests” are a sub-genre of this, where you do a quick once-over for core components to make sure stuff is a-ok.

3. Halo/Directed exploratory
This type of testing is a melding of the two above. Using a test case or design specs as a basic template for features to test, you explore around those features and poke at stuff, starting first with the core functionality and then advancing into outlying conditions and circumstances. For example, there might be a skeleton design doc saying that the LEARN command has changed. You would first play with LEARN in a variety of ways, and then try associated stuff, like TEACH or STUDY.

4. Balance testing
This type of testing is an analysis of the content itself, gauging its impact on the game. This is a much more subjective type of testing, with the focus on analyzing the game system and how new changes will integrate into it. Testing in this category should focus on core gameplay and functionality first, and then expand out to consider outlying situations, with a keen eye directed towards things like exploits and overpowered/underpowered balance. Remember that this sort of testing is NOT just about finding OP/UP scenarios, but also about finding core functionality and balance.

Bug Finding

If you find a bug while testing, try to globalize and generalize it. This means don’t just go x thing is broken and step away – instead, consider what facets of the game x thing relates to and analyze those as well. If a pair of lime green boots breaks the game when you probe them, try probing purple boots, try probing lime green pants and try equipping lime green boots. The bug might not be based in the specific circumstances that you found it in, so check related conditions.

Note time stamps. Some bugs are weird and can be tied to time tables, especially if they have to do with things like resetting items/quests.

As mentioned earlier, if you are exploratory testing, don’t just play like a normal player. Play like someone new, or derpy. Try out stuff that YOU know won’t work. Ensure it fails. It doesn’t always.

Don’t pigeonhole – don’t just test on one account, or one platform. Make a newb, play an alt, run another client. Something that works for one character or one client or one device may not work for another. With playstation, I once found a pretty fatal error for Killzone, where the ENTIRE game wouldn’t work with non 6-axis controllers – it’s imperative you test all scenarios that ANY player might encounter, so try stuff out in a variety of situations.

If you have access to code, use echoes to spit out values of variables, tables, etc to help you conceptualize where things are breaking. If I have a script that’s being a bitch, I’ll code in an echo to shout out WORKING #1, 2, 3 etc for each step to help pinpoint where stuff is going bad.

Test limits – outliers are where tons of bugs occur. Min/max, check scaling, investigate weird and cusp situations/numbers. Good numbers to test around: 255; 65,535; 4,294,967,295 and multiples/divisions of those (integer overflow). Iosyne’s order just saw a bug from this in action kill all our shrines when we hit overflow.

Be efficient – you don’t need to test every single condition ever. Analyze what could make bugs, and then test representative samples within each potential condition. Eg, don’t test 1-100 if 1 and 99 are the things that are going to bring the bug up. Just test those boundaries.

Identify patterns. If you see a common element between unrelated bugs, start investigating more. Odds are, there’s something in common – you just haven’t found it yet.

No bug is random. The conditions that make it happen just aren’t known yet.

Testing Tools

Most testing likes bug reports. In these include, if possible, steps to reproduce or an excerpt of your log/screenshot/video illustrating the issue.

Many games now have in-game bug reporting. These often have crash dumps automatically linked to them (crash dumps are a spit-out of all the code that was executing right before the crash) so it can be really helpful to write up a report asap.

When testing, some sort of recording of your methods is highly advised. Automatic logging upon session login, echo commands turned on, and/or fraps are all useful tools. When I exploratory test at work, I write up charters – this is a document that declares what I am doing to test, and then journals each step I’ve taken. You’d be surprised how hard it can be, sometimes, to remember the precise steps you’ve taken, so ensure you are documenting what you do for easier reproduction.

Google doc spreadsheets are great for collaborative, on-the-fly test casing software.

Have fun – testing is all about enjoying problem solving! You’re a code detective and enjoying the investigation is paramount to doing it well!