Adding More Quality Assurance Engineers Raises Quality Across the Board

In the last 9 months, we added a quality assurance manager and two more QA engineers to the PatientsLikeMe team as the engineering team continued to grow. As you might imagine, our quality assurance efforts have increased, testing has improved, and QA's coverage is souped up to match engineering's growing responsibilities. Quite a few unexpected benefits came our way, too.

My new colleagues draw on their previous experience regularly to improve our processes, from workflow to what tools to use. Before, we weren't always disciplined about logging test steps that we could potentially reuse at another time, relying often on digging through our ticketing system to find what was outlined or ask someone to recall a procedure to verify a bug or examine a new feature. Now we have a shared repository.

Their fresh eyes have led to better documentation. New people need to know how to do things. Let's record that information. QA folks are often good at thinking about how something could be changed to be easier, better, etc. For example, when it came time to set up their development environments, when following the step-by-step guides, they fleshed out some of the steps and made corrections along the way. When moving through regression steps for the first time, they asked lots of good questions that led to revised steps and better product coverage during manual analysis. Even steps in tickets to reproduce defects have become more thorough because we don't always know if the person most familiar with the product will be the one handling the fix. It's not just: “This instruction isn't quite right.” It's “This instruction isn't quite right. Let's change it, so the next person has an easier time.”

Nothing compares to having colleagues to consult. We just went through some complicated verifications that began as a simple request for the developers to build something reusing pieces of one of the products. It ended up being much more complicated than that. Proofing went from a few simple steps completed quickly to plans that also touched on corresponding parts of the product and tickets for future alterations to the original product. Four of our quality assurance engineers worked on aspects of the inspection. Watching what they did outside of the suggested route was educational. Parallel efforts meant we released within a tight schedule without a lot of sacrifice from any one particular team giving up a QA engineer for a prolonged period. Different perspectives led to more thorough results that were better than if just one person had scrutinized everything.

And we can take time off without worrying too much about what might happen in our absence. We can usually cover for each other. While we're not complete clones of each other (and we're not meant to be), our teams get an impartial prober instead of trying to figure out how to confirm their own changes. That almost makes it sound like we were sloppy before when we weren't. We don't release unevaluated changes. We don't test in production. Programmers don't trial their own work. Now it's much easier to follow our group's assessment guidelines and find someone who can assist with that.