If you’ve spent even the smallest amount of time talking to me about software development, you know that I am a big time "You can pay me now, or you can pay me later" kinda guy. With Software Quality, if you have not paid in advance, you will certainly pay later — and the price will be much higher than if you had just paid up-front.
I’m one of those types that puts significant value on the Quality Assurance department of any software development shop. So, I appreciated seeing the post at OnStartups: Business Geeks: Automated Software Testing as Competitive Advantage. The key here is "Automated". There’s really no way to build a scalable software business if you do not invest in good software engineering practices, combined with automated testing.
Most startups (and, heck, even many big, established companies) do a crappy job with their Software Quality Assurance programs. Why is that?
- Most CEOs and VPs of Engineering do not understand its importance. Oh, sure. They all say that "Quality is Job 1" or whatever, but when push comes to shove and the product "needs" to be shipped, quality is the first thing that is sacrificed. Executives feel that they can always patch the product later. That’s a fine approach, it will just cost you a lot more.
- It’s very hard to build a QA group with rock-star programmers. Often times, junior engineers will be hired into the QA department. They will grow there, but the first chance they get to create "product" code, they will try to transfer out of the company or, worse, quit and move on to a different company. Organizations need to reward QA programmers with equal career paths as their code-writing counterparts.
- QA is not factored into the early planning of a software product release. Software teams will spend tons of time fighting over requirements, scope, features, modules, partitioning the teams, outsourcing, private APIs, public APIs, usability, configuration files, installation, upgrade, etc. Frequently, however, the automated test suite is not given its proper attention. If you miss it while planning, you will miss it while executing. While you are doing your product design, you need to be doing your test design.
- Most organizations have no structured software development lifecycle. If you don’t know when a product can transition from Alpha to Beta to GA, or when it’s time to release a patch vs a hot-fix, or when to release a dot-release vs a dot-dot release, or when its time to do a major release, or the criteria by which you EOL a product — then how do you know when its time to test? And, if you haven’t invested in "Automated" tests, then you will be paying a pretty significant price each time you send something out to a customer.
- The testing cycle is often the last thing before product ship. Most companies treat testing as something that happens as the final "certification" that allows them to release the product to customers. That’s very bad. Testing needs to be the first thing that is done. The only way you know you are "done" is if the test suite passes.
At my last company, Cassatt, we have a rock-star group of software developers and QA engineers. We instituted what is referred to in the Agile software arena as a "Test First" software philosophy. How does that work? The developers specify the unit tests at the same time that they specify the design and the interfaces. They code the tests first, then code the module. When the tests all pass, you know you are done. This gives the QA department the ability to look beyond the unit tests and focus on multi-unit, simulation, stress, code-coverage, and GUI tests. It requires rigor and structure, and a management team that believes in the Pay Me Now form of software development.
You can Pay Me Now.
Tags: Extreme Programming, XP, Test-First, Quality Assurance, QA, Brian Berliner, brianberliner
Excellent post…I’m not that knowledgeable yet about software development yet it would seem to be a lot more satisfying to those involved to have a series of small wins that builds to a clean, functional whole.
Is the “Test First” approach widely used?
It seems more common to have the QA/testing function as the end point of the development cycle.
My business plan shows salary lines for “junior engineers” in QA…hum…may rethink that…
Hi Cate,
Thanks for the comments.
The Web and “Software as a Service” has fundamentally changed the way that we build and deploy software. We’re seeing less “packaged” software in a box or on a CD and more websites that, in fact, are the product (as a service). This fundamental change has allowed for releases to be done in a much lighter-weight way. If you have your processes in place (and an automated test suite), you can even release product features and bug fixes multiple times per day. Flickr certainly did this.
The benefit of SaaS companies is that everyone gets the upgrade and bugfix all at once. And, the software is always in a very controlled environment, being upgraded by the folks that wrote the code. These are big advantages to everyone. I’m a big SaaS fan. This model also supports strong business models.
Test-First is finding pockets of supporters, certianly. And, those pockets are growing. Each development organization tends to morph their own processes to the strengths and weaknesses of the developers. Only the best organizations can really pull of a Test-First approach, in my experience.
In building out your QA organization, make sure that you have hired the leader, put him/her as a peer to the other software developers in your org, and work with him/her on building out the QA group’s approach and metrics and reward structure. It’s OK to have lots of “junior engineers” in QA - you just need to keep them focused on writing code, not doing manual tests. Sell the job to them as the best place to write the most code possible. And, it’s true. A well-run QA group will, hands down, write more code than is in the actual shipping product.
Make QA a career within your organization.
Wonderful Brian:
SaaS is the preferred approach…fortunately the current market leader in my space has utilized this structure for a number of firms that they provide the service to…so I wouldn’t have to sell a new method.
I really like and appreciate your comments on QA. Especially the idea that the QA leader is a peer to others on the development team. And that method and process can evolve given the strengths of those involved. Not being a developer myself I’m not wedded to any methodology but I do believe that a system should be in place.
Many thanks, Cate