How Do You Know When Your Application is Ready to Launch
If you’ve ever built a mobile or web application you’ve probably experienced a very common pitfall that many software companies experience. You get close to launching your app, all the functionality you want has been built but there are bugs that just keep popping up and it never seems to end. Nobody wants to launch there pet project into the world that’s buggy and unreliable. Chances are you have endured countless hours, late nights, laughter and maybe even some heated arguments building this product. And, nobody wants to send it out in to the real world with bugs right? Wrong! The assumption that you are ever going to get to zero bugs is just silly.
The Reality of Absolute Zero Bugs
Writing bugless code is not the goal. If that is what you are aiming for, then I have some very bad news for you.
The key point is that you are vastly underestimating the complexity of software.
First things first–You’re ignoring the bigger picture of how your program runs. It does not run in isolation on a perfect system. Even the most basic of “Hello World” programs runs on an operating system, and therefore, even the most simple of programs is susceptible to bugs that may exist in the operating system.
The existence of libraries makes this more complex. While operating systems tend to be fairly stable, libraries are a mixed bag when it comes to stability. Some are wonderful. Others … not so much … If you want your code to be 100% bug free, then you will need to also ensure that every library you run against is completely bug free, and many times this simply isn’t possible as you may not have the source code.
Then there are threads to think about. Most large scale programs use threads all over the place. We try to be careful and write threads in such a way where race conditions and deadlock do not occur, but it simply is not possible to test every possible combination of code. In order to test this effectively, you would need to examine every possible ordering of commands going through the CPU. I have not done the math on this one, but I suspect that enumerating all of the possible games of Chess would be easier.
Things go from hard to impossible when we look at the machine itself. CPU’s are not perfect. RAM is not perfect. Hard drives are not perfect. None of the components within a machine are designed to be perfect–they’re designed to be “good enough”. Even a perfect program will eventually fail due to a hiccup by the machine. There’s nothing you can do to stop it.
Bottom line: Can you write “Bug free software”?
Anyone who tells you otherwise is clueless.
So How Do you Know When to Call it a Day and Launch?
In the end, bug free code is not the goal. Just try to write software that is easy to understand and maintain. Once you’ve done that, you can call it a day.
Even the best written software has bugs. That’s why you get updates on your phone from your favorite apps on a regular basis. Even Amazon, Facebook, and Yelp release bug fixes on a regular basis and if any of those companies never launched until they were 100% bug free, they would have never met the light of day.
Then What is an acceptable amount of bugs to have?
In the end no bugs are really acceptable but you can build your code with a set of standards. Once you meet those standards you can feel confident your ready to launch. We call these development standards “Acceptance Testing Guidelines. I’ve posted Space Chimp’s guidelines below that you can use as a starting point for your new startup.
Acceptance Testing Guidelines
The Following guidelines outlines Space Chimp acceptance testing for software deliverability. Once the following items are all validated then the software is ready to deliver to the client.
#1 Reliability, Scalability: Validated via a stress test. – Using LoadImpact.com or similar tool.
#2 Usability: Validated via an inspection and demonstration to the customer. Is the UI configured to their liking? Did we put the customer branding in all the right places? Do we have all the screens they asked for? Does the core functionality outlined in the scope work?
#3 Security: (aka, Securability): Validated via demonstration. Sometimes a company will hire an outside firm to do a security audit and/or intrusion testing.
#4 Maintainability: Validated via demonstration of how we will deliver software updates/patches.
#5 Configurability: Validated via demonstration of how the customer can modify the system to suit their needs.’
#6 Bug Fixing Completion: Is the software easy to understand and maintain? Does it work in most user cases? Can the business operate effectively?
Don’t try to write bug free software. Write good clean code based on a high standard set by core guidelines to hold your developers accountable for the code they write. Its not easy to do but with some processes in place it is easily doable.