4 Tips for Releasing High Quality Mobile Apps

By John Houghton on January 10, 2014

Now that we have non-software industry CEOs / VPs / Directors / Managers heading up app development projects, I’m seeing many apps that release with a lot of bugs.  Let me share with you the tips I’ve learned from top software companies about quality assurance, and apply them to mobile app development.  These are the most important tips that I feel are often overlooked in mobile app development, and if followed would make a big difference toward releasing higher quality software.

I cut my teeth at Oracle, which is known for its expertise in quickly developing high-quality, inexpensive software.  They have good processes.  I was responsible for Product Management for one of Oracle’s all-time fastest growing products, iProcurement, and a lot of the reason for our success was that we figured out our Quality Assurance (QA, meaning bug testing) strategy, and made sure the product worked well.  If you are reading between the lines, you will glean that I’m saying that a lot of software products fail because they don’t work well enough (they’re buggy).  That is true about software as a whole, and so applies to mobile apps as well.  Most of what we all know of software are the top-selling products, and these are usually pretty high-quality, but if you look at the average software product that is released, you’ll frequently find a buggy product.  Same is true if you look at a random app in the iTunes Store or Google Play Store.

1. Hire Good People.  Quality Assurance is a professional trade.  Some people think of bug testing as something that can be done by anyone, and these are the people who release low-quality products.  In reality, it takes a good strategy, with proper tactics and discipline to release a quality app.  This is why I say that software QA is a professional trade, in which people educate themselves about best practices, and build a career around it.  I know right away when someone has good training or not.  I find that it takes someone with the right background and at least a few years of experience to be good at QA.  Someone who is testing, but not trained in QA, is going to hinder the rest of the team, and drive the developers crazy.

Bugs are normally logged very systematically into a bug database.  The way they are described, they each have a context and are reproducible at any point in the future by following very clear steps.  When I see unskilled testers logging bugs, they say what’s broken, but don’t provide context (list the build number for the mobile client or the server – for apps that have a server back end), and they don’t list the reproducible steps.  The bug is fixed and it becomes patch number xyz and is associated with the bug.  As time passes and the team loses context, it becomes difficult or impossible to remember what issue you solve with this fix, because it wasn’t well-described, and there is no way to reproduce it since the build information and steps to reproduce are missing.  Now you have a bunch of patches (a.k.a. fixes), but you’re not really sure what each of them was meant to fix.  It’s a disorganized mess that started with poorly logged bugs.   This is what you can expect with inexperienced QA resources.  The tip here is to hire good QA people and respect what they do.  If they tell you they need more time, give it to them.

2. Don’t Test Without Test Plans.  At an immature company, you throw the app over the wall and tell the testers to bang on it.  At good companies, there is no testing without test plans; otherwise, how would you verify what was tested?  Normally, a QA resource (or Product Manager) will write the test plan, which outlines what will be tested and how.  The team reviews the test plan to insure that successfully executing these certain test cases will insure a high-quality (bug free) product.  When the test plan is complete, the QA team gets the product, and typically needs about half as much time to test the product as the engineers took to build it (depending on the app).

The typical problem you are going to have is that the QA resource may not be familiar with the product you’ve built.  If you’ve kept it up-to-date, you can hand them a design document, however, most design documents don’t go into enough detail to help the tester understand what the expected functionality should be.  At large companies that have good documentation policies, you will typically have a series of design documents (beyond the scope of this post), each going into greater detail.  The document you could hand the QA person would be called a Functional Specification, but realistically, a lot of teams don’t need that much documentation, especially if the company is small, there is a lot of face-to-face communication, or if the team has worked together as a whole for a long time.

3. Use a Bug Database.  You should always use a bug database, and have procedures in place on how to use it.  We cited what can go wrong in an earlier example.  What I sometimes see on small teams is people just emailing bugs to developers.  This may be fine for a weekend project, but it’s not efficient to do any amount of work like this long-term.  People’s email boxes end up full of bugs, and it takes developers a lot longer to put together the history of an issue by scouring through their emails.  Also, your developer’s time is expensive and valuable from a time-to-market perspective.  They like a good bug database to keep things organized, and they’ll become very frustrated if they don’t have one, or if people don’t follow established procedures.  Senior QA resources already know the rules because of similar procedures at their last job, but I find junior resources that are not as experienced frequenlty don’t follow the rules, even if you show them the procedure.  They don’t see the need, after all, everybody can see that it’s a bug, can’t they?  They typically don’t provide enough detail in their bugs, e.g. steps to reproduce.

I remember at Oracle, the bug database processes we followed were so good that it pretty much ran our day-to-day lives (in a good way), and kept thousands of engineers, product managers, and testers organized and marching in the same direction.  Without it, there would have been chaos on many levels.  A good bug database allows you to look back over a yearlong project and see what people were working on and how you got to where you are today.

4. Test Everything, Then Retest.  In your mobile testing you have to include every possibility – WiFi on, WiFi off, WiFi on then off in the middle of the task, Airplane Mode, getting a phone call while using the app, getting a reminder while using the app, getting a notification while using the app, turning it to portrait or landscape mode, and so on.  Sometimes, if there are different combinations of actions and settings to be tested, you need to design a matrix to make sure you’ve covered everything and all the possible combinations of actions/settings.  This matrix can produce hundreds or thousands of mini-test cases.  They all need to be tested.  Guess what happens if you skip this part?  You got it, there could be a bug back there and you missed it.  When teams take short cuts, you start to see stories like this: “Apple Map Fails: 19 Ridiculous Glitches Spotted In Apple iOS 6’s Anti-Google App.”

So there are a few basics tips for mobile app quality assurance (QA).  Software development requires a lot of different skills in different areas.  Quality assurance is one of those important areas that needs your attention and expertise.  It’s hard enough for seasoned software professionals to release high-quality mobile apps, so if you’re from outside the software industry, you’re going to have to work extra hard.  Hopefully with these tips you can release higher quality mobile apps in your next release.

Posted in Android Apps, App Development, Enterprise Mobile Apps, iOS Apps, iPad Apps, iPhone Apps, Mobile Apps

Comments

Comments