So you've received your app's first bug report, congratulations! Now what? You have to find the best way to handle your bug reports to make sure all the effort you and your testers have made pays off. Let's take a look at an app bug's life cycle from the moment it is reported until it is fixed.
The typical bug undergoes three phases in its lifecycle:
- Bug Understanding: When a bug is discovered and reported, the first order of business is understanding its contents. This involves verifying that it actually is a bug with all its needed information -- not a feature request or general feedback, and not a duplicate of an existing bug. Additionally, the reported severity level is verified to ensure there is no exaggeration.
- Bug Triage: Triage is originally a medical term that refers to prioritizing what needs to be treated first. In this stage, the "triager" determines the priority of the bug based on its severity and several other considerations specific to the project. The triager then determines the component having the bug and assigns it to the corresponding developer to work on a fix.
- Bug Resolution: Finally, this is when the developer works on identifying the cause of the error in the code and (hopefully) smooths out the kinks. However, the resolution does not always mean the bug is fixed; for instance, if the bug is too trivial (or too complex) it can be deferred for a later version or marked as WontFix.
According to their adopted development style, teams will break down each phase into several steps with associated "statuses". In practice, these steps are not linear and bug reports frequently revert to an earlier phase. For example, if a bug with a "FIXED" status starts to appear in a new build, it is "RE-OPENED", or an "IN_PROGRESS" bug can be reverted to "OPEN" or "UNCONFIRMED" with a request for more information after the developer was not able to reproduce it.
Below is the default bug life cycle or "workflow" in Bugzilla and JIRA. Notice how they have slightly different steps. This is because when it comes to workflow, there is no standard.
Workflows are extensively modified by different developers and companies according to the needs of their project. They can adopt or eliminate the steps that suit their team and project. Factors like size and complexity of the app, number of users, developer team size, and project management style will all affect the way incoming bug reports are handled. Therefore, all good bug trackers offer workflows that can be customized to suit the needs of the user.
Understanding the bug life cycle is crucial to determine the best way to deal with your bugs. Making room for factors like the type and number of the reports you expect and guiding your bug through its phases with a well-hatched workflow would be tricky without it.
Now let's take a closer look at these steps, their possible statuses, and what they mean.
This is the first touchpoint for a submitted report where triagers make sure it is a valid bug with all the needed information included. While this step is not necessary, it can be useful for large collaborative projects like open source software or where an initial triage can be done by a community of expert or dedicated users and testers. It is also an important step when unexperienced end users are the ones expected to file the reports. This allows the developers to dedicate their time to actually fixing the bug.
Bugs are given this status while waiting for a triager to verify them (if the UNCONFIRMED status is not used). After confirming the severity, triage will determine the bug's priority and assign it to the relevant developer based on the affected component or functionality of the app. Besides severity, the priority depends on many considerations like the number of affected users, frequency of occurrence, ease of fix, and other project-specific conditions.
Assigning a bug to a developer to work on a fix gives the bug the "IN_PROGRESS" status to indicate exactly what the name suggests.
While this might sound like your problem is solved, it doesn't have to be; this status is used to indicate that the assigned developer has finished work on this bug. There are many possible resolutions and teams have different "sets" and definitions, but the most commonly used are:
- FIXED: The bug is fixed by the developer.
- WONTFIX: The issue is not a bug or is deemed too trivial to take priority.
- WORKSFORME: The developer has not been able to reproduce the bug.
- DUPLICATE: The same bug has been submitted before.
- DEFERRED: Other bugs or features take priority and the issue is dealt with in a later release.
This is, hopefully, the end of the line for a bug report. After successful QA testing, they will mark the bug with this status to indicate a job well done.
From a RESOLVED or CLOSED status, the bug can be re-opened if the circumstances around it change. For instance, a WORKSFORME is updated with more details or a VERIFIED bug that reappeared in production.
As you document bugs and track them, you will be able to leverage them to glean insights. When you analyze incoming bugs and track their metrics, you will start to identify areas in your development process that might need improvement. Here we will take a look at some of the important metrics you need to track and what they will help you discover.
Closed vs. reported bugs
This metric represents the number of closed bugs as a percentage of the total number of reported bugs. It helps you keep an eye on the overall progress of your bugs and the technical debt you are incurring.
Average time to fix
By tracking the average time your team takes to fix a bug you will have a better idea about the time you need for fixing future bugs. You will also be able to quickly discover if there are any issues delaying your team by the resulting increase in the metric.
Bug distribution by severity
Monitoring the number and percent of bugs for each severity level helps you track the rate of high severity bugs. This is crucial to keep an eye on the bugs that might block development and/or release.
Bug distribution by module
Keeping track of the number of bugs by the module or component they appear in helps you identify problematic components. The earlier you identify such a component, the better you will be able to take corrective action and prevent introducing new bugs.
Bug distribution by root cause
If you perform root cause analysis, mapping bugs to their root cause enable you to identify at what stage of development these bugs were introduced to your code. If a certain stage introduces more bugs, you can investigate the reason and make sure it is dealt with.
Leakage represents the number of bugs that were discovered in production as a percentage of the total. Measuring the number of bugs that leaked through the testing process alerts you to the performance of your testing process. By analyzing the components or the root causes that are most frequently missed by testing, you will know exactly what you need to improve.
Keep it simple
Try not to go all out when setting up your workflow and choosing your tools. Keep your choice as simple as possible to eliminate the overhead that comes with complicated processes and tools. Consider the needs of your projects and pick the simplest process that will suit them.
No blame culture
We might hate bugs but they are inevitable in software. Foster a culture that doesn't hold developers to blame for bugs or testers for missing them. Don't use metrics like the number of bugs for each developer or discovered bugs by each tester -- they are counterproductive and toxic to your culture. Implementing them as KPIs increases problems like "bug tossing" where developers and testers keep reassigning bugs back and forth to avoid the blame.
Encourage communication between all stakeholders of the development cycle to avoid bugs being thrown "over the wall". Hold regular triage meetings (or conference calls) where everyone can get aligned. Keep these meetings short and sweet to avoid them being a burden on your process.
When setting up your workflow, make sure that bugs always have one person assigned to them until they're closed. This increases accountability and makes sure no bugs are forgotten or fall through the cracks.
The person who opened or submitted the bug should always be the one to verify it is fixed and mark it as closed. In case of bugs that are submitted by end users, you should preferably verify the fix with the reporter before closing the bug. Since the opener of the bug is the one who discovered the bug, he will be able to ensure it has been fixed.
Use bug tracking tools to improve your process and enhance your bug reports. You can leverage these tools for better tracking, quick search and identification, and improved metrics and reports. Again, don't go overboard with your choice of tools and opt for the simplest tool that meets your needs.
Make sure that all stakeholders have enough visibility on the progress of bugs through the lifecycle. This should not be limited to the bugs that they are assigned to and should include all bugs as well as overall progress and metrics. Additionally, archive all closed bugs periodically to prevent them from cluttering your database. This increases your team's situational awareness and enhances collaboration among stakeholders.