When it comes to mobile app crashes, it's always important to look at how often they aren't happening. Unlike bugs, crashes completely end the user experience and a study shows that 80% of users will only attempt to use a problematic app three times or less before uninstalling. Users nowadays don't have much patience given the huge flood of applications available to them. This is why knowing how often your app isn't crashing (crash-free sessions) is essential to know how users are interacting with your app and if your app needs more work. If a user downloads an app that is constantly crashing, not only will they delete it, but they're also likely to leave very negative reviews and app store ratings.
You probably already know how problematic crashes are, and how difficult they are to prevent. The important question is how many crashes count as okay, good, or bad? Ideally, all apps would love to have no crashes whatsoever, but the larger the number of users an application has, the more impossible that undertaking becomes. So what crash-free session rate should you realistically target?
Calculating your crash-free rate
Before we can get to what's a good crash-free rate, you need to figure out what’s the best way to represent your crash-free rate. The most straightforward way to calculate a crash-free rate is:
Depending on your needs, however, you can twist that formula a bit. It doesn’t make a lot of sense to factor in all the crash occurrences that happened when your old release still had some issues that you've already resolved. Depending on how often your app is being, used the formula can be changed to:
Of course, the period you're looking into is up to you to set and it depends on the number of sessions your app generates. If you have less than 100 daily users, measuring your daily crash-free rate wouldn’t get you very useful results. Also, sessions are not the only variable that you should take into consideration.
Swapping out sessions for users determines the percentage of users that haven't experienced a crash. It makes sense that the user crash-free rate would be lower than the occurrences crash-free rate since the total number of sessions is much higher than the total number of users. The equation would instead be:
Benchmarking crash-free rates
Now that you’ve found the best way to determine your app’s crash-free rate, how will it hold up compared to industry standards? Crash-free session rates have improved over the years with easier testing tools and progressively better development techniques. In 2017 Qualitrix published an acceptable target of crash-free rates for mobile apps.
Let’s look into what these percentages actually mean. Our equation for calculating crash-free sessions would get you the crash-free session percentage. This is the most general use case because it highlights the whole app performance in a simple calculation. How many times your app has been launched, compared to how many times your app hasn't crashed after launch.
The crash-free user percentage gives you a better indication of the user experience your app delivers. Focusing on user crashes highlights different environments for your app's performance. A certain device could be a common factor in crashes or a random occurrence with a particular user,, for example. And since a single user can have many sessions it follows that they are more likely to experience a crash.
Looking at both kinds of crash-free rates will paint a more holistic picture of your users’ app experience when it comes to troublesome crashes. And catching an uptick in crashes early on can help you identify and solve problems in your app before users react negatively.
Can crash-free rates be misleading?
Crash-free rates can tell you a lot about how your app is performing, but they don't always give you the whole picture. One crash might not upset a power user since they're more likely to assume that it's a random crash and an unusual event. A new user, on the other hand, won't be as forgiving. The bigger problem occurs when a single user experiences multiple crashes, especially in a row. The frequency makes it impossible to ignore.
So, you should always aim to have crashes very unlikely to occur--but how exactly can you do that?
How to improve your crash-free rate
It’s essential to perform thorough testing of the entire app experience. It's easy to get lost in the development cycle and start testing feature by feature as they’re developed. However, testing the whole app experience daily is much more effective. Understanding what the end-user will be experiencing gives you important insights and exposes you to issues that won’t be seen from the developer's side.
More importantly, when a crash occurs, you need to know about it before hearing about it from negative reviews or emails. Using a tool to track your app’s crashes goes a long way to help you manage testing and address issues before they impact your users.
- Top Mobile Crash Reporting Tools
- Why Does Your App Keep Crashing?
- Test-Driven Development Tutorial for iOS
- The Benefits of Beta Testing: Beyond Bugs and Crashes
Instabug empowers mobile teams to maintain industry-leading apps with mobile-focused, user-centric stability and performance monitoring.