Saturn Leverages Instabug to Streamline Bug Reporting and Reduce QA Process Time

Introduction

Saturn is a social calendaring app that helps high schoolers manage their academic and social lives. As the team and app began to scale, they recognized the need for a more effective QA process. With the start of each school year, the daily user and team headcounts increase rapidly, and Director of Engineering DJ Tarazona wanted to ensure that the app's performance and quality remain consistent during this period of growth.

‍

"At Saturn, our goal is to create a sense of community through the use of our calendar app. Initially targeting high schools, we assist tens of thousands of students at hundreds of schools daily to manage their time and stay connected with friends. The majority of our users are active on a daily basis."

‍

Objective

‍

At Saturn, measuring success is crucial to achieving their mission of building communities around their interactive calendar. The app, which is a time-based social network, simplifies chaotic high school schedules and helps students stay organized. To keep up with the growing number of commitments and connections, Saturn employs a team of engineers that increases in size before the start of each school year to handle the increased workload.
‍

Tarazona has firsthand experience with the rapid scaling of the company. He said, “When I joined Saturn, there were 20 employees total, but now we’re at 40-50 employees. Our team grows rapidly as we prepare for the start of the school year and bring on contractors.”

‍

‍

Challenge

‍

The app was designed to make the lives and schedules of students easier, but its own QA processes needed to be streamlined to reach some of the team’s performance goals. Saturn’s fast-paced, planned growth presented the team with challenges, including how to expand existing workflows and communication channels to include new employees without slowing down their mean time to resolution or missing potential release blockers. 

Tarazona notes that this growth had a particularly significant impact on their beta testing process.

‍

“Almost every Saturn team member is asked to participate in test flight builds and report any issues,” said Tarazona. “This was manageable when we were a smaller team, but as we had more people testing our app in staging environments, it became increasingly difficult to manage all the different channels through which bug reports were coming in.”

‍

Reports came in through a variety of channels, such as direct Slack messages, dedicated Slack threads, and even text messages. The mobile app team was also frequently collecting feedback from app users, and the engineering and client service teams had their own issues or suggestions to report from the students using the current version. Additionally, unplanned issues were sometimes dropped into a team’s to-do column within a sprint that was already in progress, or left in an inbox where they might be overlooked. The fragmented approach became increasingly challenging for Saturn.

‍

“Just keeping track of all of the reports became very challenging,” said Tarazona. “And we found that engineers were often asking the same questions regarding the reports.”

‍

The team could track the reports internally and link them to the tester for more details, but they wanted to find a way to reduce the time between an issue being discovered and its resolution. This required more comprehensive, consistent reports than they had access to through their informal question and answer approach.

‍

“We always felt that there was not enough information available, and engineering needed more information to debug the issue. We realized that Instabug could capture the data we needed to move forward.”

‍

Solution

Tracking and managing the reports also required a significant amount of time and resources. 

“Having a QA team member collect, organize, delegate, and route all these bugs was taking a lot of time. When we evaluated our resources, we realized that we were spending a significant amount of time on these tasks,” said Tarazona.

An internal analysis showed that the QA process of assigning issues alone was taking approximately 16 hours per week. With a weekly release cycle and new issues being reported with each cycle, this equated to about 40 percent of QA time spent on sorting and assigning bugs to the correct teams.

To address this issue, Saturn decided to automate as much of the reporting and QA process as possible. After integrating Instabug’s bug and crash reporting tools into their workflows, beta testers could now simply take a screenshot of an issue to invoke an alert.

‍

“We direct any issues to Instabug because it provides us with a lot of information and it's a simplified process,”

Having a single source of reports reduced the chances of bugs slipping through other communication channels. It also streamlined the reporting process, encouraging more internal testing and helping to prevent bugs from getting into the final release. The simple screenshot trigger initiates the alert, which is automatically categorized according to Saturn’s team designations to save QA time assigning tickets. Using Instabug’s automated alerts, the appropriate team is notified of any issue in their category, and the issue is automatically routed to their inbox.

At that point, the team has the issue, a comprehensive Repro Steps log for detailed information, and the ability to quickly triage based on urgency. They no longer have to search through multiple channels for tickets or reach out for further details, and now automatically receive a log of all the UI interactions the users and testers had before reporting a bug.

‍

Results

By using Instabug's bug and crash reporting tools, rules and alerts functions, and workflow integrations, Saturn was able to significantly reduce the QA process time. Specifically, they saw a reduction of over 12 hours, with a decrease of over 85% in the time spent collecting and sorting issues, now taking only 2 to 3 hours per weekly cadence. This allowed the team to focus on more pressing projects and effectively scale during critical growth periods.

The use of a single source for issue reports and data from beta testers created a more direct pipeline from the issue to the resolution, regardless of the number of employees added. Tarazona noted that the team has seen a significant reduction in the time it takes to get a release from the RC stage to the App Store since integrating Instabug, and engineers are better equipped to triage release blockers.

“We have integrations with Instabug to manage our releases and automatically label them. Then we're able to go in very quickly and look at all the issues that were reported on that release version. This helps us triage and quickly prioritize those issues to determine whether or not there is going to be a release blocker.”

Instabug has helped Saturn become more efficient in their workflows and QA processes, empowering their team to continue to innovate for students across the country. 

‍