Join CTO Moataz Soliman as he explores the potential impact poor performance can have on your bottom line. 👉 Register Today

ebook icon

Best Practices

General

Best Practices for Beta Testing Mobile Apps

Have specific objective(s) for your beta testidentify anomalous behavior without losing sight of big picture20-30% participationstructured (breaking the product, "pro" testers) vs unstructured (feature requests, end users, usability feedback)feedback priorities, ranking, next steps (project management system, reports to whom and when)find errors, assess a specific feature, compare features, out-of-the-box reactionstasks i.e. update appCompile a group of testers within a defined set of parametersend-user demographicsprofessional testersone in five will registerone in x will provide feedbackfirst-adopters, power users of complementary businesses/apps/sitesWhen compiling your beta testers, target real userstarget user: device (VR, high-end), location (localizations), age (ex: transportation apps to know if it's suitable), other (people who have driver's license), battery states, multiple networks, real users:feedback different (design issues, IT field follow different techniques, combining different data combinations to find errors - proactively trying to break it)mobile device fragmentationend users open beta no knowledge about product or tech used in product like anyone using app for the first time one or two weeks opinion of users from first use after long duration they will get used to the product and their opinion won't be as effectiveprivate beta to go live asapcraft messaging around beta tester persona you're targeting. frame it as an opportunityinvites for other testersTreat your beta testers with respectSave your beta testers' contact info in a database and don't spam them. Provide incentives like a reward system. NDS. recognize efforts and maintain long-term relationships. thank you letter (slack crash log reproduce).two approaches:1. paid and incentives (promo codes for lifetime, coupons, subscriptions, swag) for constructive feedbackkeep your promises (rejected issues and not paying)2. unpaid (better for bigger companies)points and contests http://www.cadalyst.com/cad/solidworks/the-alpha-beta-testers-15695Beta-ready productPlannumber of testers23stripe 53instagram 100 https://www.quora.com/How-many-beta-users-did-Instagram-have-right-before-launchduration of testspecific items to testtimelinedetails of processmaterialpoint of contactlegallogisticstools: documented electronic communication, electronic formsTimelineslack 3-5 days10 weeks http://www.cadalyst.com/cad/solidworks/the-alpha-beta-testers-15695distribution:1. IT field: .APK (Android) or .IPA (iOS) file2. real users: upload to cloud (like TestFlight for iOS) and people install with one click3. web apps: host new version on different domaincyclesrestricted: paid, find as many remaining bugs as possibleopen: user feedback, cultural fit of users, testing market fit, usability measures (ex: testing a new feature uber add location before ordering car)Multiple test cycles (ask testers to verify fixes/updates)Use many of the previous testers who provided good feedback plus some new testers for fresh perspectiveRapid multiple cycles or big feature updates with new usability functionssmall releases in one cyclenote: multiple versions not allowed on testflight (ios)rules (restricted):what to report or not to report (bugs they already know about, issues they still need to work on)documents needed (screenshots, video, crash logs)specs to testInstructions for using/installing should be written for the wider public http://newsok.com/5-things-google-glass-has-taught-me-about-beta-testing/article/3861336/https://help.lyft.com/hc/en-us/articles/214583247-Lyft-BetaTerms of use https://help.lyft.com/hc/en-us/articles/214212437-Lyft-Beta-Terms-of-Use"secret"CommunicationChannel to communicate for any questionsOpen (last line of testing): different opinions about product no communicationContact testers for more details in specific cases ask for more informationforums https://beta.autodesk.com/oid/login.html?msg=&ref=https://beta.autodesk.com/overview/engagement model (strict vs relaxed) http://www.sonicstate.com/news/2013/12/05/confessions-of-a-beta-tester/http://www.smartcompany.com.au/startupsmart/startupsmart-technology/hello-code-co-founder-beth-belle-cooper-on-how-to-actually-get-good-feedback-during-a-beta-round-or-testing-process/invitations https://www.quora.com/What-are-the-best-examples-of-beta-invite-emailsInterviews (Autodesk Gold program) https://www.quora.com/What-questions-should-you-ask-your-existing-private-beta-usershttp://autodesk.blogs.com/between_the_lines/2013/05/autodesk-beta-programs-what-is-it-and-how-does-it-work.htmlhttps://beta.autodesk.com/faq/welcome e-mail intro call limited automationinstructionsdeadlinesmultiple feedback types (bug reports and surveys)Avoid changes to product or extra asks late in the cycleNo revisions or product changesRapid response team (daily or weekly basis)Indicators of successhttps://www.quora.com/We-are-releasing-our-game-app-beta-on-the-Beta-Program-on-Google-Play-Store-What-are-some-good-indicators-we-have-a-successful-game-on-our-handsnet promoter score: how likely to recommend to friend (0-10) = (% 9-10) - (%0-6)5-star rating system + comment (like app stores)https://www.fastcompany.com/3002926/4-steps-building-bulletproof-beta-test

Tags

No items found.

Start Your 14-Day Free Trial for Realtime Contextual Insights

In less than a minute, integrate the Instabug SDK for iOS, Android, React Native, Xamarin, Cordova, Flutter, and Unity mobile apps