Unity’s Supersonic rolls out more accurate A/B testing

Supersonic, a division of Unity, has announced enhancements to its platform to equip game developers with better A/B testing.

A/B testing lets game devs experiment with player preferences in terms of features in a game. The changes enable insights and more precise decision-making capabilities regarding a games’ growth.

The latest updates, centered on A/B testing and a revamped “All games” page, offer developers a panoramic view of their game’s performance metrics, Supersonic said

New A/B Testing Page

The innovative A/B testing page introduces a unique approach to test analysis, acknowledging the potential inaccuracies stemming from outliers and smaller test groups. Its key features include:

• Test Calculator: This tool assists developers in understanding the necessary audience size for a specific test, factoring in desired Key Performance Indicators (KPIs), error rates, variants, and more. It provides insights into the test duration required to reach statistical significance and the number of users achieving predetermined milestones.
• Statistical Significance: Developers are presented with a comprehensive range of test results, eliminating outliers and offering a clear understanding of the majority results. This refinement allows for more precise decision-making.
• Effect on Multiple Metrics: Beyond the primary KPI, the tool assesses the test’s impact on additional metrics, empowering developers to comprehend the overall effect of a test on various aspects of their game.

According to Supersonic, this update ensures that positive test results aren’t dismissed hastily. It enables developers to grasp the broader implications of a test, considering other essential metrics that might exhibit significant improvements despite the primary KPI showing no statistical change.

Revamped “All games” page

The updated “All games” page offers developers a consolidated view of all their games, allowing customization based on specific testing stages and prioritized KPIs. With nearly 30 KPIs available, including CPI, playtime, retention rate, and average revenue per paying user, developers can tailor their view to assess the performance across channels and cohort periods.

By presenting crucial data in a single interface, developers gain immediate insights into their games’ status, enabling swift decisions on allocating attention, focusing on specific channels, and identifying areas requiring enhancements.