How to prioritize player experience with data and analytics | Chartboost Academy
Insights & Best Practices
July 20, 2023
In an endless sea of 500,000+ games on Google Play and 200,000 on Apple App Store, how can a developer possibly hope to stand out? Data. That’s it. That’s the blog post.
Well, not really. More specifically, you should collect and process information about your users, analyze how it relates to your mobile game data, and put the puzzle pieces together to understand your player experience.
While every team has a different goal, focusing on the player experience should sit at the heart of your data and analytics journey. Zynga’s Lead Product Manager for Ad Monetization Kyle Waring says, “We prioritize user experience key performance indicators first, then monetization. Prioritizing engagement and retention over monetization metrics is critical for maintaining long-term value for our players.”
Kyle would know. Zynga’s portfolio of iconic franchises have generated over 6 billion downloads on mobile and achieved great success monetizing its games.
“We don’t want to jeopardize long-term user experience for short-term monetization gains,” he adds. “That’s what kills games!”
What data and analytics can help you accomplish
Data and analytics can improve lifetime value (LTV), retention, and ad monetization. Tapping into this info helps you find a balance between the number of ads that are shown and the types of ads that users are most engaged with. This can help with ad fatigue, which leads to user dropout and uninstalls your game.
Mobile game analytics help you measure the success of your game and performance over time. They can include:
- Total number of unique users.
- Player location.
- Types of mobile devices.
- Length of time spent in the game.
- Number of levels completed.
- Most popular features.
After gathering data, you are better able to make informed decisions on areas for improvement, such as design and gameplay experiences to enhance your monetization strategies.
Read on as we cover how to go about prioritizing player experience to reduce mobile ad fatigue featuring insights from Kyle. Let's start with the key performance indicators (KPIs) and other analytics to focus on.
7 key performance indicators
To make the most of mobile game analytics, specific metrics will help you gain valuable insights into player behavior, engagement, and overall performance. The data from these metrics will help you see how often and how long users are immersed in your game. This information can help determine revenue projections, such as how likely players are to become paying users, for example.
The data also reveals weak points in your game so you can formulate theories about how to improve them. You can then A/B test and get answers.
If you’re new to the industry, make it a point to memorize these KPIs.
Daily active users (DAU) are the unique number of users who played your game in the last 24 hours. It’s an important metric that tells you how interested players are, plus allows you to identify trends and patterns in behavior.
Weekly active users (WAU) tracks the number of users who have played in seven days, while monthly active users (MAU) covers the last 30 days. MAU is great for tracking longer-term engagement and trends.
DAU, WAU, and MAU can help measure the success of specific campaigns, as well as a certain in-game event or new feature.
2. Average session length
Each time a user plays your game, it’s considered to be a session. Average session length is the number of sessions divided by total session time of all sessions.
Total session time (all sessions)
———————————————— = Average session length
Number of sessions
This metric is useful for comparisons. You can stack it up against previous sessions or an industry average, such as 19.1 minutes per session. Your average session length can also give you a better idea of which areas in your game need improvement.
This is the number of users who play your game and return to the game after the initial session. A high retention rate is a positive indication that players enjoy the game, want to keep playing, and are willing to spend money.
It’s a critical part of user acquisition, engagement, and loyalty. This metric helps you better understand:
- If you’re serving too many ads, which can lead to ad fatigue (i.e., players dropping quickly).
- Game mechanics that appeal to users.
- Whether you need to fix bugs or other technical issues.
Retention is usually marked by time:
- 1 day: The number of users who returned after playing your game for the first time.
- 7 days: The number of users who returned to your game within seven days from the first time they played.
- 28 days: The number of users who come back within 28 days from the first visit. The logic is that if players return after 28 days, they are engaged and are likely to make a purchase.
4. Lifetime value
Known as LTV, this number tells you how interested players remain in your game from the time of initial download and the last time they played. The longer a player stays, the higher their brand loyalty and likelihood of spending money.
LTV should be tracked closely to gauge:
- The success of UA campaigns; generally, LTV should be higher than the cost to acquire a player.
- How much time and money users have spent in a game.
5. Conversion rate
This is the percentage of users who make an in-app purchase or complete a desired action. It measures monetization and helps you discover areas that need improvement for building revenue.
6. Churn rate
This is the percentage of users who stop playing the game over a given period. It measures user retention and helps identify potential areas for improvement in the game. (Also see how to prevent player churn.)
7. ARPU and ARPPU
The average revenue per user (ARPU) is the amount of revenue divided by the number of unique players. ARPPU stands for average revenue per paying user. ARPPU removes free and trial players. Both metrics measure monetization efforts and can help you optimize your strategy.
How data, analytics, and AI help improve personalization
Data and analytics drive every monetization decision Zynga makes. “Players generate thousands of events that help us fine-tune our products and features,” says Kyle. “Many of these events feed into personalization algorithms to predict player behavior and provide even more value for users.”
Personalization in this context is creating a tailored experience based on player preferences and behavior so you can segment. It can include things like:
- Providing the right recommendations on games users may also like.
- Offering rewards based on user behavior and patterns.
- Gameplay skill levels.
- Spending habits.
This information is analyzed to better understand:
- How players interact with your game.
- Which parts they are dropping out of.
- How often they return.
- If they’re interacting with monetization tactics, like making an in-app purchase to level up or buying a better weapon.
Ad experiences can be personalized through cross-promotion. According to Kyle, “Zynga’s portfolio operates like a network. If we know players have engaged with a slots game in the past with a certain level of engagement, they’re likely interested in that genre and might be interested in playing another slots title. We handle that personalized signal by actively prioritizing cross-promotion ads over ads from third-party advertisers.”
Artificial intelligence (AI) individualizes player data and formulates predictive models to better understand what users want from your game. Predictive models can help you figure out in-game purchases, prevent churn, and optimize LTV.
These predictive analysis capabilities are more important than ever, with mobile app users having the choice of opting-in to be tracked. Apple’s App Tracking Transparency (ATT) update in 2021 required mobile developers to ask for user permission to gather tracking data, limiting how much information can be parsed from a campaign performance standpoint.
To improve the player experience, start by identifying pain points with game mechanics. Next, personalize content to align with the player preferences.
Examples might include:
- Understanding drop-off patterns: Can you offer players a tool or extra life to keep them engaged? If so, set a notification with a personalized message at a key moment.
- Finding the right balance of difficulty: Based on player data, you can make the game more difficult or easier. Identify elements users are most and least engaged with. Analyze the specific levels that players are most frustrated with and refine.
- Make more personalized offers: If you know certain players engage with specific game mechanics or characters, tailor offers and products for purchase. For example, you can alert players of updates for certain characters through push notifications to increase engagement or conversions.
Where to start with A/B testing
After you collect data and analyze your key metrics, identify bottlenecks or other problems in your game and use experimentation to get answers.
For example, Zynga focuses on rewarded video engagement across all of its games. “Most of our experiments try to align the rewards users receive from ads with their level of engagement in the games. If a player is at a certain level or coin amount, we want to offer the opportunity to watch an ad for something meaningfully rewarding,” says Kyle.
“This finding is key to keeping rewarded video engagement high as a player progresses through the game.”
Start with a quantifiable goal in mind, such as retention rates of one, seven, or 28 days. If players aren’t making it through the onboarding process on day one, it may indicate the design or messaging (or both) in your tutorial needs work. Pinpoint where players are dropping off in this phase.
When looking at retention-related issues, check:
- Onboarding completion: How many players finish the tutorial?
- Progression funnel: How are users getting through the levels?
- Click-through rate (CTR) of push notifications: Do these alerts get users to come back?
You can also use benchmarks to show how your tests are going; otherwise, it’s hard to gauge performance. For instance, how would you determine if the following retention rates are above or below industry averages?
- Day 1: 40%
- Day 30: 6%
- Average retention rate Day 1 for iOS: 25.65%, Android: 23.01%
- Average retention rate Day 30 for iOS: 4.13%, Android: 2.59%
Once you figure out your trouble areas, come up with ideas to fix them. Maybe you’ve identified devices that aren’t loading the game properly. Perhaps making an adjustment to graphic sizes would allow things to run more smoothly.
If users are visiting your in-game store but aren’t making a purchase, redesign the store’s layout to be more user-friendly. We provide more ways to tweak your creatives and design, below.
Reducing ad fatigue with data
As consumers, we’ve all lost interest after being served too many ads. At that point, the chances of interacting with the ad are nonexistent. This kind of poor user experience often results in players abandoning or uninstalling your game.
A user should see the same ad 1.8 to four times, but often it’s closer to 10.
Kyle points out that ad fatigue may occur if you notice engagement levels for rewarded videos are low. “If users don’t care to spend 30 seconds viewing your rewarded videos, you likely have a value-exchange problem.”
Perhaps trying a different reward or trying a different format could be a way to test what users engage with more. “Offer more compelling rewards that align with the user in their stage of the player lifecycle,” Kyle suggests.
“Looking at rewarded videos and engagement can be an insightful metric to compare across different titles, especially within the same genre, if you’re managing ad monetization across a portfolio of games.”
How to know if users are tired of seeing ads
If your retention levels have taken a dip but your ad campaigns performed really well with lots of installs, your users may be experiencing ad fatigue.
These are other clues that may indicate users may experience ad fatigue:
- Engagement rate decline: Falling CTR.
- Costs are increasing: Metrics such as cost per install (CPI) and cost per click (CPC) are going up.
- Negative reviews: Users take to the App Store or Google Play to complain about too many ads.
If your engagement and retention data are taking a dip, it could be because of too many ads. It may be time to optimize. You can start decreasing ad fatigue by targeting ads that are less disruptive with gameplay and time them correctly with customized offers.
Best practices to reduce ad fatigue
Every business and goal is different. There’s no catch-all strategy. But there are guidelines to consider. Start with the following but tailor them to suit your situation and needs:
- Make a seamless experience: Follow the data to figure out when you can show ads without interrupting the player. For example, in between levels or when there is a natural break in the game.
- Be intentional: Personalization plays an important role because players who see ads that match their interests or behaviors are more inclined to engage.
- Rewarded video ads: Think about how to use rewarded ads more strategically in your game.
- Timing: Use the data to find out when players are dropping out and present the right offer, such as times when extra gems or lives are the most opportune.
- Set frequency caps: If possible, do this on campaign level to ensure you’re not showing too many ads.
Test different variations of ad creatives
Showing the same ad over and over is boring. Creative strategies get stale if you’re not constantly testing and iterating.
After you’ve tested your creatives and have a winner, make more versions of it so you feel more confident when you allocate budgets to your campaigns. Add and test new concepts with your creatives by using different colors, backgrounds, and messaging.
The possibilities are endless, but you can start with:
- Ad types (videos, playable, display)
- Character variations
- Gameplay footage
Keep testing different variations, creating a large number of ads to keep your audience engaged. Check what your audience is writing in their reviews and keep an eye on what your competitors are doing.
Coming soon in part 2, we look at some specific tools you can use to gather data and analyze your mobile games. Stay tuned!
3 tools to analyze your mobile games
With so many tools to gather data and to analyze your mobile games, it can be hard to know which are best suited for your needs. If you’re new to the industry and unsure where to start, consider Google Analytics for Firebase, Unity, and Game Analytics. They’re well known and commonly used to help developers and publishers better understand player behavior by tracking actions in a game, including:
- Level completion rates
- In-app purchase conversion rates
- Ad performance
They also allow you to build and release games, view reports, customize dashboards, and A/B test. However, each one is slightly different, with pros and cons to each, which we will cover below.
1. Google Analytics for Firebase
Google Analytics for Firebase is popular and convenient in that it can integrate with other Google tools such as Google Analytics to help measure performance. It offers a variety of features such as real-time database, authentication, cloud messaging, and analytics, which make it easy for beginners to quickly scale and create high-quality apps or games.
It’s great for building apps and releasing them because it has a fully-managed backend infrastructure. If you’re making an Android game using its Android SDK, for example, you may pick Firebase. Many developers use Firebase with Unity as a game engine, as well.
Firebase is considered one of the less expensive options (you can start for free) for small-scoped games, but some developers complain it’s too barebones and not robust enough for their needs. Others have noted it has limited querying capabilities, prices that can quickly get expensive for high usage, and a lack of support for certain programming languages.
2. Unity Analytics: User-friendly platform for building games
Unity Analytics is known to be a solid game development engine for mobile game developers. Unity is another comprehensive platform that allows you to track and monitor important game metrics such as player progression, length of sessions, and retention rates.
Unity can help you A/B test your different features and designs, and analyze how they affect engagement. Like Firebase, Unity comes with reporting capabilities, customizable dashboards, and the ability to segment players. If you’re only interested in A/B testing, you could use a small plugin to test within Unity only.
However, Unity’s marketplace may not be as robust as some developers would like, and there are technical issues that sometimes arise, such as compatibility issues after updates and incorrect packages for installation.
3. GameAnalytics: Free tool for understanding profitability
GameAnalytics is a free analytics tool for increasing specific KPIs for your entire portfolio of titles. You can use GameAnalytics to assess player behaviors based on certain segments and identify patterns while also seeing a holistic view of your game’s overall performance. The clarity and data visualization includes retention and session length. It also comes with a robust funnel that helps you analyze data more in-depth.
However, GameAnalytics may not be the best at tracking custom events and may require a more manual process for setting up the dashboard.
Best practices for data and analytics
Data and analytics are only as valuable as your team’s ability to understand them and take the necessary steps to optimize and improve. Here are some general guidelines to keep in mind:
- Define your KPIs: Make sure they’re aligned with your goals, which could be engagement, retention, level completion rates, or conversions.
- Use analytics tools: Implement and configure these tools to track elements in your game, such as events, player behaviors, and actions.
- Segment players into different cohorts: You can do this based on demographics, behavior, or spending habits to compare and contrast how specific changes in your game affect things like engagement and retention.
- A continual process: Gathering data, testing, and coming to conclusions is ongoing. Constantly test different features, mechanics, and methods for ad monetization and measure how those changes affect users.
- Prioritize player privacy: Know the data collection practices and then communicate them clearly and obtain user consent where needed.
Common mistakes when using data and analytics
Once you’re testing and collecting data, you’re on the right track to gaining valuable insight into why users are dropping out during onboarding or on day seven. Seeing this kind of concrete information is the best-case scenario when you’re crunching the numbers. But you won’t always come to meaningful conclusions or connections to your hypothesis. It can be frustrating and tedious.
Therefore it’s important to set the right expectations when you’re knee-deep in data and analytics for your game. Some of the biggest mistakes developers tend to make include focusing too much on short-term results and not keeping A/B tests open to analyze the longer-term ad LTV.
Watch out for these other common mistakes:
- Seeking a perfect answer: Your data won’t always provide you with solutions that help you reach a conclusion or a decision. Even if it’s not 100% accurate, trust the process and keep in mind that you may uncover answers later on.
- Don’t be afraid to retest: Different analytical tools have varying metrics definitions, so be sure you know what those are before you launch into experimentations. If you feel like the data might be wrong, test it again or use another method. For example, you can download the raw data and perform the analysis manually to reach a conclusion.
- Starting with error-prone data: Skipping data cleansing can lead to inaccurate or flat out wrong results. Be sure to scrub the data so you understand what is lacking and what could be incorrect or limiting during analysis.
Tips for collecting data and analyzing it effectively
You don’t need to test everything. Only test what matters and where you want to improve, such as the drop-off rate in your tutorial and re-engagement efforts around that.
Here are a few tips to help you along in your process:
- Define events: These can be in-game purchases or completing onboarding, but you could also look at things like which characters players chose or level completions.
- Create funnels: Calculate metrics such as conversion rates by establishing specific events within the player’s experience. Analytics platforms may allow you to customize events, such as when a user logs in and when they make a first purchase. Go granular, and analyze the conversions between these steps to see where funnel drop-offs are happening. You can then optimize and become better informed when designing future in-app events.
- Stay on top of your ad creatives: Know which ad creatives perform the best and what sources bring in the highest-value players. Pinpointing channels and sources lets you see what areas you can focus on to re-engage users.
- Predict long-term performance: When you see signs of early engaged users, you can save time by using the data and analytics to predict how certain campaigns will perform.
Mobile game analytics can better equip you to understand player behavior. By knowing your players, you put yourself on the path to increase monetization and drive higher retention while decreasing ad fatigue. You’ll be able to confidently offer a personalized experience that caters to your players’ preferences.
Overcoming hurdles in your mobile game will require using the right tools, staying current with industry trends, and adopting a testing mindset. A combination of all three will help support and drive important decisions to provide the best user experience possible and drive up revenue to new heights.