Data-Driven Design
Conducted qualitative research to generate insights that defined quantitative data collection which informed design choices.
Goal
Drive long-term player engagement through a competitive ranking system.
Problem Statements
We needed to balance competitiveness and accessibility to ensure the ranking system motivated all player types and minimized churn.
Results
+40%
User retention in Ranked Play
+300%
User engagement in Ranked Play
88%
Metacritic score
Team Make-up
Ranking System
The ranking system consisted of seven progressive tiers: Bronze, Silver, Gold, Platinum, Diamond, Master, and Grandmaster.
Group by Population
Design
The first five tiers were evenly distributed by population, 20 percent each, while Master and Grandmaster represented the top 2 percent and 1 percent. This structure balanced competitive progression with a perception of fairness across all player types.
User Research
Players appreciated the fairness of the population distribution and found it intuitive. They loved the tier icons, but many felt they were getting stuck and not progressing through the ranks.
Group by Skill
Data
Forum feedback highlighted which players were frustrated, and analysis of their MMR showed most were in Bronze or Diamond. Bronze players struggled to progress, while Diamond players were promoted to that tier and felt they had reached the end, knowing they were unlikely to advance to Master despite strong performance. The data showed that both groups were actually improving their skills significantly.
Design
I concluded that grouping by population was causing the issue and that organizing tiers by skill would be more effective. A key consideration was player perception, so I engaged with the community about the proposed change. While some players worried it might make Bronze players feel even more discouraged, most were pleased at the prospect of advancing into higher tiers.
Reception
After playing the updated system, players responded positively. They focused on their own progress rather than comparing themselves to others and appreciated that they were advancing in a meaningful way.
Continuous Feedback and Iteration
User Research
As time went on, we continued to gather feedback and noticed that some players still felt they were getting stuck. Analysis showed that these concerns were primarily coming from Bronze and Silver players.
Data
Analysis of the data revealed that Bronze players were improving their skills significantly but were not receiving meaningful rewards. In contrast, Platinum and Diamond players were experiencing similar skill growth but were highly satisfied with the system.
I monitored online discussions daily to capture user sentiment.
Personas
Based on these insights and additional analysis, we divided the player population into five data-driven personas.
Bronze / Silver
Who: Value winning even a single match.
Findings: High drop-off. Progress feels slow.
Design Implications: Frustration and limited satisfaction.
Platinum / Diamond
Who: Demonstrate a deep understanding of skill.
Findings: Moving between tiers enhances the sense of accomplishment.
Thoughts: Design should reinforce progression milestones.
Gold
Who: Players at the peak of the bell curve.
Findings: Happy with placement.
Thoughts: Progression should be intentionally vague.
Master / Grandmaster
Who: The top 5% of players, highly competitive and focused on skill mastery.
Findings: Only concerned with their exact rating.
Thoughts: Engagement for this group depends on features that reflect true skill.
Design
Groups by data-driven design
Rather than grouping players strictly by population or skill, we organized tiers based on insights derived from interpreting the data.
Ranking System
Bronze, Silver, and lower Gold tiers used a progression-based system, allowing players to advance even with a win rate below 50 percent.
