Activision’s CoD skill-based MMR experiment flopped

Matches Made in…Heaven?

Activision, the developer behind the popular Call of Duty franchise, recently conducted a significant experiment within Call of Duty: Modern Warfare 3. The experiment involved a temporary reduction in the importance of skill-based matchmaking (SBMM) for a portion of North American players. This two-week trial aimed to assess the impact of altering the matchmaking algorithm on player retention. The results, detailed in a 25-page white paper, paint a stark picture of player reaction to the change. Activision’s experiment ultimately demonstrates a crucial role for skill-based matchmaking in the long-term health and viability of Call of Duty multiplayer.

Currently, Activision’s matchmaking system prioritizes factors beyond raw skill, including connection quality, match entry time, and recent map/mode data. However, skill itself remains a significant component. A sophisticated calculation considers individual match data like kills, kill/death ratios, and even kill/death ratios relative to enemy skill levels. This intricate algorithm adjusts skill ratings after each match to provide a dynamic and responsive system that adapts to player performance fluctuations. The goal is to create a balanced playing experience where players don’t face severe skill mismatches and where fluctuations in skill are accounted for.

The experiment, focusing on reducing the weight of skill in the matchmaking algorithm, had a demonstrably negative impact on player retention. Data from the two-week test clearly indicated a substantial drop in the percentage of players who returned to the game. The study revealed that player retention plummeted for the majority of players who experienced the test. While a minority of the highest skilled players showed a slight increase in return rate, the overall trend highlighted the negative consequences of deemphasizing skill-based matchmaking on overall player retention. The study suggests that removing skill-based matchmaking, instead of improving player experience, would likely lead to a significant decline in the player base, eventually affecting the game’s long-term appeal and viability.

Skill Issues

• Activision’s experiment also included a period of tightening skill-based matchmaking parameters. Unexpectedly, this stricter application negatively affected high-skilled players’ return rates, while lower-skilled players showed increased retention.
• These findings suggest a crucial balance between player skill and the overall gaming experience.
• The experiment also revealed a correlation between reduced SBMM and an increased frequency of “blowouts.” In other words, when SBMM was reduced, games became significantly more lopsided, with one team often overwhelmingly victorious. This outcome is widely considered less enjoyable for the majority of players.
• Conversely, the experiment resulted in a lower kill-per-minute rate in matches with players of varied skill levels. This demonstrates that the skill-based matchmaking algorithm aids in a more engaging experience overall.

These data points support the hypothesis that while skill-based matchmaking may initially seem disadvantageous to some players, its removal results in an overall degradation of the player experience. Players will eventually leave the game at higher rates if a skill-based matchmaking system is not in place. Activision prioritizes the health of its player base and actively aims to keep the overall experience fair.

Is SBMM Good or Bad for FPS Casual Playlists?

The debate surrounding skill-based matchmaking (SBMM) in first-person shooter (FPS) games, particularly in casual playlists, continues to resonate within the online gaming community. While some players advocate for a system that rewards skill, others find the system to be an obstacle in their pursuit of a more balanced and enjoyable gaming experience. The controversy stems from the fundamental tension between player satisfaction and the need for a functioning and sustainable online game.

Skill-based matchmaking has both supporters and detractors. Some argue that SBMM ensures that players are matched with opponents of similar skill, leading to more competitive and rewarding matches for those who prefer an elevated challenge. The idea that SBMM is an obstacle in casual gaming is also an idea often presented by players. It argues that this feature disrupts the casual atmosphere and detracts from the relaxing and fun aspects of casual play, leading players to seek games with less strenuous challenges. The tension arises between these divergent perspectives.

Activision’s recent experiment highlights the critical role of skill-based matchmaking in maintaining a healthy and sustainable online player base. The experiment’s results suggest that the algorithm in place does a good job of maintaining a stable player experience across many skill levels. These results suggest that skillful players are not punished or rewarded by being matched with those of similar skill, but their experience will likely be worse if SBMM is removed from the game entirely. This experiment provides valuable insights into the complexities of designing engaging and fair online gaming experiences.