Steam Customer Review Changes Could Be Bad News for Crowdfunded Games

The changes made by Steam to their score system on September 12th may seem fairly minor, but there are some potentially major ramifications for crowdfunded games. User scores on Steam will no longer include reviews linked to key-activated games. Which, of course, is how almost all rewards for Kickstarter and Indiegogo games are distributed to backers.

This new initiative by Steam is in response to concerns that certain developers were manipulating their scores by distributing game keys to fake accounts. While the move to end this shady practice may have good intentions, it seems like a bit of an over-reaction. In fact, Steam officials admit only a very small percentage of developers were bending the rules. Indie developers, in particular those who who use crowdfunding to help finance their games, have some very legitimate concerns.

Fun With Filters

The initial worry is that some game review scores may fall as the reviews from enthusiastic and supportive backers are omitted. However, as long as the backers were being honest in their reviews this shouldn’t be a problem. And thanks to the new review filtering options also introduced by Steam, it’s possible to confirm this. I checked a number of the more well known Kickstarter games (Pillars of Eternity, Massive Chalice, Wasteland 2, Shadowrun Returns, Divinity: Original Sin, The Banner Saga, Shovel Knight, FTL) and the overall Steam score had either remained the same or seen a change of 1% – either up and down.

Even for the games that have far fewer user reviews, (and with a bigger proportion of those from key activations rather than purchases), the overall scores haven’t really changed. I examined Chaos Reborn, Sunless Sea, Concrete Jungle, Sheltered, That Dragon Cancer, Jotun, Jagged Alliance: Flashback, Superhot and Sheltered to verify this. Even Orion Trail, with only 110 user reviews of which 23 are from key activations, seemed unaffected.

However for more recent releases there can be more of an impact, as a greater proportion of those reviews will presumably be from backers. Halcyon 6: Starbase Commander sees 249 of its 603 reviews affected, and its overall score drop from 86% to 80%. Things are even worse for Kim, (admittedly still in Early Access), which loses 21 out of its 25 reviews and sees its score fall from 92% to 80%.

Mighty No. 9 benefits even more

It works both ways however. I’ve already mentioned how some games have seen a slight increase but Mighty No. 9 benefits even more than that. With over a third of its 1,500 reviews from key activations and no longer taken into account, the overall score has jumped from 47% to 60%. While it’s still not a great score, it shows how this new policy from Steam can have a major impact.

An Uncertain Future

The major concern from my point of view however, is how this will impact new releases. In all the previous examples the games did at least temporarily benefit from the early momentum of backer reviews. The Steam marketplace is extremely crowded and competitive, so without a high overall score it’s hard to get noticed. Without getting noticed it will be difficult to generate reviews – an unfortunate catch-22 scenario.

There also seems to be an existing barrier put in place by Steam, whereby a game can’t earn a label of ‘very positive’ unless it has at least 50 reviews. Similarly the ‘overwhelmingly positive’ designation is unavailable until 500 reviews have been received. Without the extra boost generated by backer reviews, it will be increasingly difficult for smaller Kickstarter releases to get an early foothold in these upper tiers.

It’s worth mentioning that although reviews from key-activated games will no longer count towards the overall score, the actual text comments from the reviews are still available. But will people feel as motivated to leave a review if they know it won’t contribute to the overall score? I imagine it will definitely have an impact, although it’s hard to say at this stage.

The new Steam policy is unlikely to have a significant impact on the long-term success of crowdfunded games, but it’s certainly making things more difficult for them in the short-term. I’d like to think Steam will listen to the concerns of smaller developers, but realistically I can’t see that happening. Developers will have to adapt to this new policy and ensure the project they’ve worked so hard on doesn’t immediately sink after launch.