Steam again modifying its customer review process, irking some indie devs

Valve is trying to target inflated scores and and 'deceptive tactics.' 

4

In its ongoing effort to smooth out issues with its customer review process, Steam is modifying its filters for users to sort through reviews. Most telling is that recent and overall score will not count reviews from users with a Steam key.

The change, detailed in a blog post, appears to be in response to feedback that some developers have been inflating their scores by giving out batch Steam keys, with Valve questioning just how legitimate some of those reviews can be. The company said that "the review score has become a point of fixation for many developers, to the point where some developers are willing to employ deceptive tactics to generate a more positive review score." They listed giving out Steam keys in exchange for positive reviews and using Steam keys on alternate accounts to generate multiple reviews as problem areas. Some organizations have even been created with the express purpose of selling positive reviews. "It is too easy for these keys to end up being used in ways that artificially inflate review scores," the post said.

Valve analyzed its library of games and found at least 160 titles with "a substantially greater percentage of positive reviews by users that activated the product with a cd key, compared to customers that purchased the game directly on Steam." They acknowledge that there are some legitimate reason for that, but abuse "is clear and obvious, such as duplicated and/or generated reviews in large batches, or reviews from accounts linked to the developer. In those cases, we've now taken action by banning the false reviews and will be ending business relationships with developers that continue violating our rules."

Valve said the change will affect about 14% of the games on Steam, with some seeing changes in score both up and down. "Most changes in the review score category are a result of games being on the edge of review score cut-offs such as 69% positive or 70% positive. A change of 1% in these cases can mean the difference between a review score category of 'Mixed' and 'Positive'. About 200 titles that only had one or two reviews will no longer have a score at all until a review is written by a customer that purchased that item via Steam. In all of these cases, the written reviews still exist and can easily be found in the review section on that store page."

Some indie developers (via Gamasutra) found the changes disconcerting, saying that it would be a severe blow to their effeorts to get their games noticed.

Valve said it is still evaluating its review process, with even more changes coming. It just needs to figure out the best way to implement a change, such as the case of some games that have a positive score, but only negative reviews were marked as helpful. They are also looking at ways to ensure that users marking items as helpful don;t have excess influence in the process.

Contributing Editor
From The Chatty
  • reply
    September 14, 2016 8:48 AM

    John Keefer posted a new article, Steam again modifying its customer review process, irking some indie devs

    • reply
      September 14, 2016 9:14 AM

      That's funny since they don't do anything about the trolls that play a game for 40 hours and then claim they hated every second of it. Oh and demand a refund on top of that.

      • reply
        September 14, 2016 9:16 AM

        I don’t think there is ever a way to truly stop review systems from being gamed. Just do the best you can to eliminate the obvious ones, but at some point you have to trust the reader to figure it out. Showing time played helps as you noted.

        I think the main focus is trying to get a system for better sorting since the thumbs up versus thumbs down is not really serving its purpose.

      • reply
        September 14, 2016 10:42 AM

        And they didn't remove how easy it is for people to move all the negative reviews on to the front page by upvoting them.

    • reply
      September 14, 2016 10:14 AM

      I was looking for a composite to HDMI converter yesterday on Amazon and I saw a ton of positive reviews, only to find that the overwhelming majority of them were from people who received the product at a discounted or free rate. At least they're required to say so in the review, but how many don't? There should be a way to filter that shit.

      If you're bored check out the positive comments on this item. Most of them mention that they received the item in exchange for a review:

      https://www.amazon.com/Akale-Composite-Converter-Adapter-Blue-Ray/dp/B01HFGUQUK/ref=sr_1_4?ie=UTF8&qid=1473873106&sr=8-4&keywords=composite+to+hdmi+converter

      • reply
        September 14, 2016 10:14 AM

        I'm not saying indie devs do this, by the way, the article just reminded me of what I saw yesterday.

Hello, Meet Lola