Valve has found some creative ways to impact player behavior in Dota 2. Having hired psychologist Dr. Mike Ambinder, the studio has been changing seemingly minor aspects that have ended up having a big impact on players. Ambinder gave one particularly sneaky example recently, shedding some light on how these player experiments work.
Speaking at Glitch Gaming (via The Escapist), Ambinder said that they shifted the questions from a general one about the match quality to more specific questions. One asks about how cooperative your teammates were, while the more crucial one asks how cooperative you were. That's where the manipulation comes in.
"What's going to happen, though, is that if you rate yourself really highly as a teammate, 'Yeah, I'm a five-star teammate' but you weren't, because you know that you were being a dick in the game, you're going to go into what psychologists like to call cognitive dissonance. When you're holding two conflicting notions in mind, right? One, that you were actually a bad teammate, and two, that you rated yourself highly as a good teammate.
"You're going to want to answer the question honestly. You're going to want to actually rate yourself accurately as a good teammate, so the hope is that because you had this dissonance between how you want to rate yourself and how you actually behaved, it's going to push your behavior more towards wanting to be a more cooperative teammate."
So the question isn't really meant to gauge player behavior, but rather to change behavior because players know the question is coming. And it seems to have worked, because Ambinder says that those questions correlated with a 10-15% reduction of in-game reports. Those clever head scientists.