Valve issues statement about Christmas Day Steam kerfuffle

Following a major security disaster on Christmas Day, Valve has responded directly to Steam users.

33

On Christmas Day, Steam had one of its worst days ever, with users able to see sensitive account information of other Steam users. That led to people being able to see Steam libraries, Wallet information, and even email addresses. While Valve issued a vague statement to the gaming press, it had yet to address the Steam user base directly. Today, the company explained to the users exactly what happened.

The following was posted on Steam, noting that what happened was not the direct result of a DDoS attack, but rather resulted from a response to a DDoS attack:

On December 25th, a configuration error resulted in some users seeing Steam Store pages generated for other users. Between 11:50 PST and 13:20 PST store page requests for about 34k users, which contained sensitive personal information, may have been returned and seen by other users.

The content of these requests varied by page, but some pages included a Steam user’s billing address, the last four digits of their Steam Guard phone number, their purchase history, the last two digits of their credit card number, and/or their email address. These cached requests did not include full credit card numbers, user passwords, or enough data to allow logging in as or completing a transaction as another user.

If you did not browse a Steam Store page with your personal information (such as your account page or a checkout page) in this time frame, that information could not have been shown to another user.

Valve is currently working with our web caching partner to identify users whose information was served to other users, and will be contacting those affected once they have been identified. As no unauthorized actions were allowed on accounts beyond the viewing of cached page information, no additional action is required by users.

Early Christmas morning (Pacific Standard Time), the Steam Store was the target of a DoS attack which prevented the serving of store pages to users. Attacks against the Steam Store, and Steam in general, are a regular occurrence that Valve handles both directly and with the help of partner companies, and typically do not impact Steam users. During the Christmas attack, traffic to the Steam store increased 2000% over the average traffic during the Steam Sale.

In response to this specific attack, caching rules managed by a Steam web caching partner were deployed in order to both minimize the impact on Steam Store servers and continue to route legitimate user traffic. During the second wave of this attack, a second caching configuration was deployed that incorrectly cached web traffic for authenticated users. This configuration error resulted in some users seeing Steam Store responses which were generated for other users. Incorrect Store responses varied from users seeing the front page of the Store displayed in the wrong language, to seeing the account page of another user.

Once this error was identified, the Steam Store was shut down and a new caching configuration was deployed. The Steam Store remained down until we had reviewed all caching configurations, and we received confirmation that the latest configurations had been deployed to all partner servers and that all cached data on edge servers had been purged.

We will continue to work with our web caching partner to identify affected users and to improve the process used to set caching rules going forward. We apologize to everyone whose personal information was exposed by this error, and for interruption of Steam Store service.

Senior Editor

Ozzie has been playing video games since picking up his first NES controller at age 5. He has been into games ever since, only briefly stepping away during his college years. But he was pulled back in after spending years in QA circles for both THQ and Activision, mostly spending time helping to push forward the Guitar Hero series at its peak. Ozzie has become a big fan of platformers, puzzle games, shooters, and RPGs, just to name a few genres, but he’s also a huge sucker for anything with a good, compelling narrative behind it. Because what are video games if you can't enjoy a good story with a fresh Cherry Coke?

Filed Under
From The Chatty
  • reply
    December 30, 2015 3:57 PM

    Ozzie Mejia posted a new article, Valve issues statement about Christmas Day Steam kerfuffle

    • reply
      December 30, 2015 4:00 PM

      I love the use of kerfuffle in this headline. 10/10, would Ozz again.

    • reply
      December 30, 2015 4:03 PM

      5 days later is pretty pathetic for them to address that there was an issue but at least they did. So I guess that's something.

      • reply
        December 30, 2015 5:21 PM

        [deleted]

        • reply
          December 30, 2015 6:20 PM

          They responded to some questions. They didn't put anything official on their site.

          • reply
            December 30, 2015 7:05 PM

            I'd wager they'd want to analyze who as actually affected, and how sever the problem was. Making a statement before and then having to clean it up isn't going to make you look any more competent. It's like posting a breaking news story first, except half the information is incorrect.

            • reply
              December 30, 2015 7:41 PM

              No, that's not true. "We're investigating a situation where X happened and will update when we have more information" is all it would have taken.

              • reply
                December 30, 2015 7:43 PM

                Which is exactly how LastPass and other companies have handled it. Silence for 5 days is not particularly good.

              • reply
                December 30, 2015 7:45 PM

                Their responses to the news sites didn't indicate that they were doing any further investigation so even if you did see those articles all you saw was "Yeah, we messed up a configuration, it's fixed now." which is a pretty bad response.

                • reply
                  December 30, 2015 8:13 PM

                  I'd imagine they're busy working on the problem and trying to restore normalcy on Christmas day. Communicating to news sites is a poor way to get that information out anyway. Gathering information about the scope of the problem and getting that directly to your users is ideal.

                  • reply
                    December 30, 2015 8:15 PM

                    The same people investigating the issue are not the same people that could have put out an announcement that there was an issue and that they are looking into it. That's a horrible excuse.

                    • reply
                      December 30, 2015 9:06 PM

                      I know when I'm trying to solve some major problem at work, I always work faster and better when people are pestering me for updates so they can go tell someone else that shit is still broken.

                      • reply
                        December 30, 2015 9:09 PM

                        There's no need at all to involve any of the people investigating the issue.

                  • reply
                    December 30, 2015 8:17 PM

                    With LastPass I didn't get up to the minute information about what data was accessed. Rather after the problem had been rectified they sent an email with details about what had happened, and steps I'd need to take to protect myself. Pretty similar to what's happening here.

                    • reply
                      December 30, 2015 8:19 PM

                      You didn't get up to the minute information but they publicly acknowledged that an issue occurred on their site and that they were looking into it.

                      • reply
                        December 30, 2015 8:41 PM

                        they released a statement to media. sorry it didn't live up to your expectations or land in the place you thought proper. maybe you should delete steam in protest.

                        basically, the very small vocal minority that has a beef about this wouldn't even hurt them if they all stopped buying today. they, like any business, are addressing their largest consumer base.

                        how much credence do you think microsoft would give a 2% userbase of abrasions not happy with their edge use cases? you're that userbase write now. this was a breach classified as 'minor' and received the appropriate response.

                        you work software, right? how do you not understand that security bugs or any bugs in general are classified based on risk, impact, and business value?

                        • reply
                          December 30, 2015 8:46 PM

                          s/write/right/

                        • reply
                          December 30, 2015 8:49 PM

                          I do understand and the *only* thing I'm asking for is that they told the people who were asking questions that they were investigating instead of the hand waving statement that they issued.

                          They didn't have to send an email to everyone with an account saying something happened. Personally, I would have liked to see an official blog post stating that something happened and they were investigating. But I would have been happy with the replies to the forum posts and news inquiries saying that they were investigating and would provide further updates.

                          The "It was a configuration issue and it's fixed now" response that they had was in no way sufficient.

                          • reply
                            December 30, 2015 9:25 PM

                            Why would a blog post that a user may not know about be the best way for a user to get information about something that could personally affect them? That's silly.

                            What is odd is all of the things you're saying they should have done, they did. You've stated they should make statements to media. This very site reported on the indecent hours after it happened, with statements from Valve about who was affected, what they were doing about it, and what users should do.

                            • reply
                              December 30, 2015 9:37 PM

                              I don't think you actually read my post.

                              • reply
                                December 30, 2015 9:42 PM

                                You going on about news sites:

                                Their responses to the news sites didn't indicate that they were doing any further investigation so even if you did see those articles all you saw was "Yeah, we messed up a configuration, it's fixed now."

                          • reply
                            December 30, 2015 9:50 PM

                            [deleted]

                      • reply
                        December 30, 2015 9:44 PM

                        Original Tweet

                        By the way, this is not a security breach. This is page caching gone rogue. Most likely not respecting Cache-Control headers.
                        3:57 PM - 25 Dec 2015

                        Update

                        Update (5:33PM PT): Valve has issued the following statement to Shacknews:
                        Steam is back up and running without any known issues. As a result of a configuration change earlier today, a caching issue allowed some users to randomly see pages generated for other users for a period of less than an hour. This issue has since been resolved. We believe no unauthorized actions were allowed on accounts beyond the viewing of cached page information and no additional action is required by users.

                        Not even 2 hours

                        • reply
                          December 30, 2015 9:45 PM

                          Actually it's probably closer to 5 hours since I should be looking at post time.

                        • reply
                          December 30, 2015 9:47 PM

                          @SteamDB is not Valve, FYI.

                          So the actual Valve statement says nothing other than "We fucked up and fixed the issues that allowed users to see $vaguedescriptionthatdoesn'tmentionpersonalinfo, that's all, move along.". Which is not good.

          • reply
            December 30, 2015 7:08 PM

            forensics takes a while.

      • reply
        December 30, 2015 6:59 PM

        I agree with you on principle but if it took a few days to be certain as to exactly what happened I can accept a late response thatis more accurate.

      • reply
        December 30, 2015 7:42 PM

        What did you want them to say? lol we don't know what happened wtflol or wait for a proper reply when they have info?

        • reply
          December 30, 2015 7:45 PM

          If they didn't know at the time, they should've put a notice up in the Steam client, and disabled all profile activities or god forbit, actually shut the service down for the couple hours it took to sort it out.

          • reply
            December 30, 2015 7:46 PM

            They did shut it down.

            • reply
              December 30, 2015 7:50 PM

              I saw the reddit / Shack threads that day before I'd used Steam yet for the day, so I couldn't confirm personally (I just exited the client) but the complaints I was reading was that it was up for a couple hours while this issue persisted.

              • reply
                December 31, 2015 1:13 AM

                Correct. Evaluation takes time, imagine that.

          • reply
            December 31, 2015 1:12 AM

            Oh, you mean like the time period they did actually shut it down?

      • reply
        December 30, 2015 7:52 PM

        No it's not. You take the time you need to make a definitive, clear statement. Anything sooner is "we're not sure and are investigating" or "we currently think X but need more time to confirm".

        • reply
          December 30, 2015 7:52 PM

          The latter is exactly what they should have done and didn't do.

          • reply
            December 30, 2015 8:21 PM

            You should NEVER say what you think it is until you're sure. If you say you think it's X but it turns out to not be X then people will complain about X for years. People will discuss how your system failed because of X. Whatever X is it will never, ever be forgotten. No matter how many posts you put up saying how it WASN'T X will stop people from writing about how you fucked up X.

            • reply
              December 30, 2015 8:25 PM

              I never said they had to say WHAT happened. They just had to say SOMETHING happened and that they were looking into it.

              http://www.shacknews.com/article/92647/steam-bug-causing-user-account-information-to-appear-to-other-customers-update-valve-responds

              See Update 3.

              If that statement was on their site with one more sentence at the end saying "We are investigating the issue and will respond with more information when we have completed the investigation" I wouldn't have had any issue.

            • reply
              December 30, 2015 8:29 PM

              This is, they were saying things, but only if you asked.

              That's a foul for the 99% of folks that didn't ask.(or know to ask.)

            • reply
              December 30, 2015 9:53 PM

              As the (only) software programmer for a GPS tracking company that deals with 2k+ trucks carrying millions of dollars in goods, i have to TOTALLY agree with you on that. Besides, unless actual credit card information was displayed or there was massive outage for more than a couple of hours, pasting that kind of news all over your business on your most profitable week of the year will only make your income drop to zero for weeks, even if the problem is already fixed, or did only affect a small ammount of customers to start with.

              • reply
                December 31, 2015 2:07 AM

                this is the worst username I ever seen
                why didnt you add xxx's to the beginning and end of your name

      • reply
        December 31, 2015 8:15 AM

        5 days from impact to press release is not that bad. You would prefer them to issue a PR before they've fully identified the problem, scope and resolution?

    • reply
      December 30, 2015 4:05 PM

      10/10 would kerfuffle again

    • reply
      December 30, 2015 5:08 PM

      I don't even know what that means

    • reply
      December 30, 2015 5:23 PM

      What a mess that was, I couldn't login for days, and even when I could I got missing pages, errors, etc.

      • reply
        December 31, 2015 1:42 AM

        The problem only lasted an hour and a half. Sounds like you had other issues.

      • reply
        December 31, 2015 7:45 AM

        Days?

    • reply
      December 30, 2015 6:51 PM

      I wonder how often and how much bullshit is slung at Steam servers in an attempt to cause disruption and what sort of uber dudes they have working there to prevent that sorta stuff. In fact I wonder how big Steam really is compared to Amazon/Google in the whole scheme of things.

      • reply
        December 30, 2015 7:01 PM

        Steam is big, but compared to Google and Amazon, it's kind of a smashed bug on the windshield I'd think.

      • reply
        December 30, 2015 10:00 PM

        Steam pales in comparison to amazon. Netflix runs on AWS and they alone account for something like 40% of internet traffic at peak.

      • reply
        December 31, 2015 10:49 PM

        Fuck the guys who make Valve devs spend Christmas holidays at the office, deflecting attacks.

        Even if the devs who are doing the work don't celebrate Christmas, they are likely in the Pacific Northwest which is cold and wet.

        They better not give up and sell Steam to Microsoft for 2 billion dollars so it can be used in AR demos to hype Windows 11.

    • reply
      December 30, 2015 7:16 PM

      Who is the web caching partner?

      • reply
        December 30, 2015 8:07 PM

        They use akamai for some assets, so might be them.

    • reply
      December 30, 2015 7:31 PM

      [deleted]

Hello, Meet Lola