Bungie Explains Halo 3 NXE Install Slowdown

14
The New Xbox Experience's ability to load games from the hard drive is intended to help games load more quickly or even make them look better, but Bungie's Halo 3 is an exception--it actually loads more slowly than without installing the game.

To explain the phenomenon, Bungie tapped engineer and "Chief Caching Officer" Mat Noguchi to give both a technical explanation and the "short answer." The short answer is in essence that the game has always used the Xbox 360's hard drive to shorten load times:

The Xbox 360 HDD has a section for games to use called the utility partition. Games can use this section for whatever they want to; Halo 3 uses the utility partition to cache maps as they will load faster off the HDD ... Think of it as an on demand install of Halo 3 to some scratch space on the HDD.

Asked whether Halo 3 would receive an update to take advantage of the new system feature, Noguchi all but shot down the idea. "It would be a significant undertaking" to do it, wrote the engineer, adding that it would be too great a risk for too little a reward especially in light of the studio's work on Halo 3: Recon.

Noguchi's highly technical account is reproduced in full below:

The Xbox 360 HDD has a section for games to use called the utility partition. Games can use this section for whatever they want to; Halo 3 uses the utility partition to cache maps as they will load faster off the HDD than off the DVD. As a side note, the utility partition can be deleted when other games are played. This is why maps can take longer to load when you play another game in between various Halo 3 sessions. (As was the case with Halo 1 and Halo 2.)

So when Halo 3 runs, if a HDD is present, we copy maps from the DVD to the utility partition (on the HDD). Think of it as an on demand install of Halo 3 to some scratch space on the HDD. Halo 3 doesn't actually know where it's running from, so it always assumes it's running from a DVD. This is an unfortunate consequence of new features (namely, install to HDD) being added to the Xbox 360 after Halo 3 shipped. And as a result, it means that even if Halo 3 is already installed to the HDD, it will still copy maps to the utility partition.

So then the real question is why is copying from HDD to HDD slower than copying from DVD to HDD? In the first case, you are reading from one I/O device (HDD) and writing to the same I/O device (HDD). In the second case, you are reading from one I/O device (DVD) and writing to a different I/O device (HDD). In the first case, because we are reading and writing to the same device, the total copy time is the amount of time it takes to read the map plus the time it takes to write the map. Ultimately this is because for the HDD, you read and write through the same mechanism, i.e., the hard drive read/write head, and those reads and writes cannot occur simultaneously through a single mechanism. (If they could, it would be awesome, and I wouldn't have to document any of this. Unfortunately, it's not easy to do for other reasons.) For the second case, because we are reading and writing from different devices, the total copy time is the larger of the total time to read the map and the total time to write the map. This is because we can read from one device (DVD) at the same time we are writing to another device (HDD). There is some overhead because you can't start writing data until you read it, but it is dwarfed by the time to copy a map. (About 128k vs. 500 MB or a ratio of 4000:1.)

There are other factors that contribute to load times, such as preloading data from the map you are about to start so that you don't have to sit through a non-interactive loading screen; the only difference in load times occurs when we copy a map that has not been cached to the utility partition or has been evicted from the utility partition because some other maps were run. Once a map has been cached, the time to load it will be identical to running Halo 3 off the DVD with a HDD.

From The Chatty
  • reply
    November 19, 2008 4:39 PM

    I like that they explained it more, but it still seems like they could simply check whether Halo 3 is installed, and if it is, load it from the HD. It sounds more like they don't want to do it.

    • reply
      November 19, 2008 4:52 PM

      Like he said too big risk not enough reward....Sure the install times would increase abit, but then you lose harddrive space and you really get no gain.

      They also have to put in quiet abit of work it wouldn't be as simple as saying, if(Halo3 == playing) don't load....they have to take the whole feature out of loading from the disc. Which could cause so many bugs.

      I like his explanation and it actually seems to make sense (and be valid)
      It really is too bad that by making their game better before it is now worse off though.

      • reply
        November 19, 2008 4:55 PM

        If it shouldn't happen, the dashboard should block installation.

        • reply
          November 19, 2008 4:58 PM

          o....get microsoft to block the install i thought you meant get bungie to make it not load its maps.

          Um. Yea I guess I don't really see a downside to that, and it would be as simple as saying if (Halo3 == in) don't enable install

          • reply
            November 19, 2008 6:12 PM

            Yeah, that was not really related to the parent post. It was just an idea I thought of later. Live users would download an install blacklist that removes the install option for problematic games.

    • reply
      November 19, 2008 5:14 PM

      Halo 3 doesn't actually know where it's running from, so it always assumes it's running from a DVD.

      And I don't think the games *can* know if it's run off the DVD or harddrive. And this is why we can install any game, even old ones, because to the game it thinks it's still running off the DVD.

      • reply
        November 19, 2008 5:30 PM

        Hopefully there is a new API for the games to know so that they can decide whether to do Halo 3-style caching (when running from the DVD) or not bother (when installed on the HDD).

        There's no way for old games to know since this is a new feature but it would be kinda silly if MS didn't provide a way for new games to detect it, I think.

    • reply
      November 19, 2008 7:19 PM

      It does load from the hard drive. The issue is that the original caching scheme is designed to put data onto the hard disk to load from, thus speeding load. If you are really smart about what you put on the hard drive and load, and what you continue to stream off the DVD, you can have better performance than purely playing off the hard drive. The slow part of reading sata off a drive is that the heads have to be phyisically moved to the track the data you need resides on.

      Installing to the hard drive then playing from the hard drive means that the seeks that were previously split between the DVD and hard drive are all off the hard drive, and many of them exist purely for the purpose of copying data from the main store on the drive to the utility partition, wasting a lot of throughput.

      If this feature had existed before Halo 3 shipped, then there could possibly have been an API call which would indicate whether the game had been installed to drive and they could have chosen to disable caching, but the feature didn't exist before the game shipped, so how are they supposed to have checked for it?

Hello, Meet Lola