Oblivion uses the X360 hard drive extensively to cache (copy and reuse) game data. Quite a few people have complained about excessive load times, often caused by the cached data being overly fragmented. To solve this problem, reset your Xbox, and hold down the A button as the game is booting up, which will clear its hard drive cache, and create a new one. Once you see the Bethesda Softworks logo video, the game is already loaded, and the cache should have been cleared. This does not affect any save games.
More articles about The Elder Scrolls IV: Oblivion