Jeff pretty much nailed the reason we don't have more whimsical or creative worlds in games these days: they just aren't as successful lately.
The reason? I think they're less accessible. I love creative, weird worlds because they provide me with new places to explore and abilities to play with. But for the average person, they just see a bunch of rules that they have to learn.
Garnett - By your own admission you preferred Company of Heroes to Dawn of War, because you intuitively know more about the WW2 universe. You know what type of guns can do what, so you can immediately jump in. You have less to learn, less to be confused about, so you can just appreciate the gameplay and strategy. For me, somebody who really doesn't know anything about WW2 tech, I preferred the DOW universe with all its varied races and abilities.
I go back and forth with my friends on CoD and Halo. I prefer Halo, they prefer COD, and we drag each other into games of each. They HATE how many shots it takes to kill somebody in Halo. They just don't get it. And I can explain to them how the shields work, how they make for longer, more dramatic gunfights, and let better players win despite lucky shots or surprise attacks from lesser players.
It doesn't matter. They've grown up on movies and other media that have taught them, BULLETS KILL QUICKLY. So they prefer games, where that expectation is immediately fulfilled.