Gaming Performance
Users expecting to use the current Vista beta as a gaming platform are going to be the most disappointed. The results of our gaming benchmarks largely speak for themselves.
The overall performance penalty on the current version of Vista depends largely on what the game is. Among the games tested, there doesn't seem to be any rhyme or reason as to why some games do worse than others. The only clear pattern is that Vista gaming performance is almost universally worse than under XP.
With antialiasing off, with the exception of Black and White 2, everything takes a moderate to significant performance hit. 3DMark, which we normally shy away from but tends to make a good diagnostic tool in situations like this, shows Vista x86 only trailing XP by a small margin, whereas the x64 version is doing significantly worse. However, in the games that would work under Vista x64 (HL2 would not work; its 64-bit binary would not accept our benchmark), Vista x64 is doing no worse than Vista x86.
If this means that Vista x64 is only suffering under a fraction of games or if this is an issue with 3DMark isn't something we can determine, but we're leery of the x64 version of Vista at this point anyhow, due to other reasons (more on that later). As for the slightly more compatible Vista x86, half of our tests have it coming in 25% slower under XP when antialiasing is off, which doesn't bode well for the fledgling operating system at this point. How much of this is early drivers, an early OS, or the result of still-active debugging code remains to be seen.
Antialiasing seems to be a significant sticking point for Vista right now, as our entire suite of test scores dive hard with it enabled on Vista versus XP. Half-Life 2 ended up facing the largest drop, more than 30% off from its numbers under XP, and while Doom 3 actually suffered less of a drop than without antialiasing, it's clear that there's something holding up our test bed when antialiasing is enabled. This unfortunately is a double whammy for gaming enthusiasts using top of the line cards like the 7900 and X1900 series, as this kind of slowdown affects them the most since they're the most capable of using antialiasing in the first place. It's hard to recommend gaming under the current Vista beta given that these performance drops are the largest for those who are most likely to be able to cope with a beta operating system and the high system requirements.
It's also worth noting that with our OpenGL title, Doom 3, we did not encounter any issues other than the normal performance slowdown. There had been some concern earlier about how using a full OpenGL client driver will force Vista to disable the desktop compositing engine since OpenGL takes full control of the GPU, and that GPU makers may switch to a slower OpenGL wrapper for Direct3D to keep the compositing engine working. While firing up Doom 3 did indeed disable the DCE, it's not a problem since Doom 3 is a full screen game that doesn't even normally allow Alt+Tabbing. OpenGL engines may not be as popular as they once were (largely in part to the slower adoption of the Doom 3 engine), but between the gaming market and the professional workstation market, both of which desire a full-performance OpenGL implementation, it doesn't seem like OpenGL is going to suffer on Vista as much as earlier feared. We may yet see some issues with certain professional applications, but many of the workstation users we know are still running Windows 2000 anyway.
Users expecting to use the current Vista beta as a gaming platform are going to be the most disappointed. The results of our gaming benchmarks largely speak for themselves.
| Gaming Performance (1280x1024) | |||
| XP | Vista x86 | Vista x64 | |
| 3DMark 2006 | 2749 | 2533 | 2088 |
| Doom 3 | 79.3 | 59.5 | 60.5 |
| Doom 3 4xAA | 58.8 | 47.7 | 49.3 |
| Half-Life 2 | 81.46 | 61.19 | N/A |
| Half-Life 2 4xAA | 76.25 | 49.73 | N/A |
| Black and White 2 | 22.3 | 23.3 | 23.6 |
| Black and White 2 4xAA | 19.5 | 15.3 | 15.3 |
The overall performance penalty on the current version of Vista depends largely on what the game is. Among the games tested, there doesn't seem to be any rhyme or reason as to why some games do worse than others. The only clear pattern is that Vista gaming performance is almost universally worse than under XP.
With antialiasing off, with the exception of Black and White 2, everything takes a moderate to significant performance hit. 3DMark, which we normally shy away from but tends to make a good diagnostic tool in situations like this, shows Vista x86 only trailing XP by a small margin, whereas the x64 version is doing significantly worse. However, in the games that would work under Vista x64 (HL2 would not work; its 64-bit binary would not accept our benchmark), Vista x64 is doing no worse than Vista x86.
If this means that Vista x64 is only suffering under a fraction of games or if this is an issue with 3DMark isn't something we can determine, but we're leery of the x64 version of Vista at this point anyhow, due to other reasons (more on that later). As for the slightly more compatible Vista x86, half of our tests have it coming in 25% slower under XP when antialiasing is off, which doesn't bode well for the fledgling operating system at this point. How much of this is early drivers, an early OS, or the result of still-active debugging code remains to be seen.
Antialiasing seems to be a significant sticking point for Vista right now, as our entire suite of test scores dive hard with it enabled on Vista versus XP. Half-Life 2 ended up facing the largest drop, more than 30% off from its numbers under XP, and while Doom 3 actually suffered less of a drop than without antialiasing, it's clear that there's something holding up our test bed when antialiasing is enabled. This unfortunately is a double whammy for gaming enthusiasts using top of the line cards like the 7900 and X1900 series, as this kind of slowdown affects them the most since they're the most capable of using antialiasing in the first place. It's hard to recommend gaming under the current Vista beta given that these performance drops are the largest for those who are most likely to be able to cope with a beta operating system and the high system requirements.
It's also worth noting that with our OpenGL title, Doom 3, we did not encounter any issues other than the normal performance slowdown. There had been some concern earlier about how using a full OpenGL client driver will force Vista to disable the desktop compositing engine since OpenGL takes full control of the GPU, and that GPU makers may switch to a slower OpenGL wrapper for Direct3D to keep the compositing engine working. While firing up Doom 3 did indeed disable the DCE, it's not a problem since Doom 3 is a full screen game that doesn't even normally allow Alt+Tabbing. OpenGL engines may not be as popular as they once were (largely in part to the slower adoption of the Doom 3 engine), but between the gaming market and the professional workstation market, both of which desire a full-performance OpenGL implementation, it doesn't seem like OpenGL is going to suffer on Vista as much as earlier feared. We may yet see some issues with certain professional applications, but many of the workstation users we know are still running Windows 2000 anyway.
Facebook
Twitter
RSS
75 Comments
View All Comments
Pirks - Friday, June 16, 2006 - link
Excuse me, what? How about this then: http://www.macworld.com/2006/05/reviews/osxfirewal...">http://www.macworld.com/2006/05/reviews/osxfirewal..."The emphasis is on incoming. As it ships from Apple, the firewall does not monitor traffic that may be originating from your own computer. If your Mac gets possessed by a malware application that then attempts to attack or infect other computers via your Internet connection (a not-uncommon trick), OS X’s firewall won’t, by default, pay any attention. And, there’s no way to change this default setting from your System Preferences. To force the firewall to monitor outbound traffic, you must use Terminal’s command-line interface."
See - IT CAN monitor and block outbound traffic, contrary to what you say. It's just a matter of configuring it properly. You should at least correct your article and stop saying OSX ipfw CAN'T track outbound connections. You can say this: it's SET UP not to monitor outbound connections BY DEFAULT but anyone can CONFIGURE it to monitor outbound connections either through third party GUI like Flying Buttress or via command line. Then you won't look like a liar to any Mac guy who cares to read your review.
Ryan Smith - Friday, June 16, 2006 - link
I see your point, but I believe there's nothing in the article that needs changing. Tiger's firewall can't block outbound connections without having to drop to the terminal to muck with IPFW, I do not classify that as an ability any more than I classify Vista x64 as being amateur driver programmer friendly(since you need to drop to the terminal to turn off the x64 integrity check). When a version of Mac OS X ships with a proper GUI for controlling outbound firewalling(as is the Apple way), then it will be capable by a reasonable definition. Right now it's nothing more than a quirk that results from using the BSD base.Pirks - Friday, June 16, 2006 - link
Excellent point! So, when (and if) Mac OS X will see its share of virii and malware, THEN Apple will incorporate outbound connection settings in OS X GUI - right now it's not needed by Mac users, and the rare exceptions are easily treated with third party apps and command line.OK, got your point, agreed, issue closed. Thanks :)
bjtags - Friday, June 16, 2006 - link
Vista x64I have been pounding on it for 4 days never crash or even farted once!!!
Have all HalfLife 2 and CS running Just Great!!!
Had at one time 10 IE windows open, MediaPlayer, Steam updating, download driver,
updating windows drivers, and 3 folder explorer windows open, and tranfering
4gig movie to HD!!!
Still ran fine... I do have AMD 4800 x2 with 2gigs...
Poser - Friday, June 16, 2006 - link
Two questions:1. What's the ship date for Vista supposed to be? Q4 of 2006?
2. I seem to remember that speech recognition would be included and integrated with Vista. Is it considered too much of a niche toy to even mention, not considered to be part of the OS, or am I just plain wrong about it's inclusion?
It was a extremely well written article. Very nice job.
Ryan Smith - Friday, June 16, 2006 - link
1. Expected completion is Q4 with some business customers getting access to the final version at that time. It won't be released to the public until 2007 however.2. You're right, speech recognition is included. You're also right in that given the amount of stuff we had to cover in one article it was too much of a niche; voice recognition so far is still too immature to replace typing.
ashay - Friday, June 16, 2006 - link
"Dogfooding" is when a company uses their own new product (not necessarily beta) for internal use.(maybe even in critical production systems).Term comes from "eat your own dog-food". Meaning if you're a dog food maker, the CEO and execs eat the stuff. If they like it they dogs hopefully will.
http://en.wikipedia.org/wiki/Eat_one%27s_own_dog_f...">Wikipedia link
fishbits - Friday, June 16, 2006 - link
Yes, I know it's still beta, we'll see. The UAC and signed drivers schemes sound like they'll be flops right out of the gate. Average user will quickly realize he can't install or use anything until he adopts a "just click 'Yes'" attitude, which will reward him with a functioning device/running program. I've lost count of how many drivers I've installed under XP that were for name-brand devices, yet didn't have the official seal of approval on them. Again, get trained to "just click 'Yes'" in order to be able to do anything useful. Without better information given to the user at this decision point, all the scheme does is add a few mouse-clicks and no security. Like when you install a program and your security suite gives a "helpful" warning like "INeedToRun.exe is trying to access feccflm.dll ... no recommendation."As expected, it looks like the productivity gains of GPU-acceleration were immediately swallowed up by GUI overhead. Whee! "The users can solve this through future hardware upgrades." Gotcha. For what it's worth, the gadgets/widgets look needlessly large and ugly, especially for simply displaying things like time, cpu temp/usage. Then it sounds like we're going to have resource-hungry programs getting starved because of GPU sharing, or will have an arms-race of workarounds to get their hands on the power they think they need.
Ah well, I've got to move to 64-bit for RAM purposes relatively soon. Think I'll wait a year or two after Vista 64 to let it get stable, faster, and better supported. Then hopefully the programs I'll need to upgrade can be purchased along the lines of a normal upgrade cycle. Games I'm actually not as worried about, as I expect XP/DX9 support to continue for a decent bit and will retain an XP box and install Vista on a brand new one when the time comes.
shamgar03 - Friday, June 16, 2006 - link
I really hope that will mean for BETTER GPU performance not worse. I would really just like to be able to boot into a game only environment where you have something like a grub interface to pick games and it only loads the needed stuff for the game.darkdemyze - Friday, June 16, 2006 - link
beta implies "still in developement". chances are very high performance will see an increase by the time of release. I agree with your seconds statement though.