Gaming Performance
Users expecting to use the current Vista beta as a gaming platform are going to be the most disappointed. The results of our gaming benchmarks largely speak for themselves.
The overall performance penalty on the current version of Vista depends largely on what the game is. Among the games tested, there doesn't seem to be any rhyme or reason as to why some games do worse than others. The only clear pattern is that Vista gaming performance is almost universally worse than under XP.
With antialiasing off, with the exception of Black and White 2, everything takes a moderate to significant performance hit. 3DMark, which we normally shy away from but tends to make a good diagnostic tool in situations like this, shows Vista x86 only trailing XP by a small margin, whereas the x64 version is doing significantly worse. However, in the games that would work under Vista x64 (HL2 would not work; its 64-bit binary would not accept our benchmark), Vista x64 is doing no worse than Vista x86.
If this means that Vista x64 is only suffering under a fraction of games or if this is an issue with 3DMark isn't something we can determine, but we're leery of the x64 version of Vista at this point anyhow, due to other reasons (more on that later). As for the slightly more compatible Vista x86, half of our tests have it coming in 25% slower under XP when antialiasing is off, which doesn't bode well for the fledgling operating system at this point. How much of this is early drivers, an early OS, or the result of still-active debugging code remains to be seen.
Antialiasing seems to be a significant sticking point for Vista right now, as our entire suite of test scores dive hard with it enabled on Vista versus XP. Half-Life 2 ended up facing the largest drop, more than 30% off from its numbers under XP, and while Doom 3 actually suffered less of a drop than without antialiasing, it's clear that there's something holding up our test bed when antialiasing is enabled. This unfortunately is a double whammy for gaming enthusiasts using top of the line cards like the 7900 and X1900 series, as this kind of slowdown affects them the most since they're the most capable of using antialiasing in the first place. It's hard to recommend gaming under the current Vista beta given that these performance drops are the largest for those who are most likely to be able to cope with a beta operating system and the high system requirements.
It's also worth noting that with our OpenGL title, Doom 3, we did not encounter any issues other than the normal performance slowdown. There had been some concern earlier about how using a full OpenGL client driver will force Vista to disable the desktop compositing engine since OpenGL takes full control of the GPU, and that GPU makers may switch to a slower OpenGL wrapper for Direct3D to keep the compositing engine working. While firing up Doom 3 did indeed disable the DCE, it's not a problem since Doom 3 is a full screen game that doesn't even normally allow Alt+Tabbing. OpenGL engines may not be as popular as they once were (largely in part to the slower adoption of the Doom 3 engine), but between the gaming market and the professional workstation market, both of which desire a full-performance OpenGL implementation, it doesn't seem like OpenGL is going to suffer on Vista as much as earlier feared. We may yet see some issues with certain professional applications, but many of the workstation users we know are still running Windows 2000 anyway.
Users expecting to use the current Vista beta as a gaming platform are going to be the most disappointed. The results of our gaming benchmarks largely speak for themselves.
| Gaming Performance (1280x1024) | |||
| XP | Vista x86 | Vista x64 | |
| 3DMark 2006 | 2749 | 2533 | 2088 |
| Doom 3 | 79.3 | 59.5 | 60.5 |
| Doom 3 4xAA | 58.8 | 47.7 | 49.3 |
| Half-Life 2 | 81.46 | 61.19 | N/A |
| Half-Life 2 4xAA | 76.25 | 49.73 | N/A |
| Black and White 2 | 22.3 | 23.3 | 23.6 |
| Black and White 2 4xAA | 19.5 | 15.3 | 15.3 |
The overall performance penalty on the current version of Vista depends largely on what the game is. Among the games tested, there doesn't seem to be any rhyme or reason as to why some games do worse than others. The only clear pattern is that Vista gaming performance is almost universally worse than under XP.
With antialiasing off, with the exception of Black and White 2, everything takes a moderate to significant performance hit. 3DMark, which we normally shy away from but tends to make a good diagnostic tool in situations like this, shows Vista x86 only trailing XP by a small margin, whereas the x64 version is doing significantly worse. However, in the games that would work under Vista x64 (HL2 would not work; its 64-bit binary would not accept our benchmark), Vista x64 is doing no worse than Vista x86.
If this means that Vista x64 is only suffering under a fraction of games or if this is an issue with 3DMark isn't something we can determine, but we're leery of the x64 version of Vista at this point anyhow, due to other reasons (more on that later). As for the slightly more compatible Vista x86, half of our tests have it coming in 25% slower under XP when antialiasing is off, which doesn't bode well for the fledgling operating system at this point. How much of this is early drivers, an early OS, or the result of still-active debugging code remains to be seen.
Antialiasing seems to be a significant sticking point for Vista right now, as our entire suite of test scores dive hard with it enabled on Vista versus XP. Half-Life 2 ended up facing the largest drop, more than 30% off from its numbers under XP, and while Doom 3 actually suffered less of a drop than without antialiasing, it's clear that there's something holding up our test bed when antialiasing is enabled. This unfortunately is a double whammy for gaming enthusiasts using top of the line cards like the 7900 and X1900 series, as this kind of slowdown affects them the most since they're the most capable of using antialiasing in the first place. It's hard to recommend gaming under the current Vista beta given that these performance drops are the largest for those who are most likely to be able to cope with a beta operating system and the high system requirements.
It's also worth noting that with our OpenGL title, Doom 3, we did not encounter any issues other than the normal performance slowdown. There had been some concern earlier about how using a full OpenGL client driver will force Vista to disable the desktop compositing engine since OpenGL takes full control of the GPU, and that GPU makers may switch to a slower OpenGL wrapper for Direct3D to keep the compositing engine working. While firing up Doom 3 did indeed disable the DCE, it's not a problem since Doom 3 is a full screen game that doesn't even normally allow Alt+Tabbing. OpenGL engines may not be as popular as they once were (largely in part to the slower adoption of the Doom 3 engine), but between the gaming market and the professional workstation market, both of which desire a full-performance OpenGL implementation, it doesn't seem like OpenGL is going to suffer on Vista as much as earlier feared. We may yet see some issues with certain professional applications, but many of the workstation users we know are still running Windows 2000 anyway.
Facebook
Twitter
RSS
75 Comments
View All Comments
dev0lution - Friday, June 16, 2006 - link
I'd happily boot into Vista everyday if all of my hardware devices would work. Not MS's fault, but rather my fault for buying a smaller manufacturer's product who has yet to post even beta drivers.In combination with Office 207 Beta2 and IE 7, Vista x86 has run fine and rather stable for me. It does tend to eat up a bit of memory, but I should probably add another GB anyhow. If I could just solve a couple app related problems and get Media Center (and MC remote) to change the channel on my set top box, I wouldn't be running from my MCE disk much at all anymore.
I kind of like the new layout and explorer...
RogueSpear - Friday, June 16, 2006 - link
I've had been using Vista on one of my computers until shortly after the Beta 2 was released for public consumption. Once I saw that there was no appreciable improvements in that release, I finally decided to revert back to the relative comfort and superior performance of XP. First off, I have nightmares when I think of the mass confusion that will ensue among the mass of computer neophytes that are just now getting over the adjustment from moving off of 98/ME to XP. These will be trying days for help desk staff and even those are the "computer guy" in their family.More importantly, changes that are allegedly substantial, seem to me more cosmetic than anything. Yes, I realize that there are a lot of serious changes under the hood, but the benefits you can see and touch appear very superficial at best. This seems like an extreme makeover in an attempt to get people signed up for even more pervasive and hideous DRM. I know I'm living in the past, but I'll always be nostalgic for the days when my computer was actually my computer and the software/media I paid for were mine to use as I saw fit.
Pirks - Friday, June 16, 2006 - link
I noticed this sentence: "As currently implemented, UAC surpasses Tiger's security features by giving more information about what application is requesting privilege escalation" Could you please elaborate a little on what "more information" exactly Vista provides in UAC dialogs that Tiger does NOT provide?From my experience Tiger gives the same information, I probably misunderstood you on that, could you please explain in more detail?
johnsonx - Friday, June 16, 2006 - link
Ok, these two sentences seem contradictory. First you say you don't know if 32-bit and 64-bit versions will come on the same disc with an installer that can pick correctly, then in the next sentence you say the installer will pick based on product key because both versions will use the same install media.
So which is it, or there there something I'm not getting?
Ryan Smith - Friday, June 16, 2006 - link
There's something you're not getting. A disc can install any variation of Vista(e.g. 1 disc can install Home Basic x86, Home Premium x86, Ultimate x86, etc); it can only install that bit-version of Vista however.DerekWilson - Friday, June 16, 2006 - link
it is difficult to say ... i think three different editors mucked around with that sentence :-)to try a different angle, both of these are true statements:
1) the x86 disk can install any x86 version of vista
2) the x64 disk can install any x64 version of vista
dhei - Friday, June 16, 2006 - link
When you can, do a test to see how well they redid it please. Someone told me this would be noticable on those with broadband easily, not just LAN or network tests. Im really intrested in this aspect, though not sure how to really test it.Did you try a LAN benchmark vs winxp to see if any diffrence?
Ryan Smith - Friday, June 16, 2006 - link
We did not do that, it was already a 12k word article + the time to run the benchmarks we did use. We'll be taking a much heavier look at performance once we have a final version of Vista to look at.Pirks - Friday, June 16, 2006 - link
and read this while you're at it:http://developer.apple.com/internet/security/secur...">http://developer.apple.com/internet/security/secur...
You can minimize the risk of a network service being used to attack your machine by using the firewall built into Mac OS X. Called ipfw, it can prevent potential attackers from reaching these services. As of Mac OS X 10.2, Apple has included a simple GUI for configuring ipfw. The GUI is good for adding simple rules to your machine; more complex rules will require you to use either the command line tools for manipulating the firewall, or a third-party GUI that has more features.
Ryan, do you know what BSD ipfw is? It blows any XP firewall to ashes, Vista is only pathetic attempt to get to its level (well hopefully MS will get something similar in Vista, I really hope they do)
Also read this: http://personalpages.tds.net/~brian_hill/brickhous...">http://personalpages.tds.net/~brian_hill/brickhous...
That's another GUI to configure ipfw in OSX.
Otherwise an excellent article, I'm impatiently wait for your review of the final Vista release, but please don't do such stupid mistakes again, Mac boys will hack and slash you for that ;-)
"it's time for a full featured firewall for Windows and Mac OS X alike, and only the former has it" - what a funny lie :-) Please read about OSX ipfw (I gave you a couple of links) and fix it ASAP. Thanks.
Ryan Smith - Friday, June 16, 2006 - link
I'm aware of IPFW, and what it can do(and boy is it nice!). But this is a competition among what the two OS's can do on their own, without significant intervention from the user. Out of the box, Vista's firewall is a full-featured firewall that can block inbound and outbound connections. Tiger's firewall can't do the latter, and in the age of spyware(and as you saw in our spyware test), it's sometimes the last thing keeping spyware and other malware from breaking out.Tiger may not have significant malware problems at this point, but there's no good reason why it(and more so Leopard) shouldn't have outbound protection too.