Composition Engine and Spyware Performance
One particularly important change with the move to a GPU accelerated desktop is relieving some of the enormous CPU performance penalties caused by GDI+. Because of how GDI+ handles window refreshes, heavy desktop activity that involves tasks such as moving around a window result in GDI+ eating up a great deal of CPU time just to handle these refreshes. By moving to a GPU accelerated desktop, the Window Composition Engine (WCE) that now handles these effects can offload some of the work to the GPU by treating these items as polygons and textures, which the GPU is well suited to manipulating.
In order to test the performance impact of the WCE, we set up a simple test in which we opened up several windows scattered around the desktop and in different states of overlapping each other, and then dragged around a window for 10 seconds measuring the system CPU usage. If the WCE is doing its job well, the CPU usage should be reduced. All 3 of Vista's desktop rendering modes have been tested using the exact same setup, and XP has been included using a setup as similar as possible. (We can't guarantee everything was 100% identical, as we are running on a different operating system with potentially different background tasks running.)
While the results against XP should be taken with a grain of salt due to the aforementioned setup issues, it's clear that in Aero mode the composition engine is doing its job and has pushed CPU usage down to 33% in spite of all the eye-candy this mode has over all other modes. For users who will be capable of using Aero mode, this will be a win-win situation for them as they can use all the advanced features of Aero and still need less CPU power in the process.
Vista Basic however is very distressing, and we're not particularly sure why it's doing so poorly. As we mentioned before, Basic is effectively just a new skin using the XP rendering mode, so why it's maxing out our CPU we're not sure at this point; the poor overall graphics performance of Vista shouldn't be affecting Basic this much considering it doesn't utilize 3D acceleration, nor does the debug code make for an adequate explanation for it. XP clearly does much better, and Microsoft needs to get Vista's performance more in line with XP's; otherwise those who want to use Vista on systems inadequate for Aero are going to be inadvertently giving themselves CPU-bound situations.
As for Vista Classic, since it's using a very similar rendering mode as Basic, the similarly poor score isn't surprising. Clearly disabling the semi-advanced features that give Basic its more refined look can bring CPU usage down, but 78% for the barebones graphical features of Windows 2000 is still too high. We'll definitely be taking another look at this when Vista is shipping.
Spyware Protection
While there's no formal method for testing the resilience of an operating system to spyware, one of the biggest pushes from Microsoft with Vista is that it will be much harder to infect with spyware, due to the combination of the new firewall, the UAC changes, and the integration of Windows Defender. To put that claim to the test, we attempted to infect our Vista setup with the Hotbar spyware package, a moderately annoying piece of malware that displays advertising and tries to phone home a record of user activities.
Going the direct route, we visited Hotbar's site in IE7+ and downloaded the Hotbar application directly from their site. Much to our surprise, Vista did not complain about this past the fact that we were running an executable we downloaded, something that Windows XP does just as well. Vista continued to sit idly by as we ran the installer for Hotbar, and we ultimately did not encounter any issues installing it.
It was not until we tried to remove it that we realized that Microsoft did not ship Vista with any spyware definitions, which is partially the reason that Windows was so passive about it being installed. In fact, until we installed those definitions, the only thing that kept Hotbar contained was the last failsafe, the firewall, which detected Hotbar attempting to connect to the internet and allowed us to block it before any further damage could be done. Once Microsoft's definitions were installed, we were able to remove Hotbar using Windows Defender without a problem. We then tried to install Hotbar again, at which point Defender notified us that we were trying to install a known piece of spyware and allowed us to abort the installation.
Given this test, we're not terribly convinced about Windows' anti-malware abilities at this time. In spite of UAC, Hotbar seemed perfectly happy running as a user-limited process, and it was only the firewall that kept it in check. Trashing a user account is for all practical purposes equally as destructive as trashing the entire system, so this is not a significant improvement.
It also puts Windows Defender in a bad light, as it appears that it will be of limited use in the case of dealing with a piece of malware it doesn't recognize. Certainly Defender will keep a subset of computer users from consistently reinstalling something that is spyware that they don't know about, but this may very well just lead to an arms race for spyware much like viruses today, which is not an effective situation. The firewall saved us, but that's not always going to be enough.
One particularly important change with the move to a GPU accelerated desktop is relieving some of the enormous CPU performance penalties caused by GDI+. Because of how GDI+ handles window refreshes, heavy desktop activity that involves tasks such as moving around a window result in GDI+ eating up a great deal of CPU time just to handle these refreshes. By moving to a GPU accelerated desktop, the Window Composition Engine (WCE) that now handles these effects can offload some of the work to the GPU by treating these items as polygons and textures, which the GPU is well suited to manipulating.
In order to test the performance impact of the WCE, we set up a simple test in which we opened up several windows scattered around the desktop and in different states of overlapping each other, and then dragged around a window for 10 seconds measuring the system CPU usage. If the WCE is doing its job well, the CPU usage should be reduced. All 3 of Vista's desktop rendering modes have been tested using the exact same setup, and XP has been included using a setup as similar as possible. (We can't guarantee everything was 100% identical, as we are running on a different operating system with potentially different background tasks running.)
 |
| Click to enlarge |
| Windows Composition Engine Performance | ||||
| XP | Vista Aero | Vista Basic | Vista Classic | |
| CPU usage | 49% | 33% | 97% | 78% |
While the results against XP should be taken with a grain of salt due to the aforementioned setup issues, it's clear that in Aero mode the composition engine is doing its job and has pushed CPU usage down to 33% in spite of all the eye-candy this mode has over all other modes. For users who will be capable of using Aero mode, this will be a win-win situation for them as they can use all the advanced features of Aero and still need less CPU power in the process.
Vista Basic however is very distressing, and we're not particularly sure why it's doing so poorly. As we mentioned before, Basic is effectively just a new skin using the XP rendering mode, so why it's maxing out our CPU we're not sure at this point; the poor overall graphics performance of Vista shouldn't be affecting Basic this much considering it doesn't utilize 3D acceleration, nor does the debug code make for an adequate explanation for it. XP clearly does much better, and Microsoft needs to get Vista's performance more in line with XP's; otherwise those who want to use Vista on systems inadequate for Aero are going to be inadvertently giving themselves CPU-bound situations.
As for Vista Classic, since it's using a very similar rendering mode as Basic, the similarly poor score isn't surprising. Clearly disabling the semi-advanced features that give Basic its more refined look can bring CPU usage down, but 78% for the barebones graphical features of Windows 2000 is still too high. We'll definitely be taking another look at this when Vista is shipping.
Spyware Protection
While there's no formal method for testing the resilience of an operating system to spyware, one of the biggest pushes from Microsoft with Vista is that it will be much harder to infect with spyware, due to the combination of the new firewall, the UAC changes, and the integration of Windows Defender. To put that claim to the test, we attempted to infect our Vista setup with the Hotbar spyware package, a moderately annoying piece of malware that displays advertising and tries to phone home a record of user activities.
Going the direct route, we visited Hotbar's site in IE7+ and downloaded the Hotbar application directly from their site. Much to our surprise, Vista did not complain about this past the fact that we were running an executable we downloaded, something that Windows XP does just as well. Vista continued to sit idly by as we ran the installer for Hotbar, and we ultimately did not encounter any issues installing it.
It was not until we tried to remove it that we realized that Microsoft did not ship Vista with any spyware definitions, which is partially the reason that Windows was so passive about it being installed. In fact, until we installed those definitions, the only thing that kept Hotbar contained was the last failsafe, the firewall, which detected Hotbar attempting to connect to the internet and allowed us to block it before any further damage could be done. Once Microsoft's definitions were installed, we were able to remove Hotbar using Windows Defender without a problem. We then tried to install Hotbar again, at which point Defender notified us that we were trying to install a known piece of spyware and allowed us to abort the installation.
Given this test, we're not terribly convinced about Windows' anti-malware abilities at this time. In spite of UAC, Hotbar seemed perfectly happy running as a user-limited process, and it was only the firewall that kept it in check. Trashing a user account is for all practical purposes equally as destructive as trashing the entire system, so this is not a significant improvement.
It also puts Windows Defender in a bad light, as it appears that it will be of limited use in the case of dealing with a piece of malware it doesn't recognize. Certainly Defender will keep a subset of computer users from consistently reinstalling something that is spyware that they don't know about, but this may very well just lead to an arms race for spyware much like viruses today, which is not an effective situation. The firewall saved us, but that's not always going to be enough.
Facebook
Twitter
RSS
75 Comments
View All Comments
dev0lution - Friday, June 16, 2006 - link
I'd happily boot into Vista everyday if all of my hardware devices would work. Not MS's fault, but rather my fault for buying a smaller manufacturer's product who has yet to post even beta drivers.In combination with Office 207 Beta2 and IE 7, Vista x86 has run fine and rather stable for me. It does tend to eat up a bit of memory, but I should probably add another GB anyhow. If I could just solve a couple app related problems and get Media Center (and MC remote) to change the channel on my set top box, I wouldn't be running from my MCE disk much at all anymore.
I kind of like the new layout and explorer...
RogueSpear - Friday, June 16, 2006 - link
I've had been using Vista on one of my computers until shortly after the Beta 2 was released for public consumption. Once I saw that there was no appreciable improvements in that release, I finally decided to revert back to the relative comfort and superior performance of XP. First off, I have nightmares when I think of the mass confusion that will ensue among the mass of computer neophytes that are just now getting over the adjustment from moving off of 98/ME to XP. These will be trying days for help desk staff and even those are the "computer guy" in their family.More importantly, changes that are allegedly substantial, seem to me more cosmetic than anything. Yes, I realize that there are a lot of serious changes under the hood, but the benefits you can see and touch appear very superficial at best. This seems like an extreme makeover in an attempt to get people signed up for even more pervasive and hideous DRM. I know I'm living in the past, but I'll always be nostalgic for the days when my computer was actually my computer and the software/media I paid for were mine to use as I saw fit.
Pirks - Friday, June 16, 2006 - link
I noticed this sentence: "As currently implemented, UAC surpasses Tiger's security features by giving more information about what application is requesting privilege escalation" Could you please elaborate a little on what "more information" exactly Vista provides in UAC dialogs that Tiger does NOT provide?From my experience Tiger gives the same information, I probably misunderstood you on that, could you please explain in more detail?
johnsonx - Friday, June 16, 2006 - link
Ok, these two sentences seem contradictory. First you say you don't know if 32-bit and 64-bit versions will come on the same disc with an installer that can pick correctly, then in the next sentence you say the installer will pick based on product key because both versions will use the same install media.
So which is it, or there there something I'm not getting?
Ryan Smith - Friday, June 16, 2006 - link
There's something you're not getting. A disc can install any variation of Vista(e.g. 1 disc can install Home Basic x86, Home Premium x86, Ultimate x86, etc); it can only install that bit-version of Vista however.DerekWilson - Friday, June 16, 2006 - link
it is difficult to say ... i think three different editors mucked around with that sentence :-)to try a different angle, both of these are true statements:
1) the x86 disk can install any x86 version of vista
2) the x64 disk can install any x64 version of vista
dhei - Friday, June 16, 2006 - link
When you can, do a test to see how well they redid it please. Someone told me this would be noticable on those with broadband easily, not just LAN or network tests. Im really intrested in this aspect, though not sure how to really test it.Did you try a LAN benchmark vs winxp to see if any diffrence?
Ryan Smith - Friday, June 16, 2006 - link
We did not do that, it was already a 12k word article + the time to run the benchmarks we did use. We'll be taking a much heavier look at performance once we have a final version of Vista to look at.Pirks - Friday, June 16, 2006 - link
and read this while you're at it:http://developer.apple.com/internet/security/secur...">http://developer.apple.com/internet/security/secur...
You can minimize the risk of a network service being used to attack your machine by using the firewall built into Mac OS X. Called ipfw, it can prevent potential attackers from reaching these services. As of Mac OS X 10.2, Apple has included a simple GUI for configuring ipfw. The GUI is good for adding simple rules to your machine; more complex rules will require you to use either the command line tools for manipulating the firewall, or a third-party GUI that has more features.
Ryan, do you know what BSD ipfw is? It blows any XP firewall to ashes, Vista is only pathetic attempt to get to its level (well hopefully MS will get something similar in Vista, I really hope they do)
Also read this: http://personalpages.tds.net/~brian_hill/brickhous...">http://personalpages.tds.net/~brian_hill/brickhous...
That's another GUI to configure ipfw in OSX.
Otherwise an excellent article, I'm impatiently wait for your review of the final Vista release, but please don't do such stupid mistakes again, Mac boys will hack and slash you for that ;-)
"it's time for a full featured firewall for Windows and Mac OS X alike, and only the former has it" - what a funny lie :-) Please read about OSX ipfw (I gave you a couple of links) and fix it ASAP. Thanks.
Ryan Smith - Friday, June 16, 2006 - link
I'm aware of IPFW, and what it can do(and boy is it nice!). But this is a competition among what the two OS's can do on their own, without significant intervention from the user. Out of the box, Vista's firewall is a full-featured firewall that can block inbound and outbound connections. Tiger's firewall can't do the latter, and in the age of spyware(and as you saw in our spyware test), it's sometimes the last thing keeping spyware and other malware from breaking out.Tiger may not have significant malware problems at this point, but there's no good reason why it(and more so Leopard) shouldn't have outbound protection too.