Composition Engine and Spyware Performance
One particularly important change with the move to a GPU accelerated desktop is relieving some of the enormous CPU performance penalties caused by GDI+. Because of how GDI+ handles window refreshes, heavy desktop activity that involves tasks such as moving around a window result in GDI+ eating up a great deal of CPU time just to handle these refreshes. By moving to a GPU accelerated desktop, the Window Composition Engine (WCE) that now handles these effects can offload some of the work to the GPU by treating these items as polygons and textures, which the GPU is well suited to manipulating.
In order to test the performance impact of the WCE, we set up a simple test in which we opened up several windows scattered around the desktop and in different states of overlapping each other, and then dragged around a window for 10 seconds measuring the system CPU usage. If the WCE is doing its job well, the CPU usage should be reduced. All 3 of Vista's desktop rendering modes have been tested using the exact same setup, and XP has been included using a setup as similar as possible. (We can't guarantee everything was 100% identical, as we are running on a different operating system with potentially different background tasks running.)
While the results against XP should be taken with a grain of salt due to the aforementioned setup issues, it's clear that in Aero mode the composition engine is doing its job and has pushed CPU usage down to 33% in spite of all the eye-candy this mode has over all other modes. For users who will be capable of using Aero mode, this will be a win-win situation for them as they can use all the advanced features of Aero and still need less CPU power in the process.
Vista Basic however is very distressing, and we're not particularly sure why it's doing so poorly. As we mentioned before, Basic is effectively just a new skin using the XP rendering mode, so why it's maxing out our CPU we're not sure at this point; the poor overall graphics performance of Vista shouldn't be affecting Basic this much considering it doesn't utilize 3D acceleration, nor does the debug code make for an adequate explanation for it. XP clearly does much better, and Microsoft needs to get Vista's performance more in line with XP's; otherwise those who want to use Vista on systems inadequate for Aero are going to be inadvertently giving themselves CPU-bound situations.
As for Vista Classic, since it's using a very similar rendering mode as Basic, the similarly poor score isn't surprising. Clearly disabling the semi-advanced features that give Basic its more refined look can bring CPU usage down, but 78% for the barebones graphical features of Windows 2000 is still too high. We'll definitely be taking another look at this when Vista is shipping.
Spyware Protection
While there's no formal method for testing the resilience of an operating system to spyware, one of the biggest pushes from Microsoft with Vista is that it will be much harder to infect with spyware, due to the combination of the new firewall, the UAC changes, and the integration of Windows Defender. To put that claim to the test, we attempted to infect our Vista setup with the Hotbar spyware package, a moderately annoying piece of malware that displays advertising and tries to phone home a record of user activities.
Going the direct route, we visited Hotbar's site in IE7+ and downloaded the Hotbar application directly from their site. Much to our surprise, Vista did not complain about this past the fact that we were running an executable we downloaded, something that Windows XP does just as well. Vista continued to sit idly by as we ran the installer for Hotbar, and we ultimately did not encounter any issues installing it.
It was not until we tried to remove it that we realized that Microsoft did not ship Vista with any spyware definitions, which is partially the reason that Windows was so passive about it being installed. In fact, until we installed those definitions, the only thing that kept Hotbar contained was the last failsafe, the firewall, which detected Hotbar attempting to connect to the internet and allowed us to block it before any further damage could be done. Once Microsoft's definitions were installed, we were able to remove Hotbar using Windows Defender without a problem. We then tried to install Hotbar again, at which point Defender notified us that we were trying to install a known piece of spyware and allowed us to abort the installation.
Given this test, we're not terribly convinced about Windows' anti-malware abilities at this time. In spite of UAC, Hotbar seemed perfectly happy running as a user-limited process, and it was only the firewall that kept it in check. Trashing a user account is for all practical purposes equally as destructive as trashing the entire system, so this is not a significant improvement.
It also puts Windows Defender in a bad light, as it appears that it will be of limited use in the case of dealing with a piece of malware it doesn't recognize. Certainly Defender will keep a subset of computer users from consistently reinstalling something that is spyware that they don't know about, but this may very well just lead to an arms race for spyware much like viruses today, which is not an effective situation. The firewall saved us, but that's not always going to be enough.
One particularly important change with the move to a GPU accelerated desktop is relieving some of the enormous CPU performance penalties caused by GDI+. Because of how GDI+ handles window refreshes, heavy desktop activity that involves tasks such as moving around a window result in GDI+ eating up a great deal of CPU time just to handle these refreshes. By moving to a GPU accelerated desktop, the Window Composition Engine (WCE) that now handles these effects can offload some of the work to the GPU by treating these items as polygons and textures, which the GPU is well suited to manipulating.
In order to test the performance impact of the WCE, we set up a simple test in which we opened up several windows scattered around the desktop and in different states of overlapping each other, and then dragged around a window for 10 seconds measuring the system CPU usage. If the WCE is doing its job well, the CPU usage should be reduced. All 3 of Vista's desktop rendering modes have been tested using the exact same setup, and XP has been included using a setup as similar as possible. (We can't guarantee everything was 100% identical, as we are running on a different operating system with potentially different background tasks running.)
 |
| Click to enlarge |
| Windows Composition Engine Performance | ||||
| XP | Vista Aero | Vista Basic | Vista Classic | |
| CPU usage | 49% | 33% | 97% | 78% |
While the results against XP should be taken with a grain of salt due to the aforementioned setup issues, it's clear that in Aero mode the composition engine is doing its job and has pushed CPU usage down to 33% in spite of all the eye-candy this mode has over all other modes. For users who will be capable of using Aero mode, this will be a win-win situation for them as they can use all the advanced features of Aero and still need less CPU power in the process.
Vista Basic however is very distressing, and we're not particularly sure why it's doing so poorly. As we mentioned before, Basic is effectively just a new skin using the XP rendering mode, so why it's maxing out our CPU we're not sure at this point; the poor overall graphics performance of Vista shouldn't be affecting Basic this much considering it doesn't utilize 3D acceleration, nor does the debug code make for an adequate explanation for it. XP clearly does much better, and Microsoft needs to get Vista's performance more in line with XP's; otherwise those who want to use Vista on systems inadequate for Aero are going to be inadvertently giving themselves CPU-bound situations.
As for Vista Classic, since it's using a very similar rendering mode as Basic, the similarly poor score isn't surprising. Clearly disabling the semi-advanced features that give Basic its more refined look can bring CPU usage down, but 78% for the barebones graphical features of Windows 2000 is still too high. We'll definitely be taking another look at this when Vista is shipping.
Spyware Protection
While there's no formal method for testing the resilience of an operating system to spyware, one of the biggest pushes from Microsoft with Vista is that it will be much harder to infect with spyware, due to the combination of the new firewall, the UAC changes, and the integration of Windows Defender. To put that claim to the test, we attempted to infect our Vista setup with the Hotbar spyware package, a moderately annoying piece of malware that displays advertising and tries to phone home a record of user activities.
Going the direct route, we visited Hotbar's site in IE7+ and downloaded the Hotbar application directly from their site. Much to our surprise, Vista did not complain about this past the fact that we were running an executable we downloaded, something that Windows XP does just as well. Vista continued to sit idly by as we ran the installer for Hotbar, and we ultimately did not encounter any issues installing it.
It was not until we tried to remove it that we realized that Microsoft did not ship Vista with any spyware definitions, which is partially the reason that Windows was so passive about it being installed. In fact, until we installed those definitions, the only thing that kept Hotbar contained was the last failsafe, the firewall, which detected Hotbar attempting to connect to the internet and allowed us to block it before any further damage could be done. Once Microsoft's definitions were installed, we were able to remove Hotbar using Windows Defender without a problem. We then tried to install Hotbar again, at which point Defender notified us that we were trying to install a known piece of spyware and allowed us to abort the installation.
Given this test, we're not terribly convinced about Windows' anti-malware abilities at this time. In spite of UAC, Hotbar seemed perfectly happy running as a user-limited process, and it was only the firewall that kept it in check. Trashing a user account is for all practical purposes equally as destructive as trashing the entire system, so this is not a significant improvement.
It also puts Windows Defender in a bad light, as it appears that it will be of limited use in the case of dealing with a piece of malware it doesn't recognize. Certainly Defender will keep a subset of computer users from consistently reinstalling something that is spyware that they don't know about, but this may very well just lead to an arms race for spyware much like viruses today, which is not an effective situation. The firewall saved us, but that's not always going to be enough.
Facebook
Twitter
RSS
75 Comments
View All Comments
Pirks - Friday, June 16, 2006 - link
Excuse me, what? How about this then: http://www.macworld.com/2006/05/reviews/osxfirewal...">http://www.macworld.com/2006/05/reviews/osxfirewal..."The emphasis is on incoming. As it ships from Apple, the firewall does not monitor traffic that may be originating from your own computer. If your Mac gets possessed by a malware application that then attempts to attack or infect other computers via your Internet connection (a not-uncommon trick), OS X’s firewall won’t, by default, pay any attention. And, there’s no way to change this default setting from your System Preferences. To force the firewall to monitor outbound traffic, you must use Terminal’s command-line interface."
See - IT CAN monitor and block outbound traffic, contrary to what you say. It's just a matter of configuring it properly. You should at least correct your article and stop saying OSX ipfw CAN'T track outbound connections. You can say this: it's SET UP not to monitor outbound connections BY DEFAULT but anyone can CONFIGURE it to monitor outbound connections either through third party GUI like Flying Buttress or via command line. Then you won't look like a liar to any Mac guy who cares to read your review.
Ryan Smith - Friday, June 16, 2006 - link
I see your point, but I believe there's nothing in the article that needs changing. Tiger's firewall can't block outbound connections without having to drop to the terminal to muck with IPFW, I do not classify that as an ability any more than I classify Vista x64 as being amateur driver programmer friendly(since you need to drop to the terminal to turn off the x64 integrity check). When a version of Mac OS X ships with a proper GUI for controlling outbound firewalling(as is the Apple way), then it will be capable by a reasonable definition. Right now it's nothing more than a quirk that results from using the BSD base.Pirks - Friday, June 16, 2006 - link
Excellent point! So, when (and if) Mac OS X will see its share of virii and malware, THEN Apple will incorporate outbound connection settings in OS X GUI - right now it's not needed by Mac users, and the rare exceptions are easily treated with third party apps and command line.OK, got your point, agreed, issue closed. Thanks :)
bjtags - Friday, June 16, 2006 - link
Vista x64I have been pounding on it for 4 days never crash or even farted once!!!
Have all HalfLife 2 and CS running Just Great!!!
Had at one time 10 IE windows open, MediaPlayer, Steam updating, download driver,
updating windows drivers, and 3 folder explorer windows open, and tranfering
4gig movie to HD!!!
Still ran fine... I do have AMD 4800 x2 with 2gigs...
Poser - Friday, June 16, 2006 - link
Two questions:1. What's the ship date for Vista supposed to be? Q4 of 2006?
2. I seem to remember that speech recognition would be included and integrated with Vista. Is it considered too much of a niche toy to even mention, not considered to be part of the OS, or am I just plain wrong about it's inclusion?
It was a extremely well written article. Very nice job.
Ryan Smith - Friday, June 16, 2006 - link
1. Expected completion is Q4 with some business customers getting access to the final version at that time. It won't be released to the public until 2007 however.2. You're right, speech recognition is included. You're also right in that given the amount of stuff we had to cover in one article it was too much of a niche; voice recognition so far is still too immature to replace typing.
ashay - Friday, June 16, 2006 - link
"Dogfooding" is when a company uses their own new product (not necessarily beta) for internal use.(maybe even in critical production systems).Term comes from "eat your own dog-food". Meaning if you're a dog food maker, the CEO and execs eat the stuff. If they like it they dogs hopefully will.
http://en.wikipedia.org/wiki/Eat_one%27s_own_dog_f...">Wikipedia link
fishbits - Friday, June 16, 2006 - link
Yes, I know it's still beta, we'll see. The UAC and signed drivers schemes sound like they'll be flops right out of the gate. Average user will quickly realize he can't install or use anything until he adopts a "just click 'Yes'" attitude, which will reward him with a functioning device/running program. I've lost count of how many drivers I've installed under XP that were for name-brand devices, yet didn't have the official seal of approval on them. Again, get trained to "just click 'Yes'" in order to be able to do anything useful. Without better information given to the user at this decision point, all the scheme does is add a few mouse-clicks and no security. Like when you install a program and your security suite gives a "helpful" warning like "INeedToRun.exe is trying to access feccflm.dll ... no recommendation."As expected, it looks like the productivity gains of GPU-acceleration were immediately swallowed up by GUI overhead. Whee! "The users can solve this through future hardware upgrades." Gotcha. For what it's worth, the gadgets/widgets look needlessly large and ugly, especially for simply displaying things like time, cpu temp/usage. Then it sounds like we're going to have resource-hungry programs getting starved because of GPU sharing, or will have an arms-race of workarounds to get their hands on the power they think they need.
Ah well, I've got to move to 64-bit for RAM purposes relatively soon. Think I'll wait a year or two after Vista 64 to let it get stable, faster, and better supported. Then hopefully the programs I'll need to upgrade can be purchased along the lines of a normal upgrade cycle. Games I'm actually not as worried about, as I expect XP/DX9 support to continue for a decent bit and will retain an XP box and install Vista on a brand new one when the time comes.
shamgar03 - Friday, June 16, 2006 - link
I really hope that will mean for BETTER GPU performance not worse. I would really just like to be able to boot into a game only environment where you have something like a grub interface to pick games and it only loads the needed stuff for the game.darkdemyze - Friday, June 16, 2006 - link
beta implies "still in developement". chances are very high performance will see an increase by the time of release. I agree with your seconds statement though.