Composition Engine and Spyware Performance
One particularly important change with the move to a GPU accelerated desktop is relieving some of the enormous CPU performance penalties caused by GDI+. Because of how GDI+ handles window refreshes, heavy desktop activity that involves tasks such as moving around a window result in GDI+ eating up a great deal of CPU time just to handle these refreshes. By moving to a GPU accelerated desktop, the Window Composition Engine (WCE) that now handles these effects can offload some of the work to the GPU by treating these items as polygons and textures, which the GPU is well suited to manipulating.
In order to test the performance impact of the WCE, we set up a simple test in which we opened up several windows scattered around the desktop and in different states of overlapping each other, and then dragged around a window for 10 seconds measuring the system CPU usage. If the WCE is doing its job well, the CPU usage should be reduced. All 3 of Vista's desktop rendering modes have been tested using the exact same setup, and XP has been included using a setup as similar as possible. (We can't guarantee everything was 100% identical, as we are running on a different operating system with potentially different background tasks running.)
While the results against XP should be taken with a grain of salt due to the aforementioned setup issues, it's clear that in Aero mode the composition engine is doing its job and has pushed CPU usage down to 33% in spite of all the eye-candy this mode has over all other modes. For users who will be capable of using Aero mode, this will be a win-win situation for them as they can use all the advanced features of Aero and still need less CPU power in the process.
Vista Basic however is very distressing, and we're not particularly sure why it's doing so poorly. As we mentioned before, Basic is effectively just a new skin using the XP rendering mode, so why it's maxing out our CPU we're not sure at this point; the poor overall graphics performance of Vista shouldn't be affecting Basic this much considering it doesn't utilize 3D acceleration, nor does the debug code make for an adequate explanation for it. XP clearly does much better, and Microsoft needs to get Vista's performance more in line with XP's; otherwise those who want to use Vista on systems inadequate for Aero are going to be inadvertently giving themselves CPU-bound situations.
As for Vista Classic, since it's using a very similar rendering mode as Basic, the similarly poor score isn't surprising. Clearly disabling the semi-advanced features that give Basic its more refined look can bring CPU usage down, but 78% for the barebones graphical features of Windows 2000 is still too high. We'll definitely be taking another look at this when Vista is shipping.
Spyware Protection
While there's no formal method for testing the resilience of an operating system to spyware, one of the biggest pushes from Microsoft with Vista is that it will be much harder to infect with spyware, due to the combination of the new firewall, the UAC changes, and the integration of Windows Defender. To put that claim to the test, we attempted to infect our Vista setup with the Hotbar spyware package, a moderately annoying piece of malware that displays advertising and tries to phone home a record of user activities.
Going the direct route, we visited Hotbar's site in IE7+ and downloaded the Hotbar application directly from their site. Much to our surprise, Vista did not complain about this past the fact that we were running an executable we downloaded, something that Windows XP does just as well. Vista continued to sit idly by as we ran the installer for Hotbar, and we ultimately did not encounter any issues installing it.
It was not until we tried to remove it that we realized that Microsoft did not ship Vista with any spyware definitions, which is partially the reason that Windows was so passive about it being installed. In fact, until we installed those definitions, the only thing that kept Hotbar contained was the last failsafe, the firewall, which detected Hotbar attempting to connect to the internet and allowed us to block it before any further damage could be done. Once Microsoft's definitions were installed, we were able to remove Hotbar using Windows Defender without a problem. We then tried to install Hotbar again, at which point Defender notified us that we were trying to install a known piece of spyware and allowed us to abort the installation.
Given this test, we're not terribly convinced about Windows' anti-malware abilities at this time. In spite of UAC, Hotbar seemed perfectly happy running as a user-limited process, and it was only the firewall that kept it in check. Trashing a user account is for all practical purposes equally as destructive as trashing the entire system, so this is not a significant improvement.
It also puts Windows Defender in a bad light, as it appears that it will be of limited use in the case of dealing with a piece of malware it doesn't recognize. Certainly Defender will keep a subset of computer users from consistently reinstalling something that is spyware that they don't know about, but this may very well just lead to an arms race for spyware much like viruses today, which is not an effective situation. The firewall saved us, but that's not always going to be enough.
One particularly important change with the move to a GPU accelerated desktop is relieving some of the enormous CPU performance penalties caused by GDI+. Because of how GDI+ handles window refreshes, heavy desktop activity that involves tasks such as moving around a window result in GDI+ eating up a great deal of CPU time just to handle these refreshes. By moving to a GPU accelerated desktop, the Window Composition Engine (WCE) that now handles these effects can offload some of the work to the GPU by treating these items as polygons and textures, which the GPU is well suited to manipulating.
In order to test the performance impact of the WCE, we set up a simple test in which we opened up several windows scattered around the desktop and in different states of overlapping each other, and then dragged around a window for 10 seconds measuring the system CPU usage. If the WCE is doing its job well, the CPU usage should be reduced. All 3 of Vista's desktop rendering modes have been tested using the exact same setup, and XP has been included using a setup as similar as possible. (We can't guarantee everything was 100% identical, as we are running on a different operating system with potentially different background tasks running.)
 |
| Click to enlarge |
| Windows Composition Engine Performance | ||||
| XP | Vista Aero | Vista Basic | Vista Classic | |
| CPU usage | 49% | 33% | 97% | 78% |
While the results against XP should be taken with a grain of salt due to the aforementioned setup issues, it's clear that in Aero mode the composition engine is doing its job and has pushed CPU usage down to 33% in spite of all the eye-candy this mode has over all other modes. For users who will be capable of using Aero mode, this will be a win-win situation for them as they can use all the advanced features of Aero and still need less CPU power in the process.
Vista Basic however is very distressing, and we're not particularly sure why it's doing so poorly. As we mentioned before, Basic is effectively just a new skin using the XP rendering mode, so why it's maxing out our CPU we're not sure at this point; the poor overall graphics performance of Vista shouldn't be affecting Basic this much considering it doesn't utilize 3D acceleration, nor does the debug code make for an adequate explanation for it. XP clearly does much better, and Microsoft needs to get Vista's performance more in line with XP's; otherwise those who want to use Vista on systems inadequate for Aero are going to be inadvertently giving themselves CPU-bound situations.
As for Vista Classic, since it's using a very similar rendering mode as Basic, the similarly poor score isn't surprising. Clearly disabling the semi-advanced features that give Basic its more refined look can bring CPU usage down, but 78% for the barebones graphical features of Windows 2000 is still too high. We'll definitely be taking another look at this when Vista is shipping.
Spyware Protection
While there's no formal method for testing the resilience of an operating system to spyware, one of the biggest pushes from Microsoft with Vista is that it will be much harder to infect with spyware, due to the combination of the new firewall, the UAC changes, and the integration of Windows Defender. To put that claim to the test, we attempted to infect our Vista setup with the Hotbar spyware package, a moderately annoying piece of malware that displays advertising and tries to phone home a record of user activities.
Going the direct route, we visited Hotbar's site in IE7+ and downloaded the Hotbar application directly from their site. Much to our surprise, Vista did not complain about this past the fact that we were running an executable we downloaded, something that Windows XP does just as well. Vista continued to sit idly by as we ran the installer for Hotbar, and we ultimately did not encounter any issues installing it.
It was not until we tried to remove it that we realized that Microsoft did not ship Vista with any spyware definitions, which is partially the reason that Windows was so passive about it being installed. In fact, until we installed those definitions, the only thing that kept Hotbar contained was the last failsafe, the firewall, which detected Hotbar attempting to connect to the internet and allowed us to block it before any further damage could be done. Once Microsoft's definitions were installed, we were able to remove Hotbar using Windows Defender without a problem. We then tried to install Hotbar again, at which point Defender notified us that we were trying to install a known piece of spyware and allowed us to abort the installation.
Given this test, we're not terribly convinced about Windows' anti-malware abilities at this time. In spite of UAC, Hotbar seemed perfectly happy running as a user-limited process, and it was only the firewall that kept it in check. Trashing a user account is for all practical purposes equally as destructive as trashing the entire system, so this is not a significant improvement.
It also puts Windows Defender in a bad light, as it appears that it will be of limited use in the case of dealing with a piece of malware it doesn't recognize. Certainly Defender will keep a subset of computer users from consistently reinstalling something that is spyware that they don't know about, but this may very well just lead to an arms race for spyware much like viruses today, which is not an effective situation. The firewall saved us, but that's not always going to be enough.
Facebook
Twitter
RSS
75 Comments
View All Comments
Squidward - Friday, June 16, 2006 - link
Having beta tested Windows XP when it was released, I have to say that so far I'm not very impressed with Vista. Granted there is still quite some time before final release but even with RC1 of XP it was a rock solid stable OS that I used as my full time OS and never had any issues whatsoever (especially security cause no one was writing viruses and malware for it back then). Quite frankly I don't see how the beta 2 I've been looking at and the final polished out the door product is going to happen in 7 months for a Jan. launch. The real problem however lies in the fact that I know I will move up to Vista at some point, but not because it's a better OS than XP but that I'll be hindered by continuing to use an older operating system. I just haven't seen anything in it yet that made me go. "Now that's the kind of feature I've been needing!", and the few features that did make me feel that way were removed to be implemented 'at a later date'. Fancy graphical effects are nice and all, but they don't make an OS. As it stands in the betas the UAC feature is just a complete hinderance that to me seems to punish the end user because of security risks that are out there. The end user shouldn't get a pop up on every single application or item they open to be sure it's 'safe'. There are far better means of controlling permissions within an OS that would have made a lot more sense that what we have now with UAC. That being said, I believe in time and with Microsoft really listening to customer feedback they'll work out a lot of the kinks, but I won't consider purchasing Vista until they do... or force me to upgrade. :)Pirks - Friday, June 16, 2006 - link
Besides this thing being early beta, also keep in mind that it's not a cosmetic chaneg akin to upgrade from W2k to XP or from OSX 10.3 to 10.4 - this is a major OS overhaul not too far from migration from 9x to NT, of course early beta of such a grand release will be total crap (at least for many people, but some others seem to enjoy it a lot). So, comparing this early beta release to XP release candidate is indeed pretty stupid. I don't even expect Vista release to be 100% usable out of the box, ESPECIALLY x64 version - Vista 64 will take another year or two to mature, get drivers/apps ready and such. And you should also keep in mind that MS is in a big hurry to avoid Apple to chop its balls off - some more delay and you'll see Apple market share well over 10% which is pretty dangerous to MS if they wanna keep enjoying their desktop x86 OS monopoly status. Hence MS does stuff quickly, cuts off features and will probably release something buggy just to avoid serious threat from Apple. Expect something usable only after SP1 and give it at least a year - in a meantime read some rumours about Leopard and salivate a little - that'll keep you going ;-))Yet another nice point - you think MS will sit still and let Leopard to chew its (MS's) private parts with impunity? I doubt that - MS will very likely release those nice sweet WinFS and other toys there were promising for years and integrate them in the next Vista release (I hope Leopard or whatever Mr. Jobs is up to isn't going to eat that for lunch - 'cause WinFS is the last hope for MS, really - DX10 won't count, too small a market it seems). So, in two years or maybe earlier you'll get those new sexy features you want, I believe... well, Apple could probably beat MS's ass here again, which is even more likely judging how well Apple devs were performing so far, so maybe you won't be interested in Vista at all - OS scene moves very fast - bang bang and u'r dead :) Especially now when Ballmer replaced BG - I'm worried, I don't quite trust Ballmer and Ozzie and others - ol' Bill was da man, not sure Vista survives w/o him when his archrival Jobs is only started to accelerate before real takeoff (Leopard?), but we'll see, we'll see...
Oh, interesting, tell stupid us what is this "far better means of controlling permissions within an OS" instead of annoying ugly UAC, this must be something revolutionary and ingenious - maybe MS will pay you big bucks for that, who knows ;-))
ChronoReverse - Friday, June 16, 2006 - link
You tested RC1 of XP. Release Candidate 1.This is BETA 2 of Vista. Maybe when they release RC1 of Vista you can compare again.
Frallan - Friday, June 16, 2006 - link
Well i found 1 thing to be more interesting then the rest: Gaming Perfomance!!That means that at least til the games I want to play are DX10 combined with the fact that DX10 games get better results im going to stax with my XP.
Sorry M$
/F
Googer - Friday, June 16, 2006 - link
There are so many versions and the feature sets will confuse most of us.Here is a screen shot from Paul Thurott's Win Super Site.
http://pics.bbzzdd.com/users/Googer/Windows_Vista_...">Windows Vista Versions.
Ryan Smith - Friday, June 16, 2006 - link
Keep in mind that that's an old chart. Small Business Edition no longer exists, and Professional is now Business Edition.Googer - Saturday, June 17, 2006 - link
Thanks forthe update. Here is the now silghtly out of date chart but still has some usefull information.http://www.winsupersite.com/showcase/winvista_edit...">http://www.winsupersite.com/showcase/winvista_edit...
slashbinslashbash - Friday, June 16, 2006 - link
Page 8, "regulated" should be "relegated"Also in the same sentence, "Superfetc.h" (which might not be a typo)
A 14-page article with 2 minor problems.... The quality ratio here at AT just kills DailyTech.... please impose AT quality control on DailyTech!
JarredWalton - Friday, June 16, 2006 - link
Fixed - DT runs a lot of short, quick articles, and unfortunately that means they get more typos and errors. Anyway, since they are a separate entity, there's not much we can do. Feel free to post and tell them, though, but remember they're looking at probably 10X as many press releases as we do. LOLDerekWilson - Friday, June 16, 2006 - link
1) vista is perfectly capable of being a stable light weight desktop system (with some quirks) at the beta 2 stage ... but try to do anything fast or power hungry and you'd be better off sticking with xp until vista is released. right now, at beta 2, vista is a neat toy. don't try to use it for everything.2) after all the spit an polish dries, i will still prefer os x to vista
3) final verdict? same as it ever was -- i'll be running vista for games and linux for programming. and since i've recently been bitten by the switch bug, os x for everything else.