Real World DirectX 10 Performance: It Ain't Pretty
by Derek Wilson on July 5, 2007 9:00 AM EST- Posted in
- GPUs
Final Words
For now, AMD does seem to have an advantage in Call of Juarez, while NVIDIA leads the way in Company of Heroes and Lost Planet. But as far as NVIDIA vs. AMD in DirectX 10 performance, we really don't want to call a winner right now. It's just way too early, and there are many different factors behind what we are seeing here. As the dust settles and everyone gets fully optimized DirectX 10 drivers out the door with a wider variety of games, then we'll be happy to take a second look.
The more important fact to realize is that DirectX 10 is finally here. While developers are used to programmable hardware after years with DirectX 9, there is still room for experimentation and learning with geometry shaders, more flexibility, lower state change and object overhead, and (especially) faster hardware. But DirectX 10 isn't an instant pass to huge performance and incredible effects.
Let's look at it like this: there are really three ways a game can come to support DirectX 10, and almost all games over the next few years will ship with a DX9 path as well. The easiest thing is to do a straight port of features from DirectX 9 (which should generally be slightly faster than the DirectX 9 counterpart if drivers are of equal quality). We could also see games offer a DirectX 10 version with enhanced features that could still be implemented in DX9 in order to offer an incentive for users to move to a DX10 capable platform. The most aggressive option is to implement a game focused around effects that can only be effectively achieved through DirectX 10.
Games which could absolutely only be done in DX10 won't hit for quite a while for a number of reasons. The majority of users will still be on DX9 platforms. It is logical to spend the most effort developing for the user base that will actually be paying for the games. Developers are certainly interested in taking advantage of DX10, but all games for the next couple of years will definitely have a DX9 path. It doesn't make sense to rewrite everything from the ground up if you don't have to.
We are also hearing that some of the exclusive DX10 features that could enable unique and amazing effects DX9 isn't capable of just don't perform well enough on current hardware. Geometry shader heavy code, especially involving geometry amplification, does not perform equally well on all available platforms (and we're looking at doing some synthetic tests to help demonstrate this). The performance of some DX10 features is lacking to the point where developers are limited in how intensely they can use these new features.
Developers (usually) won't write code that will work fine on one platform and not at all on another. The decisions on how to implement a game are in the hands of the developers, and that's where gamers rightly look when performance is bad or hardware and feature support is not complete. Building a consistent experience for all gamers is important. It won't be until most users have hardware that can handle all the bells and whistles well that we'll see games start to really push the limits of DX10 and reach beyond what DX9 can do.
In conversations with developers we've had thus far, we get the impression that straight ports of DX9 to DX10 won't be the norm either. After all, why would a developer want to spend extra time and effort developing, testing and debugging multiple code paths that do exactly the same thing? This fact, combined with the lack of performance in key DX10 features on current hardware, means it's very likely that the majority of DX10 titles coming out in the near term will only be slightly enhanced versions of what could have been done through DX9.
Both NVIDIA and AMD were very upset over how little we thought of their DX10 class mainstream hardware. They both argued that graphics cards are no longer just about 3D, and additional video decode hardware and DX10 support add a lot of value above the previous generation. We certainly don't see it this way. Yes, we can't expect last years high-end performance to trickle down to the low-end segment, but we should at least demand that this generation's $150 part will always outperform last generation's.
This is especially important in a generation that defines the baseline of support for a new API. The 2400 and 8400 cards will always be the lowest common denominator in DX10 hardware (until Intel builds a DX10 part, but most developers will likely ignore that unless Intel can manage to pull a rabbit out of their hat). We can reasonably expect that people who want to play games will opt for at least an 8600 or a 2600 series card. Going forward, developers will have to take that into account, and we won't be able to see key features of games require more horsepower than these cards provide for the next couple of years.
AMD and NVIDIA had the chance to define the minimum performance of a DX10 class part higher than what we can expect from cards that barely get by with DX9 code. By choosing to design their hardware without a significant, consistent performance advantage over the X1600 and 7600 class of parts, developers have even less incentive (not to mention ability) to push next generation features only possible with DX10 into their games. These cards are just not powerful enough to enable widespread use of any features that reach beyond the capability of DirectX 9.
Even our high-end hardware struggled to keep up in some cases, and the highest resolution we tested was 2.3 megapixels. Pushing the resolution up to 4 MP (with 30" display resolutions of 2560x1600) brings all of our cards to their knees. In short, we really need to see faster hardware before developers can start doing more impressive things with DirectX 10.
For now, AMD does seem to have an advantage in Call of Juarez, while NVIDIA leads the way in Company of Heroes and Lost Planet. But as far as NVIDIA vs. AMD in DirectX 10 performance, we really don't want to call a winner right now. It's just way too early, and there are many different factors behind what we are seeing here. As the dust settles and everyone gets fully optimized DirectX 10 drivers out the door with a wider variety of games, then we'll be happy to take a second look.
The more important fact to realize is that DirectX 10 is finally here. While developers are used to programmable hardware after years with DirectX 9, there is still room for experimentation and learning with geometry shaders, more flexibility, lower state change and object overhead, and (especially) faster hardware. But DirectX 10 isn't an instant pass to huge performance and incredible effects.
Let's look at it like this: there are really three ways a game can come to support DirectX 10, and almost all games over the next few years will ship with a DX9 path as well. The easiest thing is to do a straight port of features from DirectX 9 (which should generally be slightly faster than the DirectX 9 counterpart if drivers are of equal quality). We could also see games offer a DirectX 10 version with enhanced features that could still be implemented in DX9 in order to offer an incentive for users to move to a DX10 capable platform. The most aggressive option is to implement a game focused around effects that can only be effectively achieved through DirectX 10.
Games which could absolutely only be done in DX10 won't hit for quite a while for a number of reasons. The majority of users will still be on DX9 platforms. It is logical to spend the most effort developing for the user base that will actually be paying for the games. Developers are certainly interested in taking advantage of DX10, but all games for the next couple of years will definitely have a DX9 path. It doesn't make sense to rewrite everything from the ground up if you don't have to.
We are also hearing that some of the exclusive DX10 features that could enable unique and amazing effects DX9 isn't capable of just don't perform well enough on current hardware. Geometry shader heavy code, especially involving geometry amplification, does not perform equally well on all available platforms (and we're looking at doing some synthetic tests to help demonstrate this). The performance of some DX10 features is lacking to the point where developers are limited in how intensely they can use these new features.
Developers (usually) won't write code that will work fine on one platform and not at all on another. The decisions on how to implement a game are in the hands of the developers, and that's where gamers rightly look when performance is bad or hardware and feature support is not complete. Building a consistent experience for all gamers is important. It won't be until most users have hardware that can handle all the bells and whistles well that we'll see games start to really push the limits of DX10 and reach beyond what DX9 can do.
In conversations with developers we've had thus far, we get the impression that straight ports of DX9 to DX10 won't be the norm either. After all, why would a developer want to spend extra time and effort developing, testing and debugging multiple code paths that do exactly the same thing? This fact, combined with the lack of performance in key DX10 features on current hardware, means it's very likely that the majority of DX10 titles coming out in the near term will only be slightly enhanced versions of what could have been done through DX9.
Both NVIDIA and AMD were very upset over how little we thought of their DX10 class mainstream hardware. They both argued that graphics cards are no longer just about 3D, and additional video decode hardware and DX10 support add a lot of value above the previous generation. We certainly don't see it this way. Yes, we can't expect last years high-end performance to trickle down to the low-end segment, but we should at least demand that this generation's $150 part will always outperform last generation's.
This is especially important in a generation that defines the baseline of support for a new API. The 2400 and 8400 cards will always be the lowest common denominator in DX10 hardware (until Intel builds a DX10 part, but most developers will likely ignore that unless Intel can manage to pull a rabbit out of their hat). We can reasonably expect that people who want to play games will opt for at least an 8600 or a 2600 series card. Going forward, developers will have to take that into account, and we won't be able to see key features of games require more horsepower than these cards provide for the next couple of years.
AMD and NVIDIA had the chance to define the minimum performance of a DX10 class part higher than what we can expect from cards that barely get by with DX9 code. By choosing to design their hardware without a significant, consistent performance advantage over the X1600 and 7600 class of parts, developers have even less incentive (not to mention ability) to push next generation features only possible with DX10 into their games. These cards are just not powerful enough to enable widespread use of any features that reach beyond the capability of DirectX 9.
Even our high-end hardware struggled to keep up in some cases, and the highest resolution we tested was 2.3 megapixels. Pushing the resolution up to 4 MP (with 30" display resolutions of 2560x1600) brings all of our cards to their knees. In short, we really need to see faster hardware before developers can start doing more impressive things with DirectX 10.
Facebook
Twitter
RSS
59 Comments
View All Comments
titan7 - Wednesday, July 11, 2007 - link
CoH also got 59.9fps on the GTX Ultra. Today's d3d10 games can run at full frame rates on today's hardware. Just ensure you're using the latest drivers and you have the most expensive card money can buy ;)BigDDesign - Thursday, July 5, 2007 - link
I'm with Derek here. I liked your article. We need much more powerful hardware and time for DX10, Drivers, Developers & Vista to get it together. A couple of years from now, all should be good with DX10. Not any sooner methinks.WaltC - Thursday, July 5, 2007 - link
This article, I thought, was extremely poor for several reasons:(1) DX10 in terms of developer support presently is just about exactly where DX9 was when Microsoft first released it. At the time, people were swearing up & down that DX8.1 was "great" and wondering what all of the fuss about DX9 really meant. Then we saw the protracted "shader model wars" in which nVidia kept defending pre-SM2.0 modes while ATi's 9700 Pro pushed nVidia all the way back to the drawing boards, as its SM2.0 support, specific to DX9, created both image quality and performance that it took nVidia a couple of years to catch.
AnandTech did indeed mention almost in passing that DX10 was still early yet, and that much would undoubtedly improve dramatically in the coming months, but I think that unfortunately AT created the impression that DX10 and DX9 were exactly *alike* except for the fact that DX10 framerates were about half as fast on average as DX9 framerates. A cardinal sin of omission, no doubt about it, because...
(2) DX10 is primarily if not exclusively about improvements in Image Quality. It is *not* about maintaining DX9-levels of IQ while outperforming DX9. It is about creating DX10 levels of Image Quality--period. AnandTech does not seem to understand this at all.
(3)First lesson in Image Quality analysis that even newbies can readily understand is this: if the performance is not where you want it, but the IQ is where you want it, then you do the following to improve performance *without* sacrificing Image Quality (This is a lesson that AnandTech truly seems to have completely forgotten):
Instead of talking about how sorry the performance was in DX10 titles (those very few early attempts that AT looked at), AT should have seen what AT could have done to increase performance while maintaining DX10-levels of Image Quality. That is, AT should have *lowered* test resolutions and raised the level of FSAA employed to get the best balance of Image quality and performance. AnandTech did not even try to do this--which in my view is inexcusable and fairly unforgivable. It is a very bad mistake. I'm sorry--but many, many people, including me, do not use 1280x1024 *exclusively* while playing 3d games. My DX9 resolution of choice is 1152x864, for instance.
The point to be made about DX10 is *not* frame rates locked in at 1280x1024. Sorry AT--you really screwed the pooch on this one. The whole point of DX10 is *better image quality* which everyone who ever graduated from the 3d-school-of-hard-knocks is *supposed* to know!
So, just what does a bunch of *bar charts* detailing absolutely nothing except frame rates tell us about DX10? Not much, if anything at all. Gee, it does tell us that with the reduced image quality that DX9 is capable of providing contrasted with DX10, that DX9 runs faster in terms of frames per second on DX10 hardware! Gosh, who might ever have guessed....<sarcasm>
IMO, the fact that DX10 software even early on is running slower than DX9 on DX10-compliant hardware tells *me* nothing except that DX10 is demanding a lot more work out of the hardware than DX9, which means that we can expect the Image Quality of DX10 to be much better than DX9. These early games that AT tested with are merely the tip of the iceburg of what is to come. AnandTech really blew this one.
Jeff7181 - Tuesday, September 11, 2007 - link
"DX10 is *not* frame rates locked in at 1280x1024"Too bad you didn't say this earlier in your rant, I could have saved myself a few minutes and stopped reading sooner.
Thanks for the laugh though... imagine... someone who thinks 1280x1024 is a frame rate telling AnandTech they screwed the pooch when publishing this article. LOL
titan7 - Thursday, July 12, 2007 - link
This article was awesome. Too bad reality doesn't match your expectations or you'd like it too.1) You have no idea what the "shader war" was about. There isn't even a single parallel to be drawn if you *correctly* look back.
2) d3d10 is nothing about image quality. No hardware on the market or rumoured to be coming out next generation can handle running a full length SM2.0b shaders! And forget about even 3.0! Or even 2.0a that the GeForceFX supported for that matter. d3d10 is about one thing-making d3d on the PC more like a console. That means lower CPU (not GPU!) overhead (more performance in CPU limited situations) and making lives easier for developers.
3) AT has CoH at 1024x768, 1280x1024, 1600x1200, and 1920x1200 (common native LCD resolutions). These days everybody has bigger monitors and playing below 1024x768 is pretty rare. You can extrapolate your 1152x864 resolution from that.
What this shows us is if you increase the image quality (see the screen shots) today's cards don't have enough power to run at full frame rate, with the exception of the 8800 GTX Ultra. AT did a great job showing the world that with simple bar graphs and everything. It's too bad you were too angry to realize that is all it was trying to show.
strikeback03 - Friday, July 6, 2007 - link
umm, they tested Company of Heroes and Call of Juarez at 1024x768, and Lost Planet at 800x600, and some of the cheaper cards could still not maintain playable frame rates. How low on the resolution do you want them to go?jay401 - Friday, July 6, 2007 - link
I'm not sure how raising FSAA is going to improve performance?Nor how removing DX10 visuals but lowering screen res will "maintain DX10 level of visuals" either?
anandtech02148 - Thursday, July 5, 2007 - link
Excellent sincere analysis of the current hardwares situation. Which lead me to some after thoughts,- Maybe a 600buxs PS3 isn't so bad after all.
-What am i going to do with this 8800gtx and the lack of pc games, quite dry season compare to consoles.
-For those of you who hold out longer than I have, a 8800gts or 2900xt is a decent investment if you have a 1920x1200 monitor to go with it.
kilkennycat - Thursday, July 5, 2007 - link
... if you presently have a DX9 system with acceptable performance. Until the NEXT generation of DX10 hardware is released.Anybody who goes out and buys a current Dx10 card ( or even worse, dual cards ) just because "the Dx10 games are coming" ( and "I want bragging-rights") has a lot more money than sense. Buying a 8600 or 2600 for acceptable HD-decoding in your HTPC (or your mid-range PC with a weak CPU) is the only purchasing action with the current Dx10 offerings that makes total sense. All of the upcoming Dx10-capable game-titles for 2007 will have excellent Dx9/SM3 graphics. The lack of Dx10 hardware will smother "bragging-rights" but will have zero effect on playability.
nVidia has been developing the successor family to the G80-series GPU for almost a year now and the first graphics cards from this new generation are expected by the end of 2007. I would not be at all surprised if the first card out of the chute in the new family will immediately fill the cost-space between 8600GTX and 8800GTS, but with DX9 and Dx10 performance far superior to the 8800GTS. No doubt the GPU will also be 65nm, since the manufacturing/yield cost of the huge 80nm G80 die is the immovable stumbling-block to dropping the price of the 8800GTS.
KeithTalent - Thursday, July 5, 2007 - link
That's fine if you game at resolutions below 1680x1050, but some of us game at higher resolutions, and most high-end DX9 cards were struggling mightily to play the latest DX9 games (Oblivion, Supreme Commander, STALKER, etc...).These new generation cards are not only about DX10, they are also about improved performance (exponentially improved performance actually) over that last generation, and some of us actually need that power.
What you said is all well and good for you if you are still gaming at 800x600 or whatever, but I like my resolution a little higher thank you very much.
KT