City of Villains (Beta) Tests
Update: We would like to clarify that the beta version of City of Villains was stated as non-optimized in the release notes. The test server is open to any CoV subscriber and it does support PhysX hardware, but it is very possible for performance improvements to be made to the implimentation before their production release of the Issue 7 patch. In fact, we have been told by AGEIA that performance on the final code will absolutely be better than on the test server. We look forward to testing and reporting on PhysX support under production CoV code/servers as soon as possible.As with any MMOG, City of Villains has been difficult to test. The fact that PhysX hardware is only supported in a very specific area of gameplay makes it even more complicated. Luckily, we found a way around these issues. As City of Villains makes use of instanced areas for missions (multiple parties entering a single zone will each play the game in their own copy of that area), we were able to eliminate any outside influence from other players and go on a mission solo. Also key in our ability to test the Mayhem Missions (the part of City of Villains in which the PhysX hardware is used) is the fact that it is currently on the test server.
Normally, before a gamer gets a Mayhem Mission, he or she will have to do a bunch of other missions, level up a bunch of times, and unlock a contact called a "broker." The broker gives you access to "newspaper missions." After doing 3 of the newspaper missions, a broker will offer you a special job: a heist. In the future, and on the test server, this is where mayhem missions will be available. Each mayhem mission, once begun, has a 15 minute time limit. This limit can be extended by destroying things in the city, but there is no way to get enough time to actually benchmark multiple configurations of hardware. Doing 3 other newspaper missions and getting another mayhem mission isn't an option because this takes a huge amount of time (the missions can be different every time as well).
Fortunately, NCSoft offers the ability to copy your character to the test server at any time, multiple times. Our solution was to do enough missions to get to a point where we could be offered a mayhem mission (even though the missions are currently only on the test server). Then we copy our character over to the test server a bunch of times. Now we are able to accept the same mission any time we want.
This is great for testing a feature like PhysX, but doing a full test of the game is still hard. Working with a team fighting a huge number of enemies is a key element in the game. It's something that just can't be reliably repeated barring the assembly of an AnandTech guild built solely for choreographing, reenacting, and benchmarking scenes hundreds of times. For now, this is all we need. We included a screenshot of the game options with and without a PhysX card installed:
We created a couple videos to show the difference between hardware accelerated and software physics. Here are the numbers that resulted from our tests.
What we see clearly shows that the PhysX card is putting quite a strain on performance. Yes we get more mail or packing peanuts spewing forth from our rampant destruction. Yes, we do think it looks great with more stuff going on. But we are really not happy with the performance we are seeing here. It appears that our GPU is not overly loaded at these resolutions, as the difference between 800x600 and 1600x1200 gives very little in the way of performance gain without PhysX hardware installed. This implies that the bottleneck in performance is in the rest of the system. This is confirmed by the fact that performance drops considerably when running the test on a much slower CPU.
Whether on a low resolution with a fast CPU or a high resolution with a slow CPU, the PhysX hardware gives us low average framerates, very low minimum instantaneous framerates, and adds a bit of stutter to the movement of the game. Unlike the Ghost Recon test we showed last week, playing the game with the PhysX card feels much less satisfying. Multiple frames seem to take a long time to render (rather than just one or two) giving movement a choppy feel to it.
That being said, it is very important that we point out the fact that we are benchmarking code running on a test server. NCSoft makes it clear that code running on this server is not optimized and players may see degraded performance. We hope very sincerely that the performance issues will be resolved; right now it just isn't worth it.
So why include these numbers at all if it's on a test server? Well, the fact that they back up the results we saw in GRAW last week do lend some credibility the tests. What we seem to be seeing is that games which use the PhysX processor to tack on special effects physics take a performance hit when those effects are triggered. Of course, that only holds if the driver didn't fix the performance issues we saw in Ghost Recon.
Facebook
Twitter
RSS
67 Comments
View All Comments
segagenesis - Wednesday, May 17, 2006 - link
I feel so tempted to bring up the old cliche "The message is clear..." when you word it like that :)Really why is there not more "WTF" here? A better analogy to what you describe is the old "Hardware Decelerators" that say the S3 Virge was. And for $300? Damn, next thing we know they will be sub-licensing Patty-On-Patty technology from Burger King with a dual core physics processor for only $600! *groan*
They have the right idea here but this is some of the poorest execution possible in convincing people you need this product.
Magnadoodle - Wednesday, May 17, 2006 - link
Calling this a physics decelerator seems just perfect. I wish anandtech would use some biting humour now and then. But that would mean degraded relations with Asus and BFG.Oh well, let's just get nostalgic about the days of unconstrained journalism and reread those old 6% Pcgamer reviews.
abhaxus - Friday, May 19, 2006 - link
When I got my original voodoo 1 card, the first thing I did was plug it in and run a few timedemos in GLquake... surprise surprise, it was actually a few FPS slower than I was running in software mode. Of course, I was running software mode at 320x240 and GL at 640x480 and the game looked incredible.I haven't seen a PhysX card in person but the trailers for cellfactor look very impressive. With PhysX being taken advantage of throughout the design and coding process I can't wait to see what the final results are for new games... of course, new drivers and a PCIe version will help too.
That said... I really think that this card will eventually turn out to be only for people that don't have a dual core CPU. Seems like most everything could be done by properly multithreading the physics calculations.
Nighteye2 - Wednesday, May 17, 2006 - link
It's perfectly possible to remain be critical while remaining polite. Biting humour is unnecessarily degrading and does not add any value. Even 6% ratings can be given in perfectly polite wording.DerekWilson - Wednesday, May 17, 2006 - link
We certainly aren't pulling punches, and we wouldn't do anything to preserve a relationship with any company. If we make someone angry, we've still got plenty of ways to get a hold of their product.I hope we were able to make it clear that CoV giving similar results to GRAW gave us pause about the value of PhysX when applied to games that just stick in some effects here and there. We also (I hope clearly) stressed that there isn't enough value in the product for consumers to justify a purchase at this time.
But we weren't overly hard on AGEIA as we could be for a couple reasons. First, CellFactor and HangarofDoom are pretty interesting demos. The performance of them and the possibilities presented by games like them indicate that PhysX could be more useful in the future (especially with its integration into UE3 and other game engines). Second, without more tools or games we just can't determine the actual potential of this hardware. Sure, right now developers aren't making practical use of the technology and it isn't worth its price tag. But it is very premature for us to stamp a "decelerator" label on it and close the case.
Maybe we will end up calling this thing a lemon, but we just need more hard data before we will do so.
Magnadoodle - Wednesday, May 17, 2006 - link
Yes, I understand your point of view, and I don't think you're pulling any punches or being biaised. In fact, a biting review would be more biaised than anything. I was just remarking that this would have made a perfect occasion to have a bit of fun with AGEIA and drag them through the dredges. I nostalgically recalled the quite biting and humorous style PC Gamer put into their 6% reviews. PC Gamer never was a pantheon of game reviewing, but they didn't have to be nice to nobody (actually to "nobodies", because they had to be nice to big corporations). My point was more about the lack of wits and style in web publications these days than about anandtech being biaised. Not that anandtech has bad writers, just that it's more scientific than sarcastic.Anyway, good review Mr. Wilson and keep up the good work.
Seer - Wednesday, May 17, 2006 - link
Im also wondering about this claim that the driver update increased framerates. In all but two of the tests, the avg fps was either the same or a decrease. The largest increase was 1 fps, totally within the margin of error. (I'm talking about the GRAW tests). So, um, yeah, no increase there.