Console Emulators: Our Newest Benchmark
by Derek Wilson on March 3, 2004 11:33 PM EST- Posted in
- GPUs
One late night (or perhaps it was an early morning), during the testing of the ATI Radeon 9800 XT and the NVIDIA GeForce FX 5950 Ultra last Fall, we decided that the only truly "correct" way to test a video card is to test it in every possible situation for which it could be used. Of course, since there is no magic benchmark fairy that brings us numbers while we sleep, testing everything is impossible. We decided to compromise and make the 30 most popular games at any given time our standard GPU benchmark. We are still trying to hit that 30 game mark, but in the delirium of the long nights of testing, we managed to think of a few more ways we could push hardware. Unfortunately, we are all out of rubber pants and whipped cream; we had to go with our second choice: console emulation.
The original idea was to use either an NES or SNES emulator as a real world framebuffer upload benchmark. Emulators of aged consoles can use the CPU to emulate the entire system and just send frames over to the GPU as fast as they are cranked out. Something like this may still be a possibility, but as we delved deeper into the depths of the emulator community, we discovered much more exciting things that we could bring to the table. We finally settled on benchmarking a PlayStation emulator. The N64 and Dreamcast emulators are out there, but nothing is quite as popular as the PlayStation right now. There are PlayStation 2 emulators in development, but nothing that can play games quite yet. We will take a look at a very early Game Cube emulator running "The Legend of Zelda: The Windwaker" as a bit of a preview of things to come. Before getting too excited, it's not really playable yet; we just wanted to include Game Cube emulation to take a look at where we are at this point.
There are a few factors that need careful consideration when looking at PlayStation emulation. First, running a PlayStation game on a PC without upsampling textures, applying a bit of blur, and possibly adding a TV-like scanline filter won't quite deliver the look of the original game. It really is amazing how much the television can hide those jagged edges and make low resolution textures look all right. Of course, the trade off is performance, so it really is a user preference kind of issue. We wanted to make the game look and feel as much like the original as possible with the emulator settings we chose. The added bonus is that our hardware gets pushed a little harder.
And this wouldn't be an article about emulation without mentioning the fact that it is perfectly legal (because of a lovely thing called "fair use") to emulate hardware that you own in order to run software that you own. It is not legal to distribute games (or even the PlayStation BIOS for that matter).
Facebook
Twitter
RSS
37 Comments
View All Comments
TrogdorJW - Thursday, March 4, 2004 - link
Pretty interesting read. I highly doubt that emulation of any of the lesser consoles would prove to be a good benchmark, except as a CPU benchmark. Maybe N64 would be okay, but SNES, Genesis, and anything earlier are all so "simple" compared to modern 3D consoles that performance is almost a non-issue. Anyway, good article, even if it is inherently biased towards ATI. Not much that can be done about that, though, unless some skilled programmers with Nvidia hardware want to try and release a better plugin.My one thought is that even with the disclaimer up front stating that Pete owns a 9700 and thus his plugins are written and tested on ATI first and foremost, there are probably going to be some clueless people out there saying, "OMG ATI pwned Nvidia again." Then again, there's not much that can be done to help the clueless. Maybe someone should start up a charity for them? :) Anyway, I was thinking a little disclaimer at the bottom or top of each page of the article might be nice, but it's probably much more effort than it's worth.
DerekWilson - Thursday, March 4, 2004 - link
(#22) KGB:if you own a playstation (or have the legal rights to a playstation BIOS) you could rent a game from blockbuster, download the tools used in the article and play a game on your PC ... The problem with that is that my blockbuster doesn't rent out playstation games anymore :-(
There is another option if you don't own a PS BIOS ... you can use an emulator with an HLE BIOS (at the cost of perfect BIOS support for your games). PCSX has a builtin HLE BIOS.
(#21) Junkman:
I hate to spoil it for everyone, so I hope someone else watched the same cartoons as I did in my formative years ... Adult Swim on the cartoon network has become my favorite stuff now, but this reference is a throwback to one of my favorite cartoon duos of all time ...
KGB - Thursday, March 4, 2004 - link
SO if i rent a game from blockbuster, pop it in my CD-ROM drive, the game works with my pc?Junkman - Thursday, March 4, 2004 - link
Rubber pants and whipped cream????? Interesting article. Thanks DerekSDA - Thursday, March 4, 2004 - link
Depends on what you define as fair, #19. It's a real-world benchmark, so above all else it's useful for determining how video cards perform in that specific application. Whether or not performance in it can be applied to other applications, ePSXe with Pete's OGL plugin is just one of (if not THE) best method of PSX emulation out there, so why not use it for a benchmark?boboy - Thursday, March 4, 2004 - link
It's no insider secret that Pete Bernett uses ATI cards. He programs his plugins with very few outside testers, and he has a SERIOUS lacking of of nvidia testers. So much, he didn't even know his initial version of his OGL2 plugin didn't support Geforce 4 Ti cards on winXP until he signed on NGEmu a day later. He doesn't do anything like that on purpose, which is obvious since his OGL2 plugin is getting better and better with nvidia cards with every release. It's just that the usefullness of this test is next to nil because the plugin runs far better on ATi cards than NVidia cards.Although it should also be noted that OpenGL2 was made for compatibility, not visual quality. A lesser card can make a game look much prettier in full speed on his other plugins, although you'll often come across graphical glitches, and you'll need to be turning on special game fixes (hacks/workaronds) and the such for quite a few of the games. Hell, even my old TNT2 can run alot of psx games with good quality at 60fps.
All in all, this was a very interesting read, as I follow the emulation scene myself. But in the end, I don't think these kind of benchmarks can give anywhere near as fair results than other benchmarks, programs that were designed to use the full extent of both card brands equally.
elty - Thursday, March 4, 2004 - link
ePSXe and Pete's plugin used to work with GeForce better. Pete has some updates, but ePSXe remains about unchanged. So do you have any insider info that it is written based on ATi card only before you flame? Mayber later we don't need any conclusion at all becasue all you will come up with is "Card A perform poorly on Game X becasue Developer M did it on purpose" or "It shows that CPU B pays money to all developers so it will run faster than CPU D"tfranzese - Thursday, March 4, 2004 - link
Fair in my perspective, and interesting even though I don't delve into emulators much - I do find them entertaining from time to time.SDA - Thursday, March 4, 2004 - link
It's a good idea, but if FFIX is any indication it's just barely too late, as the hardware used (even the 5700 Ultra) seems to be able to handle the games being used without much trouble. I know this might sound stupid, but is it possible to force antialiasing in the driver options and have it carry over to the game? Or was something that did this just as well already enabled, or does this not do anything? I was just thinking, because if it works like it does in un-emulated games it would strain the video cards more.. probably some reason this can't be done, though. In that case, what is it?Also, while it's interesting from an academic standpoint when real-world benches such as this favor one manufacturer's products over another, it doesn't change the validity of the benchmarks. After all, in real-world benchmarks you actually want to see how everything performs in that specific benchmark. In other words, yeah, in my opinion it's perfecty fair. That's just me, though.
DerekWilson - Thursday, March 4, 2004 - link
(#11) taleril:We think that's fair enough as well; now we have:
"From our benchmarks, it is obvious that ATI Radeon cards are better suited for emulating the games that we tested under ePSXe and Pete's plugins with the settings we chose."
It is definitely important to be specific about exactly what a part was better at doing than another part. It is not good practice to do otherwise.
We feel that this is a fair comparison of ATI and NVIDIA products performance in the independant software development segment based on our statments on page 4 of the article. This is a different issue than that of corporate development since what we care about most is end user value. We are demonstrating the proper value of the hardware in the emulator space.
I hope this clears things up a bit.