Derek Gets Technical Again: Of Warps, Wavefronts and SPMD

From our GT200 review, we learned a little about thread organization and scheduling on NVIDIA hardware. In speaking with AMD we discovered that sometimes it just makes sense to approach the solution to a problem in similar ways. Like NVIDIA, AMD schedules threads in groups (called wavefronts by AMD) that execute over 4 cycles. As RV770 has 16 5-wide SPs (each of which process one "stream" or thread or whatever you want to call it) at a time (and because they said so), we can conclude that AMD organizes 64 threads into one wavefront which all must execute in parallel. After GT200, we did learn that NVIDIA further groups warps into thread blocks, and we just learned that their are two more levels of organization in AMD hardware.

Like NVIDIA, AMD maintains context per wavefront: register space, instruction stream, global constants, and local store space are shared between all threads running in a wavefront and data sharing and synchronization can be done within a thread block. The larger grouping of thread blocks enables global data sharing using the global data store, but we didn't actually get a name or specification for it. On RV770 one VLIW instruction (up to 5 operations) is broadcast to each of the SPs which runs on it's own unique set of data and subset of the register file.

To put it side by side with NVIDIA's architecture, we've put together a table with what we know about resources per SM / SIMD array.

NVIDIA/AMD Feature NVIDIA GT200 AMD RV770
Registers per SM/SIMD Core 16K x 32-bit 16K x 128-bit
Registers on Chip 491,520 (1.875MB) 163,840 (2.5MB)
Local Store 16KB 16KB
Global Store None 16KB
Max Threads on Chip 30,720 16,384
Max Threads per SM/SIMD Core 1,024 > 1,000
Max Threads per Warp/Wavefront 960 256 (with 64 reserved)
Max Warps/Wavefronts on Chip 512 We Have No Idea
Max Thread Blocks per SM/SIMD Core 8 AMD Won't Tell Us
That's right, AMD has 2.5MB of register space

We love that we have all this data, and both NVIDIA's CUDA programming guide and the documentation that comes with AMD's CAL SDK offer some great low level info. But the problem is that hard core tuners of code really need more information to properly tune their applications. To some extent, graphics takes care of itself, as there are a lot of different things that need to happen in different ways. It's the GPGPU crowd, the pioneers of GPU computing, that will need much more low level data on how resource allocation impacts thread issue rates and how to properly fetch and prefetch data to make the best use of external and internal memory bandwidth.

But for now, these details are the ones we have, and we hope that programmers used to programming massively data parallel code will be able to get under the hood and do something with these architectures even before we have an industry standard way to take advantage of heterogeneous computing on the desktop.

Which brings us to an interesting point.

NVIDIA wanted us to push some ridiculous acronym for their SM's architecture: SIMT (single instruction multiple thread). First off, this is a confusing descriptor based on the normal understanding of instructions and threads. But more to the point, there already exists a programming model that nicely fits what NVIDIA and AMD are both actually doing in hardware: SPMD, or single program multiple data. This description is most often attached to distributed memory systems and large scale clusters, but it really is actually what is going on here.

Modern graphics architectures process multiple data sets (such as a vertex or a pixel and its attributes) with single programs (a shader program in graphics or a kernel if we're talking GPU computing) that are run both independently on multiple "cores" and in groups within a "core". Functionally we maintain one instruction stream (program) per context and apply it to multiple data sets, layered with the fact that multiple contexts can be running the same program independently. As with distributed SPMD systems, not all copies of the program are running at the same time: multiple warps or wavefronts may be at different stages of execution within the same program and support barrier synchronization.

For more information on the SPMD programming model, wikipedia has a good page on the subject even though it doesn't talk about how GPUs would fit into SPMD quite yet.

GPUs take advantage of a property of SPMD that distributed systems do not (explicitly anyway): fine grained resource sharing with SIMD processing where data comes from multiple threads. Threads running the same code can actually physically share the same instruction and data caches and can have high speed access to each others data through a local store. This is in contrast to larger systems where each system gets a copy of everything to handle in its own way with its own data at its own pace (and in which messaging and communication become more asynchronous, critical and complex).

AMD offers an advantage in the SPMD paradigm in that it maintains a global store (present since RV670) where all threads can share result data globally if they need to (this is something that NVIDIA does not support). This feature allows more flexibility in algorithm implementation and can offer performance benefits in some applications.

In short, the reality of GPGPU computing has been the implementation in hardware of the ideal machine to handle the SPMD programming model. Bits and pieces are borrowed from SIMD, SMT, TMT, and other micro-architectural features to build architectures that we submit should be classified as SPMD hardware in honor of the programming model they natively support. We've already got enough acronyms in the computing world, and it's high time we consolidate where it makes sense and stop making up new terms for the same things.

That Darn Compute:Texture Ratio A Quick Primer on ILP and ILP vs. TLP Extraction
Comments Locked

215 Comments

View All Comments

  • Final Destination II - Wednesday, June 25, 2008 - link

    http://www.techpowerup.com/reviews/Powercolor/HD_4...">http://www.techpowerup.com/reviews/Powercolor/HD_4...

    Look! Compare the Powercolor vs. the MSI.
    Somehow MSI seems to have done a better job with 4dB less.
  • Final Destination II - Wednesday, June 25, 2008 - link

    Try ASUS, 7°C cooler.
  • Justin Case - Wednesday, June 25, 2008 - link

    I thought it was only Johan, and it was sort of understandable since he's not a native English speaker, but it seems most Anandtech writers don't know the difference between "its" and "it's".

    "It's" means "it is" or "it has" (just as "he's" or "she's"). When you're talking about something that belongs to something else, you use "its" (or "his" / "her").

    In a sentence such as "RV770 in all it's [sic] glory.", you're clearly not saying "in all it is glory" or "in all it has glory"; you sare saying "in all the glory that belongs to it". So you should use "its", not "it's".

    Even if you can't understand the difference (which seems pretty straightforward, but for some reason confuses some people), modern grammar checkers will pick this up 9 times out of 10.
  • CyberHawk - Thursday, June 26, 2008 - link

    I am not a native English speaker, but I am well aware of the difference. I am also sure that reviewers are also ... it's just that - with all this text, we can forgive them, can't we?

    I have a bachelor of computer science, studying for higher degree, but: I look at the technical side of the article, so I don't even notice the errors :D (although I can tell the difference I simply don't see it while reading)
  • CyberHawk - Thursday, June 26, 2008 - link

    Oh, I forgot: maybe I'm just being too enthusiastic ;)
  • JarredWalton - Wednesday, June 25, 2008 - link

    More likely is that with a 10000 word article and four lengthy GPU reviews in two weeks, errors slipped into the text. I know at one point I noticed Derek says "their" instead of "there" as well, and I can assure you that he knows the difference. I know I use Word's grammar checker, but I'm not sure Derek even uses Word sometimes. :)
  • araczynski - Wednesday, June 25, 2008 - link

    of the 4850's, slickdeals has posted a sale, between rebate and coupon off...$150 each. can't beat that bang/$ by anything from nvidia.

    first ati cards that will ever be in my computers since i've started with the voodoo/riva tnt :)
  • Denithor - Wednesday, June 25, 2008 - link

    Page 15: first reference to "GTX 280" should be "GTX 260" instead.

    Page 19: I think you meant "type" not "time" in this paragraph.
  • natty1 - Wednesday, June 25, 2008 - link

    This review is flawed. It shows greater than 100% scaling for Crossfire 4870 in Call of Duty 4. Why don't they just give us the raw numbers for both single and dual cards in the same scenario? Why use a method that will artificially inflate the Crossfire results?
  • Denithor - Wednesday, June 25, 2008 - link

    If you read the comments before yours, you'd see the answer.

    Experimental error and/or improved scaling for each card versus a single card. Read the earlier comment for more details.

Log in

Don't have an account? Sign up now