J. Storrs Hall, PhD. writes:
NVIDIA claims half a teraflop for the 8800 gtx. You need an "embarassingly parallel" problem, tho.
That claim is slightly bogus (I think they are figuring in some graphics-specific feature which would rarely if ever be used by general purpose algorithms [texture interpolation?]). The actual numbers are pretty easy to compute, I think:
128 stream processors x 1.35GHz clock = 173 billion instructions per second. Since one of the instructions is a Multiply-Add, couble that to a 346 gigaflops peak. Counting a multiply-add instruction as a two-flop is probably okay because so many algorithms can make use of it (matrix operations, convolutions, etc).
As is usually the case, memory bandwidth is a bigger issue. Access to the 768mb card memory has a bandwidth of about 80 gb/sec, which means that to keep the processors busy, one needs a compute intensity of about 8 instructions for each load of a 4-byte float. This is the primary reason that most computations don't hit the peak numbers -- for example, multiplying large matrices using their libraries can hit 100 gflops but I don't think they have improved it beyond that. Convolution could do somewhat better I think, partially because the kernel can be saved in on-chip memory.
Latency complicates the programming -- when data needs to be fetched from card memory, figure 200 cycles for it to get delivered. Other "threads" can get useful work done during that time, but for that to happen there has to be a huge number of threads (thus the "embarassingly parallel" comment you made).
The biggest limitation, of course, is the bandwidth between the main system memory and the card memory over the pci express bus, where one is lucky to get 2 gb/sec.
Sorry if this doesn't seem too much like cognitive science; I don't think it's completely off-topic though to talk about the computational resources available to AGI research.
----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=231415&user_secret=fabd7936
