> Dan Werthimer wrote: >> each GPU can handle 100 to 200 MHz dual pol depending on whether >> you are doing coherent dedispersion (timing), or spectroscopy >> (searching). >> matthew and jonathan are the experts at reading data from ibob/roach >> and >> using CPU cluster to do pulsar/transient search. >> john ford, paul demorest, scott ransom et al are the experts at using >> ibob/bee2 >> to packetize data (800 MHz dual pol) for GPU based pulsar cluster >> (see their fantastic GUPPI instrument). > We could have up to 1400 MHz at once, 8200-8600 and 31,500-32,500 MHz > but I think only one polarization. I saw that John Ford is using 8 GPUs > for 800 MHz. Can you get several GPUs on the single bus of a multi-core > host or does that cause too much of a bottle-neck? I also should think > about doing the various piggy-back tasks in parallel. I'm guessing that > setispec on a ROACH is a tight fit. How about two? The kurtosis is a > very light task, I think, so can some of the left-over resources be used > to expand the SETI bandwidth or refine the resolution? > > Anyway, for now it's some high-level wishing so I'll scope one unit at > three dual-channel ADCs, three ROACHes, two 4 core hosts, and 8 GPUs. > Does that seem reasonable? About $40K? (We have to pay Xilinx :-( .)
I think you'll run out of PCIe slots and/or bandwidth if you try to do it in 2 hosts. The 10 GbE cards need 8 lanes, and the GPUs need 16 lanes each. You'll need at least 2 10 GbE ports to service 4 GPUs. That's 4 X16 slots and 2 X8 slots. Paul Demorest spec'd out 8 hosts in our GPU cluster due to the I/O requirements, both 10 Gbe and GPU's. He may have been a bit conservative, but beware! My quick estimate says 45K or so assuming 4 hosts. It might be nice if we could come up with some benchmarks that show how much we can process with each GPU, how many GPUs and 10 GbE ports can be supported per host, etc. John > Thanks for your help > > Tom > >
