On Friday 29 January 2010 10:18:42 pm John Ford wrote: > > Dan Werthimer wrote: > >> each GPU can handle 100 to 200 MHz dual pol depending on whether > >> you are doing coherent dedispersion (timing), or spectroscopy > >> (searching). > >> matthew and jonathan are the experts at reading data from > >> ibob/roach and > >> using CPU cluster to do pulsar/transient search. > >> john ford, paul demorest, scott ransom et al are the experts at > >> using ibob/bee2 > >> to packetize data (800 MHz dual pol) for GPU based pulsar cluster > >> (see their fantastic GUPPI instrument). > > > > We could have up to 1400 MHz at once, 8200-8600 and 31,500-32,500 > > MHz but I think only one polarization. I saw that John Ford is > > using 8 GPUs for 800 MHz. Can you get several GPUs on the single > > bus of a multi-core host or does that cause too much of a > > bottle-neck? I also should think about doing the various > > piggy-back tasks in parallel. I'm guessing that setispec on a > > ROACH is a tight fit. How about two? The kurtosis is a very light > > task, I think, so can some of the left-over resources be used to > > expand the SETI bandwidth or refine the resolution? > > > > Anyway, for now it's some high-level wishing so I'll scope one unit > > at three dual-channel ADCs, three ROACHes, two 4 core hosts, and 8 > > GPUs. Does that seem reasonable? About $40K? (We have to pay > > Xilinx :-( .) > > I think you'll run out of PCIe slots and/or bandwidth if you try to > do it in 2 hosts. The 10 GbE cards need 8 lanes, and the GPUs need > 16 lanes each. You'll need at least 2 10 GbE ports to service 4 > GPUs. That's 4 X16 slots and 2 X8 slots. Paul Demorest spec'd out > 8 hosts in our GPU cluster due to the I/O requirements, both 10 Gbe > and GPU's. He may have been a bit conservative, but beware!
I just finished a travel day from hell and was going to respond exactly to this point, but John beat me too it. I think the real limitation with wideBW pulsar processing on CPUs/GPUs nowadays is the I/O. So consider this email strong support of John's comments. Scott -- Scott M. Ransom Address: NRAO Phone: (434) 296-0320 520 Edgemont Rd. email: [email protected] Charlottesville, VA 22903 USA GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989
