Hi Shu Lean
The short answer is: it depends, but you should probably buy a good Xeon
CPU if you are not sure. How many antennas and what bandwidth will you
have?
The LEDA correlator (that I work with) uses xGPU for the X-engine,
and PSRDADA for packet capture / shared memory buffers. Here's some
papers for reference:
http://psrdada.sourceforge.net/
http://www.worldscientific.com/doi/abs/10.1142/S2251171715500038?src=recsys
http://www.worldscientific.com/doi/abs/10.1142/S2251171714500020
For reference, our server specifications are:
Ubuntu 12.04.2 LTS (GNU/Linux 3.5.0-23-generic x86_64)
Supermicro 1027TR-TQF
2x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
128 GB RAM
2x NVIDIA K20X GPU
1x Mellanox ConnectX-3 40GbE NIC
Each server (11 total) is capturing 21.4 Gb/s, for 512 inputs, with 2.6
MHz bandwidth per server.
The MWA correlator (http://arxiv.org/abs/1501.05992) and the PAPER
correlator also use xGPU.
I believe PAPER are using HASHPIPE instead of PSRDADA for their pipeline
https://github.com/david-macmahon/hashpipe
The CHIME project have recently posted some pretty darn impressive info
on their correlator:
http://arxiv.org/abs/1503.06189
http://arxiv.org/abs/1503.06203
http://arxiv.org/abs/1503.06202
CHIME are not using xGPU as they are using AMD GPUs, which don't support
CUDA. I think LOFAR
might be doing something with AMD GPUs also. From the CHIME papers,
they seem to be getting amazing packet capture results by using NTOP
PF_RING ZC (Zero Copy):
http://www.ntop.org/products/pf_ring/pf_ring-zc-zero-copy/
Most NIC vendors now have "offload engines" that try and reduce load by
bypassing the kernel. For example, Mellanox has libvma, and Solarflare
has TCP OE. If you're about to build something big, I would highly
recommend you purchase some cards from different vendors and do some lab
tests. It may turn out that one card in particular has a killer feature.
Please let us all know if you do!
- Danny
shu.lean wrote:
Hi all,
if using GPU to do X Enigne, what the specification of computer should
be, like the performance of CPU, DDR memory? So the CPU can transfer
the data NIC recieved to GPU in time?
Thanks very much.
Lin
