Hi Danny,
It is 96 dual-polarization. It is named "Tainlai"(in chinese) and is built by National Astronomical Observatory of China. Best Wishes， Lin At 2015-04-02 12:02:56, "Danny Price" <[email protected]> wrote: >Hi Lin > >Sounds interesting, is that 192 dual-pol (i.e. 384 total) or single-pol >inputs? What will the correlator be used for? > >I'd suggest having a look at Mellanox OFED / libvma first (partly >because I'm interested to hear how well it works!), and then the NTOP >tool. I've heard hashpipe is quite easy to get started with also. > >- Danny > >shu.lean wrote: >> >> Hi Danny, >> >> Thanks very much for the references and suggestion. >> >> Our correlator is 192 inputs, 125 MHz bandwidth with 1024 frequency >> channel. >> And I want to do some test about how fast the computer can capture >> packet from the NIC. Now I have one NIC with dual 10GbE port made by >> Mellanox. >> >> Best Wishes, >> Lin >> >> >> >> >> >> At 2015-04-01 23:50:22, "Danny Price"<[email protected]> wrote: >> >Hi Shu Lean >> > >> >The short answer is: it depends, but you should probably buy a good Xeon >> >CPU if you are not sure. How many antennas and what bandwidth will you >> >have? >> > >> >The LEDA correlator (that I work with) uses xGPU for the X-engine, >> >and PSRDADA for packet capture / shared memory buffers. Here's some >> >papers for reference: >> >http://psrdada.sourceforge.net/ >> >http://www.worldscientific.com/doi/abs/10.1142/S2251171715500038?src=recsys >> >http://www.worldscientific.com/doi/abs/10.1142/S2251171714500020 >> > >> >For reference, our server specifications are: >> > >> >Ubuntu 12.04.2 LTS (GNU/Linux 3.5.0-23-generic x86_64) >> >Supermicro 1027TR-TQF >> >2x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz >> >128 GB RAM >> >2x NVIDIA K20X GPU >> >1x Mellanox ConnectX-3 40GbE NIC >> > >> >Each server (11 total) is capturing 21.4 Gb/s, for 512 inputs, with 2.6 >> >MHz bandwidth per server. >> > >> >The MWA correlator (http://arxiv.org/abs/1501.05992) and the PAPER >> >correlator also use xGPU. >> >I believe PAPER are using HASHPIPE instead of PSRDADA for their pipeline >> >https://github.com/david-macmahon/hashpipe >> > >> >The CHIME project have recently posted some pretty darn impressive info >> >on their correlator: >> >http://arxiv.org/abs/1503.06189 >> >http://arxiv.org/abs/1503.06203 >> >http://arxiv.org/abs/1503.06202 >> > >> >CHIME are not using xGPU as they are using AMD GPUs, which don't support >> >CUDA. I think LOFAR >> >might be doing something with AMD GPUs also. From the CHIME papers, >> >they seem to be getting amazing packet capture results by using NTOP >> >PF_RING ZC (Zero Copy): >> >http://www.ntop.org/products/pf_ring/pf_ring-zc-zero-copy/ >> > >> >Most NIC vendors now have "offload engines" that try and reduce load by >> >bypassing the kernel. For example, Mellanox has libvma, and Solarflare >> >has TCP OE. If you're about to build something big, I would highly >> >recommend you purchase some cards from different vendors and do some lab >> >tests. It may turn out that one card in particular has a killer feature. >> >Please let us all know if you do! >> > >> >- Danny >> > >> >shu.lean wrote: >> >> Hi all, >> >> >> >> if using GPU to do X Enigne, what the specification of computer should >> >> be, like the performance of CPU, DDR memory? So the CPU can transfer >> >> the data NIC recieved to GPU in time? >> >> >> >> Thanks very much. >> >> >> >> Lin >> >> >> >> >> >>
