hi mayukh, if might help people guide you if you can you say a bit more about your experiment. what instrument and signal processing is running on the FPGA ? (spectrometer? correlator ? beamformer ? ) what is the FPGA output 12Gbit/sec data stream look like (spectra? complex channelized voltage data? )
what does the processor do with the 12Gbit/sec data stream after it receives it? store it on hard disks? SSD? (for how long?) compress the data and then store it ? compress the data and then transmit it to a ground station ? send uncompressed 12Gbit/sec to the ground via a very high speed data link ? can you compress the data on the FPGA ? instead of sending the data over 100/25/10 Gbit link to an external computer, can you instead process the data using the ARM core processors on the RFSOC ? there are six ARM cores on the RFSOC's FPGA. for info on these ARM processors, see: https://www.amd.com/en/corporate/university-program/aup-boards/rfsoc4x2.html if you need to send the data from the RFSOC board to an external computer over 100G, 25G, or a pair of 10G ethernet links, berkeley casper postdoc ben godfrey sent me a spreadsheet that lists a lot of low power mini-PC computers. (please see email from ben below). one of the column lists the number and type of PCIe slots, another column lists the power consumption of the board (you need to add the NIC power to this). another berkeley casper postdoc, wei liu, sent me an email about the PCIe slot on the Raspberry Pi 5. please see email from wei liu below ben's. best wishes, dan Ben Godfrey 1:07 PM (9 minutes ago) Reply to Wei, me Not sure how useful this is, but here's a link to a Google Drive with a lot of mini PC's on it. Link: https://docs.google.com/spreadsheets/d/1SWqLJ6tGmYHzqGaa4RZs54iw7C1uLcTU_rLTRHTOzaA/edit?gid=239063037#gid=239063037 - Ben On Fri, Sep 27, 2024 at 1:02 PM Wei Liu <liuwei_berke...@berkeley.edu> wrote: > Hi Dan, Ben, > > Here is an interesting blog about pcie on rpi5: > > https://www.jeffgeerling.com/blog/2023/forcing-pci-express-gen-30-speeds-on-pi-5 > > It looks like the pcie gen can be forced from 2.0 to 3.0. > Someone tested 10G NIC on it, and got 6Gbps bandwidth. > > Best, > Wei > On Mon, Sep 23, 2024 at 11:33 AM Mayukh Bagchi <mayukh.bag...@queensu.ca> wrote: > Hey Dan, > > Thanks for getting back. > > My first concern is being able to find a compatible hardware/computer to > plug the NIC into. Are these systems commercially available? Or are most of > them custom-built? I could budget around 100 W for such a computer. I don't > think the power draw of the NICs themselves is concerning for us, we could > manage that. > > Also, I am not sure if the 100 GbE yellow block would be compatible with a > dual 10 GbE NIC or a 40 GbE NIC, if it does then that opens up quite a few > options. > > > Regards, > Mayukh > > ------------------------------ > *From:* 'Dan Werthimer' via casper@lists.berkeley.edu < > casper@lists.berkeley.edu> > *Sent:* Monday, September 23, 2024 1:48 PM > *To:* casper@lists.berkeley.edu <casper@lists.berkeley.edu> > *Subject:* Re: [casper] 100 GbE Hardware recommendation > > > > Hi mayukh, > > You might consider two 10 gbe links and a dual 10 gbe nic. Or a 40 gbe > nic. Are you concerned with the power consumption of the 100 gbe nic, or > the consumption of the computer the nic is plugged into? How much > computing power and storage do you need ? > > Best wishes, > > Dan > > On Mon, Sep 23, 2024, 1:18 PM Mayukh Bagchi <mayukh.bag...@queensu.ca> > wrote: > > Hello Casperites, > I hope you all have been doing well. > > I have been developing an RFSoC-based backend for our Balloon-borne VLBI > Experiment (BVEX) project, which will be a mobile K-band VLBI station, > launching in August next year. I am using the RFSoC 4x2 to digitize and > stream data for our IF stage. The CASPER community has been very helpful > with setup and troubleshooting. > > So far, I've successfully implemented a spectrometer and a power meter. > While the design for the 100 GbE data streaming is almost ready, I am > having trouble selecting a compatible NAS/server-side system to directly > communicate with the 100 GbE Mellanox NIC > <https://www.fs.com/products/119648.html?attribute=67745&id=3746425>. The > reason why it is getting tricky is the size, thermal regulation, and power > consumption constraints, as this will be on a balloon platform with limited > resources. Our data rates should be around ~10-12 Gbps, depending on how > much we end up re-quantizing, so we may not use the full capabilities of > the 100 GbE network. However, at this point, any suggestion on hardware > will be useful in getting us pointed in the right direction. > > Feel free to share your advice on working with 100 GbE Mellanox NICs. If > this seems challenging, I could probably lower our data rates and try the > 10GbE yellow block Ethernet core. > > Thanks in advance! > > Regards, > Mayukh > *__________* > > *Mayukh* *Bagchi* (He/Him), PhD Candidate > Department of Physics, Engineering Physics, and Astronomy > mayukhbagchi.com | mayukh.bag...@queensu.ca > [image: Queen's University Logo] <https://www.queensu.ca/> > > -- > You received this message because you are subscribed to the Google Groups " > casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB10632111BE97CE7C079D9E886916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB10632111BE97CE7C079D9E886916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups " > casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAA9jdzJmp1E%3DZknwdB4V18WypD8ak0TewhDoqe8WsTSk1i1-YA%40mail.gmail.com > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAA9jdzJmp1E%3DZknwdB4V18WypD8ak0TewhDoqe8WsTSk1i1-YA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups " > casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB1063206055042E9DCB07FFFD2916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB1063206055042E9DCB07FFFD2916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAA9jdzLWS6pRoFjaBh52WvDPCiAE_y-hrv19LD66wwDpYixoWw%40mail.gmail.com.
