Hi Jan
> There are mentions of SPEAD in for example the Simulink models. No actual > low-level nor high-level implementations though. Have checked OpenCores as > well. There is an SKA memo [1] that does mention implementations, but these > refer to Python/C++. Looking at the C++ code, it does not look fit for using > in HLS really... Wondering - are there public SPEAD implementations for FPGA > available somewhere? I'm not aware of any HDL implementations. MeerKAT's SPEAD TX and RX FPGA modules are essentially "hard coded" (well, somewhat compile-time parameterised) Simulink modules. If we receive a valid SPEAD packet that doesn't match what we expect (checking SPEAD header and Item IDs), we flag an error on the FPGA. So it doesn't support all the SPEAD features, like on-the-fly datastream redimensioning. The Science Data Processor's high performance C implementation doesn't support this, either, so we're comfortable with it. For the benefits of the mailing list, it's really only the Python PySPEAD module that supports all these things, and you pay a performance penalty there. SPEAD2, the C-accellerated version that's mostly in use now, allows for one-time configuration, and then allocates memory and beds that down, until the receiver is restarted (eg when the observation changes). > an existing FPGA-based VLBI backend should be extended instead by the backend > firmware engineers, so that it can accept SPEAD-delivered raw sample data > over parallel 10G links. We did essentially the same thing on MeerKAT's early correlator, where we used 4x10G links into a ROACH2, and then unpacked and buffered them into a single on-FPGA datastream. The F-engines received 36Gbps SPEAD data from our digitisers, and channelised this data, before retransmitting another SPEAD stream to the X-engines. So we had SPEAD RX and TX blocks in each FPGA. But, these were only partially parameterised, and they were written in Simulink. They were rather hastily cobbled together, but you're welcome to grab an example of an old ROACH2 designs here if you think they'd be of value: https://github.com/ska-sa/mkat_fpga/blob/master/source/feng_wide/r2_c856m4k.slx This uses a small BRAM for heap assembly. IIRC, there was some fiddling with byte/bit/word ordering in this design. Or, the newer, simpler, SKARAB versions for 40G, here, which uses very large off-chip HMC memory buffers: https://github.com/ska-sa/mkat_fpga/blob/master/source/xeng_wide/s_b4a4x1kf.slx > Looking at the SPEAD rev 0 document, a transmitter would appear lightweight > to implement and seems quite FPGA-suited. A receiver looks somewhat more > challenging on FPGA, easy in C++/Python though. The complex bits if I see > correctly would be large buffering for Heap reconstruction, This depends on how careful you are with your Heap partitioning. For our ADC-to-F, a heap is just a single packet, so it's trivial. For F-to-X, the heaps are larger. But we need buffer here anyway, to handle out-of-order network packet RX, so the cost is no more. We used to do this in BRAM on ROACH2s, but on SKARAB we have large, fast HMC available, so we use that. > and an interpreter engine to cope with the varying order and length of the > 'ItemDescriptors' to be generic enough and avoid code maintenance if anything > on the other side ie transmitter side is upgraded/altered. Could also be I'm > mistaken, and a receiver is easy on FPGA? Would appreciate your input and > experiences on this matter! :-) You have the right of it. So far, AFAIK, all FPGA implementations are "dumb" and search for values at known packet offsets. If the transmitter changes, we have to recompile with a new set of compile-time parameters. We manage this the old fashioned way with ICDs. The SPEAD2 software receiver is a little smarter, and allows runtime programmability of dataset dimensions at startup, but not for these to change during data transmission. PySPEAD allows it to change anytime, IIRC, but nobody uses it to move high speed data around because it's too slow. I don't think it's maintained anymore, TBH. Jason -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/C90333C6-D7C0-47CC-B148-928B8ACEB3EF%40ska.ac.za.
