Hi David, Just to point out — there’s an alternate solution to achieve interleaving of pairs of ADCs on an RFSoC board. This solution doesn’t require a custom input board. It assumes you’re designing for a particular sampling rate, not a variable sampling rate, because it uses fixed time delays, not variable ones.
What you would do is first combine the analog input with an attenuated DAC output. The DAC output would be turned off for normal operation, but would be turned on to generate calibration signals for an initial system calibration. This combined signal would go to a splitter, which splits the signals to the two ADC inputs. The cable lengths to the two ADCs is different. They're selected so that the delay to one of the ADCs is approximately half a sampling period more than the delay to the other ADC. This gives the effect of interleaved sampling. ADCs are set for synchronous sampling. The two ADCs selected need to be on the same board so they’re sampled at close to the same time instant. (To not mess up the 1/2 ADC clock separation built into the cabling.) If you’re doing beamforming and/or correlating across large numbers of channels — especially across multiple boards — the one synchronization signal from a single DAC should be split so it goes to each analog input on every FPGA board. Cable lengths from the DAC to each analog combiner should be matched. The splitter that splits the DAC signal is also a critical component and either needs to have matched phase at all outputs or have known characterization. With this configuration, the input board doesn’t need to be specially designed. It just needs to accommodate your full input bandwidth. For example, the XM500 could be used. You couldn’t use the existing baluns on that board since they don’t have the bandwidth you’re asking for, but you could use the inputs on that board that don’t have existing baluns and add your own suitable set of baluns there. This would allow you to test it without any special hardware. Note also that the much-discussed synchronization issues of the ZCU111 across multiple boards do not apply, since synchronization is accomplished as part of the calibration. ZCU111 boards can be synchronized in this way. The boards must all run from the same sampling clock, but the exact phase of that clock from board to board can be arbitrary, so long as it is calibrated once and thereafter doesn’t change. Thus you could have a system with arbitrary numbers of ZCU111 boards this way all being cross-correlated. The real trick to making this work is in the signal processing and calibration. Each ADC has its pair, with one having even samples and the other having approximately-odd samples. The even-sampled and approximately-odd-sampled each go through their own independent and separate PFBs. This is because PFBs naturally are splittable into two half-PFBs, with one half even samples and the other half odd samples. In this case, the full PFB with all even and odd samples cannot be constructed, because we only have approximately-odd samples, not odd samples. However, both of the half-PFBs can be constructed. So we do. To combine these half-PFBs into a full PFB, first it should be noted that the filter coefficients of the approximately-odd PFB are different than the even PFB. They’re offset. Ideally the offset should be by the exact time offset from even to approximately-odd samples, but instead it’s generally constructed assuming perfect odd-sample timing. The difference isn’t a major source of error. So basically, the PFB coefficients for the full PFB are generated, and the even half-PFB gets the even filter coefficients and the approximately-odd half-PFB gets the odd filter coefficients. If we were combining an odd half-PFB and an even half-PFB into a full PFB, the other thing of note is that the combination is done by the radix-2 final stage of the FFT. So we multiply the odd half-PFB frequency bins by the twiddles, and then if we want a low frequency, we add the two corresponding half-PFB bins, and if we want a high frequency we subtract them, just like the formula for a radix-2 FFT stage. What may not be commonly known is that the twiddle factors in an FFT that combine two half-FFTs into a full-FFT are a time correction. The time correction matches the time of the odd FFT samples to match the time of the even samples. This time correction is frequency-dependent, which is why the twiddles vary with frequency. The time-correcting twiddle phase at frequency bin n+FULL_FFT_SIZE/2 is exactly a factor of -1 times the time-correcting twiddle phase at frequency n. This is why one twiddle can correct bins n and n+FULL_FFT_SIZE/2 simultaneously. Differing by only a -1 in correction, they can be corrected simultaneously except for the factor of -1. The factor of -1 then goes into the radix-2 stage, with the low-frequency output of the radix-2 stage getting a +1 and the high-frequency output of the radix-2 stage getting a -1. The point is that it being a time correction, if the actual time of the approximately-odd samples is slightly off from the perfect odd-sample time, it doesn’t obviate the FFT. Instead, it changes the time correction in the final FFT stage. It’s no longer a radix-2 stage, but instead a phase-modified radix-2 stage. What we really have is a matrix operator applied to obtain the even and approximately-odd samples from the signal, and we need its inverse. If the sampling time is close to an exact half-sample, the inverse is approximately just the reverse of the original phase error. However, if the sampling time of the approximately-odd samples approaches the same time as the even samples, the matrix approaches a singularity and inversion becomes dicey. Thus it’s best to have approximately-odd samples be as close to odd as possible. Another way to view this is that the radix-2 stage is designed for perfect odd samples, and each half-PFB is an approximate frequency spectrum. Frequency spectrums can be time-shifted using a frequency-dependent phase multiply via the Fourier time-shifting property. So the approximately-odd PFB outputs can be time-shifted with a phase-multiply to be exactly what the radix-2 stage is expecting, and then the final FFT stage performed. This formulation is equivalent. It should be noted that this final FFT modified-radix-2 stage is a linear operator. Other linear operators are ADC imbalance, frequency-dependent ADC rolloff, splitter imbalances, differing cable losses, differing splitter frequency responses, and ADC sampling time errors across different analog inputs. As all of these are linear operators, all of these can be combined into a single linear correction. Furthermore, such a linear correction is already performed in a correlating system. This is the amplitude/phase adjustment prior to signal correlation. So all this can already be taken care of without any hardware architecture change. The one difference is that if there was a full PFB then the correlator would get a single input from the PFB for each correlated frequency. With two half PFBs, the correlator gets two inputs — one from each half-PFB. These two inputs are then corrected for all the linear time, frequency, and amplitude imbalances in the existing complex multiply of the correlator and combined via the equivalent of a radix-2 stage in the correlator addition. This can all done simultaneously with the other fine time corrections of the correlator that are used to point the array. To make this work, the correlator gets twice as many inputs as it might otherwise have. Note that a pre-correlation can be done on the initial FPGA to combine the two half-PFB outputs into a single output, and thus avoid any increase in I/O bandwidth to the main correlator. The exact amount of time-shift error and other imbalances are unknown, but that’s what the calibration is for. Putting some known-phase sine waves in for a variety of frequencies and correlating them on all the input channels will give this information, allowing the combined correlation-and-correction coefficients to be calculated. So long as there aren’t any phase disturbances in the system, the calibration will stay valid. In this way, you can have a system interleaving ADCs for effectively higher sampling rates, correlating across multiple FPGA boards that are frequency-locked but not phase-locked. This avoids issues with synchronized ZCU111’s, or helps if you want to avoid tricky hardware synchronizations. Only front-end cabling hardware changes are required, along with optional signal processing changes. Most especially, calibration must be performed. Calculation of correlator coefficients becomes more complicated. Regards, Ross > On Oct 26, 2020, at 10:59 AM, David Forbes <[email protected]> wrote: > > Gareth, > > I’m working on next-generation wideband digitizing with the Black Hole PIRE > group. We at the U of Arizona have taken the approach of using the most > readily available ADC, the one in the Xilinx RFSoCs. The platform is > currently the ZCU111. We plan to migrate to its follow-on, the ZCU208. We’re > currently exploring the feasibility of interleaving pairs of ADCs on these > boards for 4 GHz RF bandwidth, by making our own input board that provides > the necessary clock and signal processing. > Others in the PIRE group are pursuing other brands of ADCs, but they also use > a Xilinx development board to provide the DSP function. > > When it gets to the point that we can’t get enough boards from Xilinx, the > quantity will be high enough to justify designing and producing a CASPER > version of the ZCU series. I don't have the wherewithal to do that board > design, but I can provide input to the designers. > > > On Monday, October 26, 2020 at 5:50:00 AM UTC-7 [email protected] wrote: > Hi Casper Community > > Now that roach2 has been deprecated, I have been wondering where the CASPER > community is heading in terms of future ADC work. > > As far as I can tell there are three options available: > SNAP boards - The SNAP boards seem to support the largest number of options > 12 x 250 Msps/ 6 x 500 MSps or 3 x 1000 Msps. SNAP is used by HERA, but I > don't think it is used anywhere else. > SKARAB and the SKARAB ADC - The SKARAB ADC can sample at up to 3 GSps. From > what I can tell, it does not seem to be widely used. I imagine it would be > quite an expensive configuration. > ZCU111 RFSoC - The ZCU111 RFSoC seems to be a good board for experimentation, > but if we wanted to build a many antenna array (N > 100), XIlinx may not be > quite able/willing to provide us with that many dev boards. > Alternatively, maybe there is some cheap FMC ADC out there that could make > everyone happy? (Although then we would need to find an FMC carrier card) > > From the options available, it seems to me that SNAP is the board that is > most likely to be deployed in a large array, and the ZCU111 board is what is > most likely to be used in labs/small arrays. > > Is that a correct read of what is available? Or are there other projects in > the works? > > We have cheap COTS options for building X/F-Engines. As far as I can tell, an > easily accessible ADC board is the main bottleneck to quickly > prototyping/building a correlator. > > Gareth Callanan > Digital Signal Processing Engineer > South African Radio Astronomy Observatory(SARAO) > > -- > You received this message because you are subscribed to the Google Groups > "[email protected]" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/8dee0003-f0d3-4d76-9880-8be777ef4b39n%40lists.berkeley.edu > > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/8dee0003-f0d3-4d76-9880-8be777ef4b39n%40lists.berkeley.edu?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/4D17E9BA-E8FD-416F-AA2F-6455D1AC57B8%40ieee.org.
