I played around with ROACH2 QDR calibration a few months ago and the following must be noted;
The physical trace length between FPGA and QDR SRAM is a fixed length. This fixed length equates to a different number of clock cycles, depending on clock speed. The interfacing firmware assumes a fixed number of clock cycles latency. The calibration routine thus only works for a certain range of clock speeds. In your case it looks like (at least) one of the interfaces is writing (or reading) incorrectly, but is compensating by inserting or removing a multiple of half a clock cycle of delay when reading (or writing). When reading/writing via the other interface, a different number of compensating half clock cycles is inserted. Which repo are you using? Regards Andrew On Wed, Mar 11, 2015 at 8:15 AM, David George <[email protected]> wrote: > A small chip-in - QDR doesn't work below 120 MHz: > http://www.cypress.com/?id=4&rID=31542 > > Regards > > On 10 March 2015 at 22:17, Matt Strader <[email protected]> wrote: > > Hi Jack, > > > > Thanks for your reply. I think I understand better what's going on. I > was > > thinking in terms of 64-bit words that get written or read each cycle, > but > > looking in qdr_cal(), I should probably think of two 32-bit words read or > > written each cycle. Then, perhaps I shouldn't be too surprised if the > order > > of the two words read out doesn't match my intuition. > > > > Initially, I was compiling at 100 MHz, but the qdr_cal always failed. > Then > > I tried 200 and 250 MHz, and the qdr_cal succeeds. > > > > My tests go like this. For the 200 MHz design, > > I call qdr_cal, and it returns True. > > > > Then I write this sequence of 64-bit values via the CPU: > > [0, 1, 2, 3, 4, 5, ..., , 1048574, 1048575] > > > > I read back the sequence from the CPU, and it agrees. > > [0, 1, 2, 3, 4, 5, ..., , 1048574, 1048575] > > > > I read it in fabric and get > > [1048575<<32, 0<<32, 1<<32, 2<<32, 3<<32, 4<<32, ..., 1048574<<32, ] > > > > > > Then I try the other direction, by writing a sequence of 64-bit values in > > fabric: > > [0,1,2,3,4,5, ... , 1048574, 1048575] > > > > I read back the sequence in fabric and it agrees: > > [0,1,2,3,4,5, ... , 1048574, 1048575] > > > > I read it via the CPU: > > nBytesPerSample=8 > > struct.unpack('>{}Q'.format(nSamples), > > roach.read('qdr0_memory',nSamples*nBytesPerSample)) > > and get: > > [1<<32, 2<<32, 3<<32, 4<<32, 5<<32, ..., 1048575<<32, 0<<32] > > > > So reading and writing only by the CPU is consistent, and reading and > > writing only in fabric is consistent, but when I do one by the CPU and > the > > other in fabric, the two 32 bit words get swapped, and the sequence is > > offset by one value. I actually see the same offset in a Roach1 version > of > > the design as well. It all seems consistent, not intermittent. > > > > Thanks, > > Matt > > > > > > > > On Mon, Mar 9, 2015 at 10:42 AM, Jack Hickish <[email protected]> > wrote: > >> > >> Hi Matt, > >> > >> The qdr_cal routine calibrates the relationship of clock and data > signals > >> in the link between the FPGA and QDR signals. Basically it writes test > >> patterns and reads them back looking for glitches. It then changes the > delay > >> of IODELAY blocks so that data read from the QDR is captured reliably. > >> If this calibration script runs successfully, I'm surprised you still > see > >> glitches with 32 bits of data swapped between read and write (both from > >> PPC). This would (surely?) count as a glitch and cause the calibration > >> routine to fail(?). Do you only see strange behaviour when you read > from the > >> FPGA fabric, or do straight write/reads from the CPU also misbehave? Is > the > >> behaviour the same on every read / write (i.e., not intermittent)? > >> > >> My experience with the QDR is that at high clock speeds (not sure how > >> high, but I see this at 312 MHz) the read latency of the QDR interface > isn't > >> 10 clock cycles as it should be even with the minimum IODELAY > configuration, > >> and there are no reliable qdr_cal solutions. I solved this problem for > my > >> design by removing half a clock cycle latency from the interface, at > which > >> point the QDR would calibrate ok. What speed are you running your > design at? > >> To me, misordered 32-bit chunks of data sounds indicative of a latency > >> problem in the qdr_cal solution, since the interface is 32 bit DDR, so > half > >> a clock cycle mangles 32-bit chunks of data. > >> > >> Good luck! > >> > >> Jack > >> > >> > >> > >> On Fri, 6 Mar 2015 at 23:54 Matt Strader <[email protected]> > >> wrote: > >>> > >>> Hi all, > >>> > >>> I'm trying to use QDR on Roach2. I have a test design where I write to > >>> QDR either in firmware or with katcp, then read it out in the firmware > and > >>> check the results with snapshot blocks. It mostly works, but with some > >>> interesting quirks. > >>> > >>> I know that katcp can only write 64 bits of the 72 bit QDR word, so I > >>> changed the bitbasher blocks in qdr_mask.m to move the inaccessible > bits out > >>> of the way to the most significant bits. Now everything seems to work > well, > >>> except that in some builds (seems to depend on clock rate?), the first > 32 > >>> bits (of the 64 bits I'm writing in a cycle) get swapped with the last > 32 > >>> bits sometime between writing the word and reading it back. And there > is a > >>> sometimes a difference in whether I am writing in firmware or by katcp > to > >>> whether this swap happens. > >>> > >>> Also, when I write with katcp starting at address zero, I find that > when > >>> reading out, the write started at what seems to be address 1. Is the > >>> latency of QDR set to 10 cycles? > >>> > >>> I can live with all this, as long as it is repeatable for a given > design, > >>> but I thought I'd ask if anyone knows what is causing these quirks. > >>> > >>> Also, I understand that we should run a software calibration for qdr, > >>> which seems to be qdr_cal() in qdr.py in the ska-sa:casperfpga repo. > So, I > >>> run this first, which seems to work for my latest compiles, but I > haven't > >>> noticed a difference in the results. What exactly does this > calibration do? > >>> > >>> Thanks, > >>> Matt Strader > >>> UC Santa Barbara > > > > > >
