[casper] Inter-chip connections problem

[Jason Ray]

All,

Randy and I are closing in one completing phase 1 of our pulsar 
machine.  We have two iBOB samplers streaming samples over XAUI into 
the BEE2's FPGA1 and FPGA3, where the pfb & fft take place.  Then 
we're using the inter-chip connections to pass the FFT bin values 
over to FPGA2 where the two streams are combined and then things like 
stokes, scale/offset, vacc, data packing, and 10Ge happen.  The ADC 
clock is set to 800MHz and we're generating a 100MHz clock signal on 
each iBOB and piping it out of an SMA GPIO to the BEE2, which is set 
up to use usr_clk2x.

The problem we're working on right now is that it seems the data 
"pipes" from the FPGA1 and FPGA3 are not aligned properly when they 
are received in FPGA2.  We've been looking at things like the 
propagation delay differences between the two paths due to trace 
routing to make an educated guess about what to set the phase delays 
to (so far on the receive end only), but we have not had much luck.

We believe the two data pipes aren't aligned because the two FFT 
syncs aren't aligned.  We have previously been AND-ing the two syncs 
together to make a combined sync in FPGA2.  We saw some peculiar 
behavior with the BEE2 running at 100Mhz but it worked about 50% of 
the time.  However, when we switched to 200MHz, the combined sync 
never happens.  We added a software register to select a mux that 
allows us to either choose to AND the syncs or OR them.  Whenever AND 
is selected it never happens.  Whenever OR is selected is (obviously) 
happens every time.  This seems to be a fairly clear indicator that 
the two streams aren't aligned properly.

Has anyone had any success with using all 138 inter-chip connections 
from two corner FPGAs, and getting them aligned properly in the third 
common FPGA, at 200MHz?

Thanks in advance!
Jason