Matt- Good summary. Let me correct recent changes: B05, B10, and B15 FPGA 4 each have two active XAUI ports (one of them runs both with fiber CX4, I think, the others are a mix), and one active 10GbE port. Have several copies of similar firmware running on B16.
Correct that we do not run high speed on the center FPGA - corners only right now. Also do not have any designs with more than one 10GbE port per FPGA (most of our interchip comm is XAUI), so I don't know how helpful these numbers are. John: Have you been locking up the same FPGA every time, or can you lock up any arbitrary user FPGA? I've seen some effects (link reliability / BER, ability to drive fiber CX4) where there is measurable variation between different user fpgas. So far, I have not correlated user-fpga-location to firmware lockups, but wouldn't immediately rule it out, esp. if this wasn't a design to be run on all corners. Billy -----Original Message----- From: [email protected] on behalf of Matt Dexter Sent: Wed 2/3/2010 8:14 PM To: [email protected] Subject: Re: [casper] 10 GbE ports on a BEE2 Hi, I'm not sure this is too helpful as this system uses 10 Gbps XAUI and not 10 GbE (and I'm sure Billy Barott could say more) but for the record (and since Dan asked) I believe the ATA Beamformer's 16 BEE2s are cabled as ATA BF number of 10 Gbps ports used per BEE2 and FPGA : BEE2 Center Corner FPGAs FPGA 1 2 3 4 BF1 B01 0+0 3+0 1+2 4+0 3+0 B02 0+0 3+0 1+2 4+0 3+0 B03 0+0 3+0 1+2 4+0 3+0 B04 0+0 3+0 1+2 4+0 3+0 B05 0+0 2+2 3+0 3+0 0+0 BF2 B06 0+0 3+0 1+2 4+0 3+0 B07 0+0 3+0 1+2 4+0 3+0 B08 0+0 3+0 1+2 4+0 3+0 B09 0+0 3+0 1+2 4+0 3+0 B10 0+0 3+1 3+0 3+0 0+0 BF3 B11 0+0 3+0 1+2 4+0 3+0 B12 0+0 3+0 1+2 4+0 3+0 B13 0+0 3+0 1+2 4+0 3+0 B14 0+0 3+0 1+2 4+0 3+0 B15 0+0 2+1 3+0 3+0 0+0 etc B16 0+0 0+0 2+1 1+2 1+0 First number is number of passive copper CX4 cables. Second number is number of active fiber optic CX4 cables. The maximum number of usable active CX4 cables per BEE2 is quite low due to power supply limitiations. I believe various lab tests were done using 1 of the 2 high speed ports on the center FPGA but at the moment none are in use at the observatory (as far as I know). Matt Dexter On Wed, 3 Feb 2010, John Ford wrote: > Hi all. Has anyone seen any problems when using 4 10 GbE ports on a > single FPGA in the BEE2? > > We're having some problems where borph access from the control FPGA seems > to lock up when we start up our designs. We've done some testing with > hacked up designs, and we can only make it lock up (so far) when we have 4 > 10 gbe ports active. Interestingly, some of the 10 GbE ports also seem to > lock up at that very instant. > > The fpga still is running, but the control fpga cannot talk to it OR ANY > OF THE OTHER 3 FPGA's. If you kill the borph process on the control fpga > and restart it, it comes back to life. > > We don't know what goes on under the hood of the yellow block system, but > it seems that something is going wrong with whatever mechanism controls > the bus to the control fpga. > > John > > > > > > > >
