we mainly use Zarlink for casper-casper and casper-Fujitsu switch links
Not NICs. I think the only NIC application is Fujitsu switch to
1U server w/ Chelsio N310E-CX card (via copper and active
Zarlink and Fujitsu fiboer optic links).
The issue w/ NICs and some other CX4 devices is that they
support only passive CX4 devices. That is they don't supply
the 3.3V the active devices require. So it is more than
they "have trouble"; they don't do it at all.
I don't think we have any Myricom cards.
Matt
On Thu, 5 Aug 2010, Tom Downes wrote:
Matt, John:
I would just as soon use 10-100Mbit, but the firmware we have
developed uses the 10 gbe port. I understand this to be fairly
fundamental in that the CASPER libraries allow simpler interfacing of
the memory to the 10 gbe than to the 100 Mbit. My recollection of the
details is fuzzy (and I'm not a firmware engineer) but I don't see
this aspect of the design as likely to change. So I think that we're
stuck going with N 10 gbe ports for N boards. We have unsolved
problems to solve instead.
I'll poke around for cheaper switches.
Matt: are you using the Zarlink CX4-optical-CX4 cables with the
Myricom card? That seems to contradict John's assertion that the
Myricom has trouble driving transceivers.
Tom
On Thu, Aug 5, 2010 at 3:41 PM, Matt Dexter <[email protected]> wrote:
In the past many switch vendors advertised performance they couldn't
actually provide in our applications.
Yes there are XFP to CX4 transceivers. I'm no expert in
such things but
we are talking to a number of vendors now about switches w/
48,64,..hundreds of ports and they talk about many such
adapter options. So far most like SFP+ or QSFP relative
to the long list of interface options.
The CASPER hardware limit is much less than the CX4 specification's
15m limit for copper wiring.
At ATA we have in use the Zarlink fiber optic cables w/ CX4
connectors for 5m, 10m and 15m applications.
See the equipment_cables page for more information.
Matt
On Thu, 5 Aug 2010, Tom Downes wrote:
No.
Believe it or not I spent a long time looking everywhere but the
Casper wiki. Should have thought of that because that's where we got
the Myricom card from.
Those listed are at the higher end of the price range I've seen. Looks
like ~$10k is the amount I should be prepared to spend.
Am I right to interpret the XFP ports as something I could convert CX4
to use with a transceiver?
Tom
On Thu, Aug 5, 2010 at 3:21 PM, Matt Dexter <[email protected]> wrote:
Hi Tom,
were you aware of these ?
http://casper.berkeley.edu/wiki/Recommended_10_GbE_Hardware
http://casper.berkeley.edu/wiki/Equipment_Cables
Matt Dexter
On Thu, 5 Aug 2010, Tom Downes wrote:
Casper-folks:
Hoping to short-circuit a fair amount of research here in the hope
that someone has had to do this already. I'll soon be looking to
connect 10-20 ROACH boards by 10 gbe to a data acquisition
computer(s).
It seems like the smartest way of doing that is getting a 16-port
switch or potentially two 8-port switches. But the 10 Gbe port on the
ROACH seems to be CX4 which I take to be a less popular connector
variety.
What kind of switches have ROACH users out there used to connect up a
bunch of boards? Are there switches out there to convert CX4 to
something with a reach longer than the 15m Wikipedia quotes as the
limit of CX4. 15m is very borderline for our needs.
The prices seem to vary widely. We do not need network admin tools or
anything fancy. In fact our data rates could probably go over 10Mb
cabling, but the 10Gbe interface of the ROACH is more convenient from
the firmware perspective. This is more of a multiplexer than a switch.
Tom
