hi tom,
i suggest you purchase a myricom 10Gbe CX4 NIC board,
rather than a chelsio board. the myricom boards work
well and jason wrote a casper memo with test results on myricom nics.
myricom has drivers optimized for UDP packets,
and drivers that skip over kernel memory and transfer data
directly to application memory.
(most drivers move data from
NIC board --> kernel_memory --> application memory).
you can purchase inexpensive fiber cables that have
CX4 connectors on each end, and then you can use
cables of 100 meters or more. see
http://casper.berkeley.edu/wiki/Equipment_Cables
on the other hand, SFP+ is much nicer than CX4, and
all the big switches use SFP+, and the new Roach II boards
will support SFP+
if you want to use SFP+:
1)
it's very expensive to use a media converter to convert SFP+ to CX4.
(there are three vendors - each 10Gbit channel costs $1K or more).
2)
GMRT is working on an inexpensive add on to roach
that will convert it's CX4 connectors to SPF+.
3)
Roach II will support CX4 and SFP+, but these boards
aren't available yet - south africa is testing them now.
best wishes,
dan
On 1/28/2011 2:41 PM, Tom Downes wrote:
So Chelsio has end-of-lifed their CX4 line. They say "4-6 weeks" until
new cards come out as part of a new product line, but their sales
contact said this reflected a larger recognition that CX4 is not how
the industry is going.
My thought is that I should be buying an SFP+ card and figuring out a
way to convert to CX4, e.g. SFP+->optical, optical->CX4. Our cable
lengths that we will (eventually) need are all greater than 15m, so
outside of the CX4 spec, much less what the ROACH boards are
apparently cable of driving.
Is such a transceiver scheme plausible? I am having trouble finding
the appropriate parts.
Tom
On Mon, Aug 9, 2010 at 8:11 AM, Matt Dexter <[email protected]
<mailto:[email protected]>> wrote:
Yes - when pricing switches, or any sort of (sub-)system, a full
BOM must be used to make a meaningful comparison.
Matt
On Mon, 9 Aug 2010, John Ford wrote:
And don't forget that the switches that are XFP and SFP+ sometimes
(usually?) don't include the optics for each port in the
switch price.
With CX4, all you need is a cable, if you're within a few meters.
Yes - that list is years old.
Those Fujitsu and HP switches have been tested with the
CASPER hardware
and found to work as advertised.
There are lots of new products available. More announced
all the time. We are in contact with a number of vendors in
hopes of getting demo units to try in house with the CASPER
hardware before listing them as recommended for use.
Our tests will include running at full line rates all
ports continuously
as that's what our intended applications require.
Less demanding applications will have many more, and
cheaper, options for suitable switch vendor and model.
I have no prediction for when I will be able to add more
switch
models will to that list.
Matt
On Thu, 5 Aug 2010, Andrew Lutomirski wrote:
On Thu, Aug 5, 2010 at 3:21 PM, Matt Dexter
<[email protected] <mailto:[email protected]>>
wrote:
Hi Tom,
were you aware of these ?
http://casper.berkeley.edu/wiki/Recommended_10_GbE_Hardware
Sadly the list is out of date: some of the switches
are no longer in
production. The XG700, for example, is great and
cheap but you can't
buy one without great difficulty.
I'm not sure that manufacturers really care about CX4
anymore now that
SFP+ parts are available.
--Andy
http://casper.berkeley.edu/wiki/Equipment_Cables
Matt Dexter
On Thu, 5 Aug 2010, Tom Downes wrote:
Casper-folks:
Hoping to short-circuit a fair amount of
research here in the hope
that someone has had to do this already. I'll
soon be looking to
connect 10-20 ROACH boards by 10 gbe to a data
acquisition
computer(s).
It seems like the smartest way of doing that
is getting a 16-port
switch or potentially two 8-port switches. But
the 10 Gbe port on the
ROACH seems to be CX4 which I take to be a
less popular connector
variety.
What kind of switches have ROACH users out
there used to connect up a
bunch of boards? Are there switches out there
to convert CX4 to
something with a reach longer than the 15m
Wikipedia quotes as the
limit of CX4. 15m is very borderline for our
needs.
The prices seem to vary widely. We do not need
network admin tools or
anything fancy. In fact our data rates could
probably go over 10Mb
cabling, but the 10Gbe interface of the ROACH
is more convenient from
the firmware perspective. This is more of a
multiplexer than a switch.
Tom
