I'm talking with the engineers at Chelsio about just such a beast and expect a call back on Monday. I imagine I'll go with CX4 on both ends and Zarlink for now, but it seems like something the ROACH community needs to think about - sound like GMRT is doing something like this.
Every time I've spoken with an engineer at Chelsio or Myricom about CX4-SFP+ (probably with fiber between), the initial reaction is something like Bugs Bunny saying "Hansel". It's as if it were something they would never have considered doing in a million years. I think the conventional wisdom is not just that these transceivers can impede speed, but can also impede the ability of the link to remain up at all. Tom On Sat, Jan 29, 2011 at 9:57 AM, John Ford <[email protected]> wrote: > Can one use the zarlink (or something like it) on the ROACH end, and > connect the fiber to an SFP+ module in the computer or switch? It seems > like someone ought to make such a beast, considering there are a lot of > cx-4 ports in the field that need to be connected to new CX-4 - only > switches and NICs. > > This is, I'm afraid, the downside to throwing in your lot with commercial > products. You're at the mercy of the markets. > > John > > > > I am avoiding Myricom for the reasons Rick mentioned. It took a long time > > for me to get the sales/technical person to even understand that I wanted > > to > > go from CX4 to fiber. > > > > But Chelsio, as several have mentioned on this list, provides the power > > necessary for transceivers to work. They also have offloading cards > (which > > I > > believe is what you're describing) - or at least they did until the > > discontinued their CX4 line. Not sure what the new Chelsio product line > > will > > look like and I am somewhat dubious that they will stay on the 4-6 week > > timeframe. Every vendor that I and a collaborator have called are out of > > Chelsio CX4 stock. > > > > Intel makes 10gbe cards, but the list archives are ambiguous as to > whether > > they power the transceivers in the Zarlink cables. > > > > My primary concern is that if companies already see fit to discontinue > CX4 > > products, then (a) it is hard to connect to the ROACH now and (b) will be > > nearly impossible when something breaks in 6 years. > > > > How far along are the GMRT folks? > > > > Tom > > > > On Fri, Jan 28, 2011 at 6:06 PM, rick raffanti <[email protected]> > > wrote: > > > >> The Myricom people told me they don't make NICs with active ports- ie, > >> aux > >> power for the fiber translator. That's why we bought the Chelsio. > >> Anton is > >> getting 6Gb/s throughput with the Chelsio- we haven't tried to push it > >> further. I wasn't aware of the UDP packet handling stuff, though. > >> > >> Rick > >> > >> > >> On 1/28/2011 5:53 PM, Dan Werthimer wrote: > >> > >> > >> hi tom, > >> > >> one more note: > >> > >> if you use fiber optic CX4 cables, > >> please see the warning at > >> > >> http://casper.berkeley.edu/wiki/Recommended_10_GbE_Hardware > >> > >> not all NIC boards have built in power to support > >> fiber optic cables. check with myricom. > >> the ibob/bee2/roach boards have built in power. > >> > >> dan > >> > >> > >> On 1/28/2011 2:41 PM, Tom Downes wrote: > >> > >> So Chelsio has end-of-lifed their CX4 line. They say "4-6 weeks" until > >> new > >> cards come out as part of a new product line, but their sales contact > >> said > >> this reflected a larger recognition that CX4 is not how the industry is > >> going. > >> > >> My thought is that I should be buying an SFP+ card and figuring out a > >> way > >> to convert to CX4, e.g. SFP+->optical, optical->CX4. Our cable lengths > >> that > >> we will (eventually) need are all greater than 15m, so outside of the > >> CX4 > >> spec, much less what the ROACH boards are apparently cable of driving. > >> > >> Is such a transceiver scheme plausible? I am having trouble finding the > >> appropriate parts. > >> > >> Tom > >> > >> On Mon, Aug 9, 2010 at 8:11 AM, Matt Dexter <[email protected]> > >> wrote: > >> > >>> Yes - when pricing switches, or any sort of (sub-)system, a full > >>> BOM must be used to make a meaningful comparison. > >>> > >>> Matt > >>> > >>> > >>> On Mon, 9 Aug 2010, John Ford wrote: > >>> > >>> And don't forget that the switches that are XFP and SFP+ sometimes > >>>> (usually?) don't include the optics for each port in the switch price. > >>>> > >>>> With CX4, all you need is a cable, if you're within a few meters. > >>>> > >>>> Yes - that list is years old. > >>>>> Those Fujitsu and HP switches have been tested with the CASPER > >>>>> hardware > >>>>> and found to work as advertised. > >>>>> > >>>>> There are lots of new products available. More announced > >>>>> all the time. We are in contact with a number of vendors in > >>>>> hopes of getting demo units to try in house with the CASPER > >>>>> hardware before listing them as recommended for use. > >>>>> Our tests will include running at full line rates all ports > >>>>> continuously > >>>>> as that's what our intended applications require. > >>>>> > >>>>> Less demanding applications will have many more, and > >>>>> cheaper, options for suitable switch vendor and model. > >>>>> > >>>>> I have no prediction for when I will be able to add more switch > >>>>> models will to that list. > >>>>> > >>>>> Matt > >>>>> > >>>>> On Thu, 5 Aug 2010, Andrew Lutomirski wrote: > >>>>> > >>>>> On Thu, Aug 5, 2010 at 3:21 PM, Matt Dexter <[email protected]> > >>>>>> wrote: > >>>>>> > >>>>>>> Hi Tom, > >>>>>>> > >>>>>>> were you aware of these ? > >>>>>>> http://casper.berkeley.edu/wiki/Recommended_10_GbE_Hardware > >>>>>>> > >>>>>> > >>>>>> Sadly the list is out of date: some of the switches are no longer in > >>>>>> production. The XG700, for example, is great and cheap but you > >>>>>> can't > >>>>>> buy one without great difficulty. > >>>>>> > >>>>>> I'm not sure that manufacturers really care about CX4 anymore now > >>>>>> that > >>>>>> SFP+ parts are available. > >>>>>> > >>>>>> --Andy > >>>>>> > >>>>>> http://casper.berkeley.edu/wiki/Equipment_Cables > >>>>>>> Matt Dexter > >>>>>>> > >>>>>>> On Thu, 5 Aug 2010, Tom Downes wrote: > >>>>>>> > >>>>>>> Casper-folks: > >>>>>>>> > >>>>>>>> Hoping to short-circuit a fair amount of research here in the hope > >>>>>>>> that someone has had to do this already. I'll soon be looking to > >>>>>>>> connect 10-20 ROACH boards by 10 gbe to a data acquisition > >>>>>>>> computer(s). > >>>>>>>> > >>>>>>>> It seems like the smartest way of doing that is getting a 16-port > >>>>>>>> switch or potentially two 8-port switches. But the 10 Gbe port on > >>>>>>>> the > >>>>>>>> ROACH seems to be CX4 which I take to be a less popular connector > >>>>>>>> variety. > >>>>>>>> > >>>>>>>> What kind of switches have ROACH users out there used to connect > >>>>>>>> up a > >>>>>>>> bunch of boards? Are there switches out there to convert CX4 to > >>>>>>>> something with a reach longer than the 15m Wikipedia quotes as the > >>>>>>>> limit of CX4. 15m is very borderline for our needs. > >>>>>>>> > >>>>>>>> The prices seem to vary widely. We do not need network admin tools > >>>>>>>> or > >>>>>>>> anything fancy. In fact our data rates could probably go over 10Mb > >>>>>>>> cabling, but the 10Gbe interface of the ROACH is more convenient > >>>>>>>> from > >>>>>>>> the firmware perspective. This is more of a multiplexer than a > >>>>>>>> switch. > >>>>>>>> > >>>>>>>> Tom > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>>> > >>>> > >>> > >> > >> > > > > >
