> I'm talking with the engineers at Chelsio about just such a beast and > expect > a call back on Monday. I imagine I'll go with CX4 on both ends and Zarlink > for now, but it seems like something the ROACH community needs to think > about - sound like GMRT is doing something like this. > > Every time I've spoken with an engineer at Chelsio or Myricom about > CX4-SFP+ > (probably with fiber between), the initial reaction is something like Bugs > Bunny saying "Hansel". It's as if it were something they would never have > considered doing in a million years. I think the conventional wisdom is > not > just that these transceivers can impede speed, but can also impede the > ability of the link to remain up at all.
I got the same reaction. It's puzzling. I guess they weren't around for the original "waterhose" ethernet -> 10base2 -> 10baseT -> 10baseFX -> 100baseT -> 1000baseT progression. In other words, why the heck would you *not* have a heterogenous bunch of networks, transcievers, and media once the first working products were out for a few years? John > > Tom > > On Sat, Jan 29, 2011 at 9:57 AM, John Ford <[email protected]> wrote: > >> Can one use the zarlink (or something like it) on the ROACH end, and >> connect the fiber to an SFP+ module in the computer or switch? It seems >> like someone ought to make such a beast, considering there are a lot of >> cx-4 ports in the field that need to be connected to new CX-4 - only >> switches and NICs. >> >> This is, I'm afraid, the downside to throwing in your lot with >> commercial >> products. You're at the mercy of the markets. >> >> John >> >> >> > I am avoiding Myricom for the reasons Rick mentioned. It took a long >> time >> > for me to get the sales/technical person to even understand that I >> wanted >> > to >> > go from CX4 to fiber. >> > >> > But Chelsio, as several have mentioned on this list, provides the >> power >> > necessary for transceivers to work. They also have offloading cards >> (which >> > I >> > believe is what you're describing) - or at least they did until the >> > discontinued their CX4 line. Not sure what the new Chelsio product >> line >> > will >> > look like and I am somewhat dubious that they will stay on the 4-6 >> week >> > timeframe. Every vendor that I and a collaborator have called are out >> of >> > Chelsio CX4 stock. >> > >> > Intel makes 10gbe cards, but the list archives are ambiguous as to >> whether >> > they power the transceivers in the Zarlink cables. >> > >> > My primary concern is that if companies already see fit to discontinue >> CX4 >> > products, then (a) it is hard to connect to the ROACH now and (b) will >> be >> > nearly impossible when something breaks in 6 years. >> > >> > How far along are the GMRT folks? >> > >> > Tom >> > >> > On Fri, Jan 28, 2011 at 6:06 PM, rick raffanti <[email protected]> >> > wrote: >> > >> >> The Myricom people told me they don't make NICs with active ports- >> ie, >> >> aux >> >> power for the fiber translator. That's why we bought the Chelsio. >> >> Anton is >> >> getting 6Gb/s throughput with the Chelsio- we haven't tried to push >> it >> >> further. I wasn't aware of the UDP packet handling stuff, though. >> >> >> >> Rick >> >> >> >> >> >> On 1/28/2011 5:53 PM, Dan Werthimer wrote: >> >> >> >> >> >> hi tom, >> >> >> >> one more note: >> >> >> >> if you use fiber optic CX4 cables, >> >> please see the warning at >> >> >> >> http://casper.berkeley.edu/wiki/Recommended_10_GbE_Hardware >> >> >> >> not all NIC boards have built in power to support >> >> fiber optic cables. check with myricom. >> >> the ibob/bee2/roach boards have built in power. >> >> >> >> dan >> >> >> >> >> >> On 1/28/2011 2:41 PM, Tom Downes wrote: >> >> >> >> So Chelsio has end-of-lifed their CX4 line. They say "4-6 weeks" >> until >> >> new >> >> cards come out as part of a new product line, but their sales contact >> >> said >> >> this reflected a larger recognition that CX4 is not how the industry >> is >> >> going. >> >> >> >> My thought is that I should be buying an SFP+ card and figuring out >> a >> >> way >> >> to convert to CX4, e.g. SFP+->optical, optical->CX4. Our cable >> lengths >> >> that >> >> we will (eventually) need are all greater than 15m, so outside of the >> >> CX4 >> >> spec, much less what the ROACH boards are apparently cable of >> driving. >> >> >> >> Is such a transceiver scheme plausible? I am having trouble finding >> the >> >> appropriate parts. >> >> >> >> Tom >> >> >> >> On Mon, Aug 9, 2010 at 8:11 AM, Matt Dexter <[email protected]> >> >> wrote: >> >> >> >>> Yes - when pricing switches, or any sort of (sub-)system, a full >> >>> BOM must be used to make a meaningful comparison. >> >>> >> >>> Matt >> >>> >> >>> >> >>> On Mon, 9 Aug 2010, John Ford wrote: >> >>> >> >>> And don't forget that the switches that are XFP and SFP+ sometimes >> >>>> (usually?) don't include the optics for each port in the switch >> price. >> >>>> >> >>>> With CX4, all you need is a cable, if you're within a few meters. >> >>>> >> >>>> Yes - that list is years old. >> >>>>> Those Fujitsu and HP switches have been tested with the CASPER >> >>>>> hardware >> >>>>> and found to work as advertised. >> >>>>> >> >>>>> There are lots of new products available. More announced >> >>>>> all the time. We are in contact with a number of vendors in >> >>>>> hopes of getting demo units to try in house with the CASPER >> >>>>> hardware before listing them as recommended for use. >> >>>>> Our tests will include running at full line rates all ports >> >>>>> continuously >> >>>>> as that's what our intended applications require. >> >>>>> >> >>>>> Less demanding applications will have many more, and >> >>>>> cheaper, options for suitable switch vendor and model. >> >>>>> >> >>>>> I have no prediction for when I will be able to add more switch >> >>>>> models will to that list. >> >>>>> >> >>>>> Matt >> >>>>> >> >>>>> On Thu, 5 Aug 2010, Andrew Lutomirski wrote: >> >>>>> >> >>>>> On Thu, Aug 5, 2010 at 3:21 PM, Matt Dexter >> <[email protected]> >> >>>>>> wrote: >> >>>>>> >> >>>>>>> Hi Tom, >> >>>>>>> >> >>>>>>> were you aware of these ? >> >>>>>>> http://casper.berkeley.edu/wiki/Recommended_10_GbE_Hardware >> >>>>>>> >> >>>>>> >> >>>>>> Sadly the list is out of date: some of the switches are no longer >> in >> >>>>>> production. The XG700, for example, is great and cheap but you >> >>>>>> can't >> >>>>>> buy one without great difficulty. >> >>>>>> >> >>>>>> I'm not sure that manufacturers really care about CX4 anymore now >> >>>>>> that >> >>>>>> SFP+ parts are available. >> >>>>>> >> >>>>>> --Andy >> >>>>>> >> >>>>>> http://casper.berkeley.edu/wiki/Equipment_Cables >> >>>>>>> Matt Dexter >> >>>>>>> >> >>>>>>> On Thu, 5 Aug 2010, Tom Downes wrote: >> >>>>>>> >> >>>>>>> Casper-folks: >> >>>>>>>> >> >>>>>>>> Hoping to short-circuit a fair amount of research here in the >> hope >> >>>>>>>> that someone has had to do this already. I'll soon be looking >> to >> >>>>>>>> connect 10-20 ROACH boards by 10 gbe to a data acquisition >> >>>>>>>> computer(s). >> >>>>>>>> >> >>>>>>>> It seems like the smartest way of doing that is getting a >> 16-port >> >>>>>>>> switch or potentially two 8-port switches. But the 10 Gbe port >> on >> >>>>>>>> the >> >>>>>>>> ROACH seems to be CX4 which I take to be a less popular >> connector >> >>>>>>>> variety. >> >>>>>>>> >> >>>>>>>> What kind of switches have ROACH users out there used to >> connect >> >>>>>>>> up a >> >>>>>>>> bunch of boards? Are there switches out there to convert CX4 to >> >>>>>>>> something with a reach longer than the 15m Wikipedia quotes as >> the >> >>>>>>>> limit of CX4. 15m is very borderline for our needs. >> >>>>>>>> >> >>>>>>>> The prices seem to vary widely. We do not need network admin >> tools >> >>>>>>>> or >> >>>>>>>> anything fancy. In fact our data rates could probably go over >> 10Mb >> >>>>>>>> cabling, but the 10Gbe interface of the ROACH is more >> convenient >> >>>>>>>> from >> >>>>>>>> the firmware perspective. This is more of a multiplexer than a >> >>>>>>>> switch. >> >>>>>>>> >> >>>>>>>> Tom >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>>> >> >>>> >> >>>> >> >>> >> >> >> >> >> > >> >> >> >
