[casper] Network Switches

2014-09-02 Thread John Ford
Hi all. Has anyone tested the Mellanox SX-1024 series of switches with ROACH-2 and 10 and/or 40 gb NICs? These switches have 48 10 Gbe ports and 12 40 Gbe ports on them. Thanks! John

Re: [casper] Network Switches

2014-09-02 Thread Dan Werthimer
hi john, jack hickish tested a mellanox SX1012 (12x40Gbe, or 48x10Gbe, or mixture), when he was visiting berkeley. the SX1012 worked beautifully on roach2, after jack upgraded the switch firmware. and it's a great price. ($6K) i think jason has also tested this switch. best wishes, dan

Re: [casper] Network Switches

2014-09-02 Thread John Ford
hi john, jack hickish tested a mellanox SX1012 (12x40Gbe, or 48x10Gbe, or mixture), when he was visiting berkeley. the SX1012 worked beautifully on roach2, after jack upgraded the switch firmware. and it's a great price. ($6K) Yeah, we're looking at a 1024, or a 1036. They all use the

Re: [casper] Network Switches

2014-09-02 Thread Jack Hickish
Hi John, As Dan said I've tested (to some extent) the SX1012. I just used one ROACH2 and corner turned data through 8 x 10GbE ports. It worked well basically up to line rate, with no CRC errors after a few hours of operation, but only after - Jason put me in touch with some Mellanox guys who

Re: [casper] Network Switches

2014-09-02 Thread Lincoln Greenhill
Hi John, Email is passing in the ether. I have eyed the SX1035 and SX1036 w/ upgrades in mind. What is your understanding of the difference? Lincoln On 9/2/14, 11:54 AM, John Ford wrote: hi john, jack hickish tested a mellanox SX1012 (12x40Gbe, or 48x10Gbe, or mixture), when he was

Re: [casper] Network Switches

2014-09-02 Thread Jason Manley
The SX1012 and SX1036 definitely use the same ASIC series and both suffer from the PHY link problem. I've got seven of them here, and tested them with 20 different ROACH-2 boards. As Jack points out, some links are worse than others and swapping cables and ROACH2 mezzanine cards often sorts the

Re: [casper] Network Switches

2014-09-02 Thread Dan Werthimer
hi jason, thanks for all this info. in the leda correlator, the 10/40Gbe links are unidirectional, roach2--switch and switch--gpuserver so i don't think leda would encounter the switch--roach2 bit errors, although it wouldn't hurt to upgrade the firmware. best wishes, dan On Tue, Sep

Re: [casper] Network Switches

2014-09-02 Thread Jack Hickish
Not to steer this conversation too far off track, but does a patch exist to enable the ethernet CRC check within the 10Gb yellow block? When I tested the switch I used Dave MacMahon's CRC generator block, but it would be nice if there was the option to have the 10 gig core use the UDP crc -- or

Re: [casper] Network Switches

2014-09-02 Thread John Ford
Hi John, Email is passing in the ether. I have eyed the SX1035 and SX1036 w/ upgrades in mind. What is your understanding of the difference? I can't see any, really. The sx1035 has slightly lower latency, and is way more expensive... I don't really understand what L2 Lite Switching is,