Re: [casper] Multicast on 10 gbe on ROACH-2?

[Marc Welz]

David MacMhaon wrote:

>> I think multicast transmit is easy (just populate the ARP table
>> appropriately).  I think multicast receive is also supported with recent
>> versions of the 10GbE yellow block.  You could probably check the
>> mlib_devel git commit logs for the yellow block code to find clues.

Not sure if the arp table is involved in this - the destination mac is
(should be)
generated algorithmically from the destination multicast IP address, though
the above might be a unusual workaround.

Sending is reasonably straighforward, as there is no control traffic involved.
Reception does require a newer romfs/tcpborphserver where we get the kernel
to issue IGMP requests on the 10Gbe interfaces, so that the traffic arrives
at the particular port.

regards

marc

Re: [casper] Multicast on 10 gbe on ROACH-2?

[Wesley New]

Marc is correct. The ARP table is not involved in multicast. The multicast
MAC is just mapped from the multicast IP.

Wesley New
South African SKA Project
+2721 506 7300
www.ska.ac.za



On Thu, Nov 5, 2015 at 12:51 PM, Marc Welz  wrote:

> David MacMhaon wrote:
>
> >> I think multicast transmit is easy (just populate the ARP table
> >> appropriately).  I think multicast receive is also supported with recent
> >> versions of the 10GbE yellow block.  You could probably check the
> >> mlib_devel git commit logs for the yellow block code to find clues.
>
> Not sure if the arp table is involved in this - the destination mac is
> (should be)
> generated algorithmically from the destination multicast IP address, though
> the above might be a unusual workaround.
>
> Sending is reasonably straighforward, as there is no control traffic
> involved.
> Reception does require a newer romfs/tcpborphserver where we get the kernel
> to issue IGMP requests on the 10Gbe interfaces, so that the traffic arrives
> at the particular port.
>
> regards
>
> marc
>
>

Re: [casper] Multicast on 10 gbe on ROACH-2?

[John Ford]

Thanks, All!

John

> Marc is correct. The ARP table is not involved in multicast. The multicast
> MAC is just mapped from the multicast IP.
>
> Wesley New
> South African SKA Project
> +2721 506 7300
> www.ska.ac.za
>
>
>
> On Thu, Nov 5, 2015 at 12:51 PM, Marc Welz  wrote:
>
>> David MacMhaon wrote:
>>
>> >> I think multicast transmit is easy (just populate the ARP table
>> >> appropriately).  I think multicast receive is also supported with
>> recent
>> >> versions of the 10GbE yellow block.  You could probably check the
>> >> mlib_devel git commit logs for the yellow block code to find clues.
>>
>> Not sure if the arp table is involved in this - the destination mac is
>> (should be)
>> generated algorithmically from the destination multicast IP address,
>> though
>> the above might be a unusual workaround.
>>
>> Sending is reasonably straighforward, as there is no control traffic
>> involved.
>> Reception does require a newer romfs/tcpborphserver where we get the
>> kernel
>> to issue IGMP requests on the 10Gbe interfaces, so that the traffic
>> arrives
>> at the particular port.
>>
>> regards
>>
>> marc
>>
>>
>

Re: [casper] Multicast on 10 gbe on ROACH-2?

[David MacMahon]

As I think Jack commented, I was remembering an old trick that predated 
multicast support in the core.




> On Nov 5, 2015, at 02:51, Marc Welz  wrote:
> 
> Not sure if the arp table is involved in this - the destination mac is
> (should be)
> generated algorithmically from the destination multicast IP address, though
> the above might be a unusual workaround.

[casper] Multicast on 10 gbe on ROACH-2?

[John Ford]

Hi all.

Does the current ROACH-2 10 gbe yellow block work for multicast transmit?

Does anyone have an example of use of it, if so?

Thanks!

John

Re: [casper] Multicast on 10 gbe on ROACH-2?

[Jack Hickish]

Hi John,

It works great! Transmit only is happily also the simplest use case.
If you're doing stuff from the powerPC which you want multicasting, I
believe you have to
invoke corr.katcp_wrapper.FpgaClient.tap_multicast_add_send(). If you only
want to multicast traffic from the FPGA (which is how I've used
multicasting) all you need to do is set the destination IP to have a
multicast address (eg. I use 224.0.2.xxx). The yellow block will then
generate the appropriate destination MAC address for multicasting -- the
rest is up to the switch.*

An aside, in case it's useful: in the design where I'm using multicast I
have disabled the CPU tx/rx capabilities, so I can't use the multicast
subscribe methods provided by the corr package. Unfortunately, I also
wanted to be able to subscribe ROACHs to the multicast streams (standard FX
architecture, with F and X engines present on each board) so I set up
static multicast routes on the Mellanox SX1012 I'm using, and manually
wrote the relevant bits of the 10GbE core so that it accepts packets with
the addresses I need. It was pleasantly simple, and seems to work nicely.
I've set all my traffic to be multicast, even though most of it only has a
single destination -- I can subscribe a linux box to a few channels for
debugging or to do things like FFT zoom mode -- but for what it's worth
with 40 x ~9GbE connections going through the SX1012, it seems just as
reliable as with unicast.

If you want I can probably throw together an example design, but I'm pretty
sure for tx only it really is as simple as setting the appropriate address.
The more involved bit is turning on the relevant switch features and
writing some software to subscribe a client at the other end. I can dig out
the incantations I used with my mellanox switch, but since I did everything
with static routes I can't be much help with dynamically subscribing
software. (But it's ONLY software -- how hard can it be?!).


Cheers,
Jack

*Just read Dave's email -- I don't think you even need to set up the ARP
table with the latest yellow block, but this is probably a route to crowbar
multicast support into an old compile.

On Wed, 4 Nov 2015 at 16:58 John Ford  wrote:

> Hi all.
>
> Does the current ROACH-2 10 gbe yellow block work for multicast transmit?
>
> Does anyone have an example of use of it, if so?
>
> Thanks!
>
> John
>
>
>
>

Re: [casper] Multicast on 10 gbe on ROACH-2?

[John Ford]

> Hi, John.
>
> I think multicast transmit is easy (just populate the ARP table
> appropriately).  I think multicast receive is also supported with recent
> versions of the 10GbE yellow block.  You could probably check the
> mlib_devel git commit logs for the yellow block code to find clues.
>
> Of course, all that comes with the caveat that I've never actually used
> multicast with the 10 GbE yellow blocks.

Thanks, Dave.  We'll give it a shot!

John

>
> HTH,
> Dave
>
>
>> On Nov 4, 2015, at 08:57, John Ford  wrote:
>>
>> Hi all.
>>
>> Does the current ROACH-2 10 gbe yellow block work for multicast
>> transmit?
>>
>> Does anyone have an example of use of it, if so?
>>
>> Thanks!
>>
>> John
>>
>>
>>
>