Out of curiosity, is your IP multicast traffic coming from a single source? If
so, there will be almost no way to have it hit any processor other than the
first. I'm pretty sure that the fan out distributor relies on having different
source addresses in order to avoid breaking ordering of packet delivery. That
said, forcing the cpus list should not have *broken* traffic.
- Garrett
Garrett D'Amore
[email protected]
On Jun 29, 2012, at 4:25 PM, Sašo Kiselkov wrote:
> On 06/29/2012 12:24 AM, Dan McDonald wrote:
>> I thought 918 wouldn't be the issue here as there is no VLAN involved.
>>
>> ANYWAY, I do have a just revised today 918 and its webrev (because one of
>> our Nexenta customers was bitten by 918) here:
>>
>> http://kebe.com/~danmcd/webrevs/918/
>>
>> You should be able to drop these diffs into your illumos-gate clone, and
>> "build small" just the mac module itself. I blogged about this:
>>
>>
>> http://kebesays.blogspot.com/2011/03/for-illumos-newbies-on-developing-small.html
>>
>> I'm not sure if 918 will cure what ails you, but you should try my fix and
>> see if it does.
>
> Thanks for the many pointers. I tried and built a new mac module.
> Installed it and here are my observations:
>
> - without the patch for 918 (just plain illumos-gate from mercurial),
> as expected, I'm seeing the issue as before
>
> - with the patch, something indeed happened, but not very good. Now I'm
> not getting any multicast packets to userspace, but I can see them
> arriving on the interface via snoop.
>
> I tried disabling the dladm set-linkprop cpus=0-31 tuning and that
> restored functionality, but since by default fanout is only to a single
> CPU, I'm still hitting the old single-core fanout bottleneck. If I set
> cpus to anything that causes the fanout cpu list to include more than
> one CPU, I can't get any multicast packet to propagate to userspace.
>
> Example:
>
> ** Multicast OK **
> dladm set-linkprop -p cpus=0-2 igb3
> # echo '::mac_srs -rcv' | mdb -k
> CPU_COUNT FANOUT_CPU_COUNT
> ADDR LINK_NAME (CPU_LIST) (CPU_LIST)
> ffffff2203800340 igb3 3 1
> (00,01,02) (01)
> ffffff22037ff680 igb3 3 1
> (00,01,02) (01)
>
>
> ** Multicast BROKEN **
> dladm set-linkprop -p cpus=0-2 igb3
> # echo '::mac_srs -rcv' | mdb -k
> CPU_COUNT FANOUT_CPU_COUNT
> ADDR LINK_NAME (CPU_LIST) (CPU_LIST)
> ffffff2203800340 igb3 4 2
> (00,01,02,03) (01,02)
> ffffff22037ff680 igb3 4 2
> (00,01,02,03) (01,02)
>
> Any ideas what might be causing this? I've commented out all tunings in
> /etc/system to make sure I'm not hitting some weird behavior...
>
> --
> Saso
>
>
> -------------------------------------------
> illumos-discuss
> Archives: https://www.listbox.com/member/archive/182180/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/182180/22003744-45f01c1f
> Modify Your Subscription: https://www.listbox.com/member/?&;
> Powered by Listbox: http://www.listbox.com
-------------------------------------------
illumos-discuss
Archives: https://www.listbox.com/member/archive/182180/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be
Modify Your Subscription:
https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4
Powered by Listbox: http://www.listbox.com
