On 1/07/2012 8:35 AM, Sašo Kiselkov wrote:
> On 06/29/2012 05:28 PM, Darren Reed wrote:
>> On 30/06/2012 1:10 AM, Darren Reed wrote:
>>> This patch won't work for UDP if the udp ring count
>>> is greater than the TCP ring count. This is because the
>>> code in mac_rx_srs_fanout() limits the looping over
>>> the enqueued data structures to that of "fanout",
>>> which is only set once.
>>>
>>> The bug is the loop at the bottom of mac_rx_srs_fanout().
>>>
>>> There are a few things that could be done with it...
>> Of course I forgot to read this..
>>
>> * Since the softrings are never destroyed and we always
>> * create equal number of softrings for TCP, UDP and rest,
>> * its OK to check one of them for count and use it without
>> * any lock. In future, if soft rings get destroyed because
>> * of reduction in fanout, we will need to ensure that happens
>> * behind the SRS_PROC.
>>
>> ... so there's a separate ring count for tcp, udp and other,
>> yet all three counters are the same but used like they are
>> different when the code thinks that it is ok to be like that.
>>
>> Is one of joyent/nexenta/someone looking at this as part
>> of a (major) project?
> So I progressed with the issue and I can see that in the correct course
> of events, I see this call bad flow in the interrupt handler (when
> fanout is handled by multiple CPUs):
>
> CPU FUNCTION
> 0 -> mac_rx_srs_drain
> 0 -> mac_rx_srs_fanout
> 0 -> mac_rx_srs_long_fanout
> 0 <- mac_rx_srs_long_fanout
> 0 -> mac_rx_soft_ring_process
> 0 -> ip_input
> 0 -> ip_drop_input
> 0 | ip_drop_input:entry str: "ipIfStatsInHdrErrors"
> 0 <- ip_drop_input
> 0 <- ip_input
> 0 <- mac_rx_soft_ring_process
> 0 <- mac_rx_srs_fanout
> 0 <- mac_rx_srs_drain
>
> It would seem that once multi-CPU fanout is enabled, multicast packets
> get corrupted somewhere in the processing, which leads ip_input
> (actually an inlined ip_input_common_v4) to throw them away (exactly at
> ip_input.c:319).
In this case, it would appear that it isn't the IP header that is
corrupted but rather the mblk_t - it hasn't been prepared for
transmission into IP by setting b_rptr to the start of the IP
header. What's required is to adjust b_rptr.
In mac_sched.c, after each call to mac_rx_srs_long_fanout(),
you need to add a line like this:
mp->b_rptr += hdrsize;
immediately before the FANOUT_ENQUEUE_MP().
Cheers,
Darren
-------------------------------------------
illumos-discuss
Archives: https://www.listbox.com/member/archive/182180/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be
Modify Your Subscription:
https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4
Powered by Listbox: http://www.listbox.com
