Okay, I don't mind waiting at this point. The thought at one point was
that I should retest performance with crossbow gate to see if the
numbers were any better... but clearly crossbow-gate isn't ready for
that yet. :-)
This will hamper my ability to do meaningful performance related work
from crossbow, btw.
-- Garrett
Rajagopal Kunhappan wrote:
> Sunay Tripathi wrote:
>> Garrett D'Amore wrote:
>>> I'm testing e1000g on a T2000. Default daily build 04/30. (Built
>>> against snv-61).
>>>
>>> I'm seeing really, really dismal IP forwarding performance with
>>> 64-byte frames. Like ~5.8% line utilization (it seems to top out at
>>> around 90,000 packets per second default.) This is a default
>>> configuration (no tunables set whatsover), and yes, it turns out
>>> that interrupts for my two NICs are interrupting on CPUs 6 and 7
>>> (and yes, I know that this is a bad idea, but still, it should not
>>> show such terrible, dismal performance out of the box.) This
>>> performance is about half of the default S10 performance.
>>>
>>> I've not started using er_kernel yet to profile and find out what is
>>> happening. At first I thought the problem was my test setup, but
>>> when I lowered the test min thresholds from 10% to 1%, I get
>>> successful results.
>>>
>>> I'm going to try assigning additional cpus to create and setup some
>>> soft rings, to see how that helps.
>>>
>>> Anyone have any ideas up front about what is wrong?
>>>
>>> This default performance is, IMO, unacceptable.
>>
>> Yes, you are using code which is not complete. The soft rings have moved
>> to mac layer (the tuning auto tuning on Niagara is disabled in
>> crossbow-gate due to a bug I can't remember right now). Raj/Gopi, can
>> you help Garrett.
>
> Yes, ip_squeue_soft_ring tunable has been disabled in the
> crossbow-gate (See 6521633). You can still create soft rings and that
> too on the CPUs you want using dladm command.
>
> "dladm modify-dev -d <e1000g0> -c 0,1,2" will create soft rings on the
> fly for e1000g0. The worker thread would be bound to CPUs 0, 1 and 2.
>
> I also want to note that major changes are going on in this area of
> the code at the moment and things would become clearer by the end of
> this month atleast.
>
> Thanks,
> -krgopi
>
> -krgopi
>>
>> Cheers,
>> Sunay
>>
>>>
>>> -- Garrett
>>>
>>>
>>> _______________________________________________
>>> crossbow-discuss mailing list
>>> crossbow-discuss at opensolaris.org
>>> http://opensolaris.org/mailman/listinfo/crossbow-discuss
>>
>>
