Roch,
Rajagopal Kunhappan writes:
>
>
> Roch - PAE wrote:
> > Sebastien Roy writes:
> > > Rajagopal Kunhappan wrote:
> > > > The tuning that you have done is rather weird. By default,
> > > > ip_squeues_per_cpu has a value of 1. But that does not mean that only
> > > > one squeue will be there for a CPU. More squeues upto a maximum of 32
> > > > can get allocated for a CPU and they get allocated on demand. So setting
> > > > ip_squeues_per_cpu to 32 is not very interesting.
> > > >
> > > > What is interesting is that you have set ip_soft_rings_cnt to 2 times
> > > > ip_squeues_per_cpu which would be 64 in your case. This does not seem
> > > > right.
> > > >
> > > > If you have a 64 CPU system and want to use all of them to process
> > > > incoming packets for a NIC, then it may just be ok, but having more soft
> > > > rings than the number of CPUs does not give you any performance advantage.
> > > >
> > > > My suggestion is to not change ip_squeues_per_cpu at all but tune
> > > > ip_soft_rings_cnt only. Set it to 2 or 3 for 1Gbps NIC and anywhere from
> > > > 16 to 32 for 10 Gbps NIC. Again don't increase the soft ring count to
> > > > more than the number of CPUs in the system. Also note that CPU speed is
> > > > also important in the calculation. A simple rule of thumb is that a 1Ghz
> > > > CPU should be able to handle a 1Gbps NIC.
> > >
> > > Could all of this manual tuning be made obsolete by some simple
> > > heuristics in the code which could automatically set these things? The
> > > number of CPUs on the system is no secret to the kernel. Will any of
> > > the ongoing projects (such as Crossbow or PEF) make this situation better?
> > >
> >
> > For now, I don't understand why fanout is not happening. The
> > connections are not established in one swoop and so the flow
> > should spread to multiple squeues (ip_squeues_fanout=1). Why
> > is this not happening ?
> >
>
> ip_squeue_fanout knob was not set initially and it is not a Niagara
> system (so fanout is not default). This is my reading.
>
> -krgopi
>
So you agree that setting this flag should be sufficient to
spread the load. Shouldn't that flag default to on ?
What is this? Have you trapped me? :-).
The answer depends on CPU speed, NIC speed and other things like extra
processing due to ipfilter and so on. On x86 boxes with greater than
2Ghz CPUs, a single CPU is more than enough for 1Gbps. Turning on fanout
in this case introduces unnecessary latency due to context switch to
worker thread mode of processing. On Niagara boxes, we need fanout for
sure to get the desired throughput. So it all depends and doing this
automatically taking the various things into consideration is the way to go.
-krgopi
-r
> >
> > -r
> >
> >
> > > -Seb
> > > _______________________________________________
> > > networking-discuss mailing list
> > > [email protected]
> >
> > _______________________________________________
> > networking-discuss mailing list
> > [email protected]
> >
> _______________________________________________
> networking-discuss mailing list
> [email protected]
_______________________________________________
networking-discuss mailing list
[email protected]
