Re: [RFC] Re: Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi, On 07.12.2016 13:31, Pavel Machek wrote: On Fri 2016-12-02 15:05:21, Lino Sanfilippo wrote: Hi, There's nothing that protect stmmac_poll() from running concurently with stmmac_dma_interrupt(), right? could it be that there is also another issue concerned locking?: The tx completion

[RFC] Re: Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

On Fri 2016-12-02 15:05:21, Lino Sanfilippo wrote: > Hi, > > > > > > There's nothing that protect stmmac_poll() from running concurently > > with stmmac_dma_interrupt(), right? > > > > could it be that there is also another issue concerned locking?: > The tx completion handler takes the

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

> > You mean stmmac_xmit()? Thats also softirq AFAICT, its the TX softirq > > Regards, > Lino > > Hmm. netdevices.txt says: ndo_start_xmit: ... Context: Process with BHs disabled or BH (timer), will be called with interrupts disabled by netconsole.

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

On 05.12.2016 23:40, Pavel Machek wrote: > On Mon 2016-12-05 23:37:09, Lino Sanfilippo wrote: >> Hi Pavel, >> >> On 05.12.2016 23:02, Pavel Machek wrote: >> > >> > we need spin_lock_bh at minimum, as we are locking user context >> > against timer. >> > >> > Best regards, >> >

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

On Mon 2016-12-05 23:37:09, Lino Sanfilippo wrote: > Hi Pavel, > > On 05.12.2016 23:02, Pavel Machek wrote: > > > > we need spin_lock_bh at minimum, as we are locking user context > > against timer. > > > > Best regards, > >

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi Pavel, On 05.12.2016 23:02, Pavel Machek wrote: > > we need spin_lock_bh at minimum, as we are locking user context > against timer. > > Best regards, > Pavel > I was referring to stmmac_tx_clean() which AFAICS is only

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! > > > > Actually, I was wrong. irqlock protection is needed, since > > stmmac_tx_clean() is called from timer, and that's interrupt context, > > as you can confirm using BUG_ON(in_interrupt()); > > > > in_interrupt() can mean both softirq and hardirq context. In this case it > means

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! > >Sorry but I'm a little bit confused. I'm dropped in some mails without > >historic. I see cleanup, coalescence issue and TSO question. > >What is your main issue? Are you working on gmac4 or 3.x ? > >Can you refresh a little bit the story please ? > > let me try to do a sum, please Pavel

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! > >>We had experimented this tuning on STB IP where just datagrams > >>had to send externally. To be honest, although we had seen > >>better results w/o any timer, we kept this approach enabled > >>because the timer was fast enough to cover our tests on SH4 boxes. > > > >Please reply to

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! > the idea behind the TX mitigation is to mix the interrupt and > timer and this approach gave us real benefit in terms > of performances and CPU usage (especially on SH4-200/SH4-300 platforms > based). > In the ring, some descriptors can raise the irq (according to a > threshold) and set the

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi, > > Actually, I was wrong. irqlock protection is needed, since > stmmac_tx_clean() is called from timer, and that's interrupt context, > as you can confirm using BUG_ON(in_interrupt()); > in_interrupt() can mean both softirq and hardirq context. In this case it means softirq. So I guess

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! > >>In the ring, some descriptors can raise the irq (according to a > >>threshold) and set the IC bit. In this path, the NAPI poll will be > >>scheduled. > > > >Not NAPI poll but stmmac_tx_timer(), right? > > in the xmit according the the threshold the timer is started or the > interrupt is

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi Alex On 12/2/2016 3:26 PM, Alexandre Torgue wrote: 4.4 has no GMAC4 support. Alex, do you remember any patches to fix that? No sorry Peppe. Pavel, Sorry but I'm a little bit confused. I'm dropped in some mails without historic. I see cleanup, coalescence issue and TSO question. What is

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi Pavel and Peppe, On 12/02/2016 02:51 PM, Giuseppe CAVALLARO wrote: On 12/2/2016 1:32 PM, Pavel Machek wrote: Hi! Well, if you have a workload that sends and receive packets, it tends to work ok, as you do tx_clean() in stmmac_poll(). My workload is not like that -- it is "sending packets

Aw: Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi, > > There's nothing that protect stmmac_poll() from running concurently > with stmmac_dma_interrupt(), right? > could it be that there is also another issue concerned locking?: The tx completion handler takes the xmit_lock in case that the netif_queue is stopped. This is AFAICS

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

On 12/2/2016 1:32 PM, Pavel Machek wrote: Hi! Well, if you have a workload that sends and receive packets, it tends to work ok, as you do tx_clean() in stmmac_poll(). My workload is not like that -- it is "sending packets at 3MB/sec, receiving none". So the stmmac_tx_timer() is rescheduled and

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! > >Well, if you have a workload that sends and receive packets, it tends > >to work ok, as you do tx_clean() in stmmac_poll(). My workload is not > >like that -- it is "sending packets at 3MB/sec, receiving none". So > >the stmmac_tx_timer() is rescheduled and rescheduled and rescheduled, >

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi Pavel On 12/2/2016 9:45 AM, Pavel Machek wrote: Hi! 1 HZ, which is the lowest granularity of non-highres timers in the kernel, is variable as well as already too large of a delay for effective TX coalescing. I seriously think that the TX coalescing support should be ripped out or disabled

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! > >>1 HZ, which is the lowest granularity of non-highres timers in the > >>kernel, is variable as well as already too large of a delay for > >>effective TX coalescing. > >> > >>I seriously think that the TX coalescing support should be ripped out > >>or disabled entirely until it is

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

+ Lino On 12/2/2016 9:24 AM, Giuseppe CAVALLARO wrote: Hello On 11/24/2016 10:25 PM, Pavel Machek wrote: Hi! I'm debugging strange delays during transmit in stmmac driver. They seem to be present in 4.4 kernel (and older kernels, too). Workload is burst of udp packets being sent, pause,

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hello On 11/24/2016 10:25 PM, Pavel Machek wrote: Hi! I'm debugging strange delays during transmit in stmmac driver. They seem to be present in 4.4 kernel (and older kernels, too). Workload is burst of udp packets being sent, pause, burst of udp packets, ... ... 4.9-rc6 still has the

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! > >> drivers/net/ethernet/stmicro/stmmac/common.h > >> #define STMMAC_COAL_TX_TIMER4 > >> #define STMMAC_MAX_COAL_TX_TICK 10 > >> #define STMMAC_TX_MAX_FRAMES256 > >> > >> If I lower the parameters, delays are gone, but I get netdev watchdog > >> backtrace followed by broken

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! > >> I'm debugging strange delays during transmit in stmmac driver. They > >> seem to be present in 4.4 kernel (and older kernels, too). Workload is > >> burst of udp packets being sent, pause, burst of udp packets, ... ... > > 4.9-rc6 still has the delays. With the > > > > #define

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

From: Pavel Machek Date: Thu, 24 Nov 2016 09:55:06 +0100 > Hi! > >> I'm debugging strange delays during transmit in stmmac driver. They >> seem to be present in 4.4 kernel (and older kernels, too). Workload is >> burst of udp packets being sent, pause, burst of udp packets, ... >>

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! > What is going on with all these likely()s? Likely new hardware owners > will not be happy... or anyone running a lot of jumbo frames. (Perhaps > CPU's branch prediction can do better job here, without explicit hints?) > > if (unlikely(is_jumbo) && likely(priv->synopsys_id < >

Re: stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! What is going on with stmmac_tso_xmit() vs. stmmac_xmit()? One seems to be copy of another, with subtle differences -- like calling netif_queue_stopped() under spin_lock(>tx_lock), or not. What is going on with all these likely()s? Likely new hardware owners will not be happy... or anyone

stmmac ethernet in kernel 4.9-rc6: coalescing related pauses.

Hi! > I'm debugging strange delays during transmit in stmmac driver. They > seem to be present in 4.4 kernel (and older kernels, too). Workload is > burst of udp packets being sent, pause, burst of udp packets, ... > > Test code is attached, I use these parameters for testing: > > ./udp-test