Re: [Bug 199003] console stalled, cause Hard LOCKUP.

I'll Cc blockdev On (03/27/18 08:36), bugzilla-dae...@bugzilla.kernel.org wrote: > > --- Comment #17 from sergey.senozhatsky.w...@gmail.com --- > > On (03/26/18 13:05), bugzilla-dae...@bugzilla.kernel.org wrote: > > > Therefore the serial console is actually pretty fast. It seems that the > > >

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

I'll Cc blockdev On (03/27/18 08:36), bugzilla-dae...@bugzilla.kernel.org wrote: > > --- Comment #17 from sergey.senozhatsky.w...@gmail.com --- > > On (03/26/18 13:05), bugzilla-dae...@bugzilla.kernel.org wrote: > > > Therefore the serial console is actually pretty fast. It seems that the > > >

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On Mon 2018-03-26 14:12:22, Sergey Senozhatsky wrote: > Given how slow serial8250_console_putchar()->wait_for_xmitr() can be - > 10ms of delay for every char - it's possible that we had no concurrent > printk()-s from other CPUs. So may be we had just one printing CPU, > and several CPUs spinning

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On Mon 2018-03-26 14:12:22, Sergey Senozhatsky wrote: > Given how slow serial8250_console_putchar()->wait_for_xmitr() can be - > 10ms of delay for every char - it's possible that we had no concurrent > printk()-s from other CPUs. So may be we had just one printing CPU, > and several CPUs spinning

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

Cc-ing the kernel list and printk people. Wen Yang, any chance we can switch to email? Bugzilla is not very handful. On (03/26/18 02:40), bugzilla-dae...@bugzilla.kernel.org wrote: > https://bugzilla.kernel.org/show_bug.cgi?id=199003 > > --- Comment #11 from Wen Yang (wen.yan...@zte.com.cn)

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

Cc-ing the kernel list and printk people. Wen Yang, any chance we can switch to email? Bugzilla is not very handful. On (03/26/18 02:40), bugzilla-dae...@bugzilla.kernel.org wrote: > https://bugzilla.kernel.org/show_bug.cgi?id=199003 > > --- Comment #11 from Wen Yang (wen.yan...@zte.com.cn)

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On (03/23/18 14:16), Petr Mladek wrote: [..] > If I get it correctly, the reporter of this bug has not tried > Steven's patches yet. It's not immediately clear. It's not even completely clear if we are looking at "X cpus printk 1 cpu prints it all scenario" and it's not clear if hand off will be

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On (03/23/18 14:16), Petr Mladek wrote: [..] > If I get it correctly, the reporter of this bug has not tried > Steven's patches yet. It's not immediately clear. It's not even completely clear if we are looking at "X cpus printk 1 cpu prints it all scenario" and it's not clear if hand off will be

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On Fri 2018-03-23 21:06:22, Sergey Senozhatsky wrote: > > > Sometimes I really wish we had detached consoles. Direct printk()->console > > > is nice and cool, but... we can't have it. I don't want to pressure for > > > printk_kthread, but look at all those consoles. There are not so many ways > >

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On Fri 2018-03-23 21:06:22, Sergey Senozhatsky wrote: > > > Sometimes I really wish we had detached consoles. Direct printk()->console > > > is nice and cool, but... we can't have it. I don't want to pressure for > > > printk_kthread, but look at all those consoles. There are not so many ways > >

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

Hello Steven, On (03/22/18 18:25), Steven Rostedt wrote: > > static void wait_for_xmitr(struct uart_8250_port *up, int bits) > > { > > unsigned int status, tmout = 1; > > > > /* Wait up to 10ms for the character(s) to be sent. */ > > for (;;) { > >

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

Hello Steven, On (03/22/18 18:25), Steven Rostedt wrote: > > static void wait_for_xmitr(struct uart_8250_port *up, int bits) > > { > > unsigned int status, tmout = 1; > > > > /* Wait up to 10ms for the character(s) to be sent. */ > > for (;;) { > >

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On Thu, 22 Mar 2018 11:14:37 +0900 Sergey Senozhatsky wrote: > Looking at > > printk()->call_console_drivers()->serial8250_console_putchar()->wait_for_xmitr() > > ... wait_for_xmitr() can spin for over 1 second waiting for the UART_MSR_CTS > bit. > >

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On Thu, 22 Mar 2018 11:14:37 +0900 Sergey Senozhatsky wrote: > Looking at > > printk()->call_console_drivers()->serial8250_console_putchar()->wait_for_xmitr() > > ... wait_for_xmitr() can spin for over 1 second waiting for the UART_MSR_CTS > bit. > > static void wait_for_xmitr(struct

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On (03/22/18 11:14), Sergey Senozhatsky wrote: [..] > Looking at > > printk()->call_console_drivers()->serial8250_console_putchar()->wait_for_xmitr() > > ... wait_for_xmitr() can spin for over 1 second waiting for the UART_MSR_CTS > bit. [..] > a 1+ second long busy loop in the console driver

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On (03/22/18 11:14), Sergey Senozhatsky wrote: [..] > Looking at > > printk()->call_console_drivers()->serial8250_console_putchar()->wait_for_xmitr() > > ... wait_for_xmitr() can spin for over 1 second waiting for the UART_MSR_CTS > bit. [..] > a 1+ second long busy loop in the console driver

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

Hello Steven, On (03/21/18 09:44), Steven Rostedt wrote: [..] > > #4 [881ffb0b7540] delay_tsc at 81326724 > > #5 [881ffb0b7548] __const_udelay at 81326678 > > #6 [881ffb0b7558] wait_for_xmitr at 814056e0 > > #7 [881ffb0b7580] serial8250_console_putchar

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

Hello Steven, On (03/21/18 09:44), Steven Rostedt wrote: [..] > > #4 [881ffb0b7540] delay_tsc at 81326724 > > #5 [881ffb0b7548] __const_udelay at 81326678 > > #6 [881ffb0b7558] wait_for_xmitr at 814056e0 > > #7 [881ffb0b7580] serial8250_console_putchar

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

[ Adding LKML too ] Link: https://bugzilla.kernel.org/show_bug.cgi?id=199003 >From the bugzilla report: > Watchdog detected hard LOCKUP on cpu 16: > [ 557.369547] Kernel panic - not syncing: Hard LOCKUP > [ 557.369548] CPU: 16 PID: 11511 Comm: kubelet Tainted: GW OE >

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

[ Adding LKML too ] Link: https://bugzilla.kernel.org/show_bug.cgi?id=199003 >From the bugzilla report: > Watchdog detected hard LOCKUP on cpu 16: > [ 557.369547] Kernel panic - not syncing: Hard LOCKUP > [ 557.369548] CPU: 16 PID: 11511 Comm: kubelet Tainted: GW OE >

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On (03/20/18 09:34), bugzilla-dae...@bugzilla.kernel.org wrote: [..] > Thanks very much. > commit e480af09c49736848f749a43dff2c902104f6691 avoided the NMI watchdog > trigger. Hm, okay... But "touch_nmi_watchdog() everywhere printk/console-related" is not exactly where I wanted us to be. By the

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

On (03/20/18 09:34), bugzilla-dae...@bugzilla.kernel.org wrote: [..] > Thanks very much. > commit e480af09c49736848f749a43dff2c902104f6691 avoided the NMI watchdog > trigger. Hm, okay... But "touch_nmi_watchdog() everywhere printk/console-related" is not exactly where I wanted us to be. By the

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

Hello, Let's switch to email. Cc-ing Petr and Steven On (03/05/18 07:50), bugzilla-dae...@bugzilla.kernel.org wrote: > > https://bugzilla.kernel.org/show_bug.cgi?id=199003 > > yangwen (wen.yan...@zte.com.cn) changed: > >What|Removed |Added >

Re: [Bug 199003] console stalled, cause Hard LOCKUP.

Hello, Let's switch to email. Cc-ing Petr and Steven On (03/05/18 07:50), bugzilla-dae...@bugzilla.kernel.org wrote: > > https://bugzilla.kernel.org/show_bug.cgi?id=199003 > > yangwen (wen.yan...@zte.com.cn) changed: > >What|Removed |Added >