Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On (10/19/16 15:34), Peter Zijlstra wrote: > On Wed, Oct 19, 2016 at 03:18:36PM +0200, Petr Mladek wrote: > > On Tue 2016-10-18 19:07:54, Peter Zijlstra wrote: [..] > It might make sense to go there, but allow early_console to print on the > go, keeping synchronous output available. We would still need the logbuf > to become NMI safe wrt adding entries though. > as a side (spam) note, I sent a [TECH TOPIC] suggestion to ksummit-discuss a while ago https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2016-July/002740.html and a later update https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2016-September/004006.html which covers some of the printk problems. so I'm sure I can tell whether the suggestion has been approved or rejected, but hopefully we can find some time to sit down and talk about the issues. -ss
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On (10/19/16 15:34), Peter Zijlstra wrote: > On Wed, Oct 19, 2016 at 03:18:36PM +0200, Petr Mladek wrote: > > On Tue 2016-10-18 19:07:54, Peter Zijlstra wrote: [..] > It might make sense to go there, but allow early_console to print on the > go, keeping synchronous output available. We would still need the logbuf > to become NMI safe wrt adding entries though. > as a side (spam) note, I sent a [TECH TOPIC] suggestion to ksummit-discuss a while ago https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2016-July/002740.html and a later update https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2016-September/004006.html which covers some of the printk problems. so I'm sure I can tell whether the suggestion has been approved or rejected, but hopefully we can find some time to sit down and talk about the issues. -ss
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On Wed, Oct 19, 2016 at 10:53:57AM +0900, Sergey Senozhatsky wrote: > not to miss out a DEFERRED_WARN patch set... > //hm, I can't find it online > > Subject: [RFC 0/5] printk: Implement WARN_*DEFERRED() > Message-Id: <1474992135-14777-1-git-send-email-pmla...@suse.com> That never hit lkml, that msgid does not resolve on marc and I can only find a few replies in my local lkml archive. Can't say I like the idea though. Its propagating the printk() failure up into more APIs instead of actually fixing it. Esp. for WARN and the like you do not want DEFERRED, because DEFERRED means you'll likely never actually get to see the problem because we're dead by the time 'later' happens.
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On Wed, Oct 19, 2016 at 10:53:57AM +0900, Sergey Senozhatsky wrote: > not to miss out a DEFERRED_WARN patch set... > //hm, I can't find it online > > Subject: [RFC 0/5] printk: Implement WARN_*DEFERRED() > Message-Id: <1474992135-14777-1-git-send-email-pmla...@suse.com> That never hit lkml, that msgid does not resolve on marc and I can only find a few replies in my local lkml archive. Can't say I like the idea though. Its propagating the printk() failure up into more APIs instead of actually fixing it. Esp. for WARN and the like you do not want DEFERRED, because DEFERRED means you'll likely never actually get to see the problem because we're dead by the time 'later' happens.
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On Tue 2016-10-18 19:07:54, Peter Zijlstra wrote: > On Wed, Oct 19, 2016 at 12:40:39AM +0900, Sergey Senozhatsky wrote: > > Hello, > > > > RFC > > > > This patch set extends a lock-less NMI per-cpu buffers idea to > > handle recursive printk() calls. The basic mechanism is pretty much the > > same -- at the beginning of a deadlock-prone section we switch to lock-less > > printk callback, and return back to a default printk implementation at the > > end; the messages are getting flushed to a logbuf buffer from a safer > > context. > > So I think you're not taking this far enough. You've also missed an > entire class of deadlocks. > > The first is that you still keep the logbuf. Having this global > serialized thing is a source of fail. It would be much better to only > keep per cpu stuff. _OR_ at the very least make the logbuf itself > lockfree. So generate the printk entry local (so we know its size) then > atomically reserve the logbuf entry and copy it over. This is close to what the lockless ring_buffer does. I mean kernel/trace/ring_buffer.c that is used by trace_printk. It has per-CPU buffers, does reservations, ... Sadly the ring_buffer code is very tricky. Only a single reader is allowed at a time. And locating the information in the crash dump is a task for prisoners. Note that it still does _not_ solve problems with the console output. I am not sure if going this way would be a real win. Sigh, a genius idea would help. > The entire class of deadlocks you've missed is that console->write() is > a piece of crap too ;-) Even the bog standard 8250 serial console driver > can do wakeups. I wonder if all the hard problems are actually related to the console handling. There are problems with the single logbuffer but these should get eliminated by the NMI/safe temporary per-CPU buffers. All might be easier if we always offload the console handling into a kthread or so and trigger it via the minimalist irq_work. It would kill huge bunch of possible deadlocks. It will even allow to get rid of printk_deferred() and the uncertainty where it is needed. The penalty would be "slightly" delayed console output. But is it a real problem? It should not be a big deal when everything works. We could always try hard when panicking. And there always might be a fallback with that direct early_console(). Best Regards, Petr
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On Tue 2016-10-18 19:07:54, Peter Zijlstra wrote: > On Wed, Oct 19, 2016 at 12:40:39AM +0900, Sergey Senozhatsky wrote: > > Hello, > > > > RFC > > > > This patch set extends a lock-less NMI per-cpu buffers idea to > > handle recursive printk() calls. The basic mechanism is pretty much the > > same -- at the beginning of a deadlock-prone section we switch to lock-less > > printk callback, and return back to a default printk implementation at the > > end; the messages are getting flushed to a logbuf buffer from a safer > > context. > > So I think you're not taking this far enough. You've also missed an > entire class of deadlocks. > > The first is that you still keep the logbuf. Having this global > serialized thing is a source of fail. It would be much better to only > keep per cpu stuff. _OR_ at the very least make the logbuf itself > lockfree. So generate the printk entry local (so we know its size) then > atomically reserve the logbuf entry and copy it over. This is close to what the lockless ring_buffer does. I mean kernel/trace/ring_buffer.c that is used by trace_printk. It has per-CPU buffers, does reservations, ... Sadly the ring_buffer code is very tricky. Only a single reader is allowed at a time. And locating the information in the crash dump is a task for prisoners. Note that it still does _not_ solve problems with the console output. I am not sure if going this way would be a real win. Sigh, a genius idea would help. > The entire class of deadlocks you've missed is that console->write() is > a piece of crap too ;-) Even the bog standard 8250 serial console driver > can do wakeups. I wonder if all the hard problems are actually related to the console handling. There are problems with the single logbuffer but these should get eliminated by the NMI/safe temporary per-CPU buffers. All might be easier if we always offload the console handling into a kthread or so and trigger it via the minimalist irq_work. It would kill huge bunch of possible deadlocks. It will even allow to get rid of printk_deferred() and the uncertainty where it is needed. The penalty would be "slightly" delayed console output. But is it a real problem? It should not be a big deal when everything works. We could always try hard when panicking. And there always might be a fallback with that direct early_console(). Best Regards, Petr
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On Wed, Oct 19, 2016 at 03:18:36PM +0200, Petr Mladek wrote: > On Tue 2016-10-18 19:07:54, Peter Zijlstra wrote: > > The entire class of deadlocks you've missed is that console->write() is > > a piece of crap too ;-) Even the bog standard 8250 serial console driver > > can do wakeups. > > I wonder if all the hard problems are actually related to the console > handling. > > There are problems with the single logbuffer but these should get > eliminated by the NMI/safe temporary per-CPU buffers. > > All might be easier if we always offload the console handling > into a kthread or so and trigger it via the minimalist > irq_work. It would kill huge bunch of possible deadlocks. > It will even allow to get rid of printk_deferred() and > the uncertainty where it is needed. > > The penalty would be "slightly" delayed console output. But is > it a real problem? It should not be a big deal when everything works. > We could always try hard when panicking. And there always > might be a fallback with that direct early_console(). Right, so I've had to debug quite a few situations where that 'later' simply never happens... I agree that such a scenario is basically all the regular console drivers are capable of though. It might make sense to go there, but allow early_console to print on the go, keeping synchronous output available. We would still need the logbuf to become NMI safe wrt adding entries though. It would however place hard requirements of early_console implementations; already violated by for instance the USB3-debug-port implementation I just looked at: lkml.kernel.org/r/20161019130943.ga3...@twins.programming.kicks-ass.net
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On Wed, Oct 19, 2016 at 03:18:36PM +0200, Petr Mladek wrote: > On Tue 2016-10-18 19:07:54, Peter Zijlstra wrote: > > The entire class of deadlocks you've missed is that console->write() is > > a piece of crap too ;-) Even the bog standard 8250 serial console driver > > can do wakeups. > > I wonder if all the hard problems are actually related to the console > handling. > > There are problems with the single logbuffer but these should get > eliminated by the NMI/safe temporary per-CPU buffers. > > All might be easier if we always offload the console handling > into a kthread or so and trigger it via the minimalist > irq_work. It would kill huge bunch of possible deadlocks. > It will even allow to get rid of printk_deferred() and > the uncertainty where it is needed. > > The penalty would be "slightly" delayed console output. But is > it a real problem? It should not be a big deal when everything works. > We could always try hard when panicking. And there always > might be a fallback with that direct early_console(). Right, so I've had to debug quite a few situations where that 'later' simply never happens... I agree that such a scenario is basically all the regular console drivers are capable of though. It might make sense to go there, but allow early_console to print on the go, keeping synchronous output available. We would still need the logbuf to become NMI safe wrt adding entries though. It would however place hard requirements of early_console implementations; already violated by for instance the USB3-debug-port implementation I just looked at: lkml.kernel.org/r/20161019130943.ga3...@twins.programming.kicks-ass.net
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On (10/19/16 00:40), Sergey Senozhatsky wrote: [..] > Deadlock scenarios that printk_safe can handle: > > a) printk recursion from logbuf_lock spin_lock section in printk() > printk() > raw_spin_lock(_lock); > WARN_ON(1); > raw_spin_unlock(_lock); > > b) printk from sem->lock spin_lock section > printk() > console_trylock() > down_trylock() > raw_spin_lock_irqsave(>lock, flags); > WARN_ON(1); > raw_spin_unlock_irqrestore(>lock, flags); > > c) printk from logbuf_lock spin_lock section in console_unlock() > printk() > console_unlock() > raw_spin_lock(_lock); > WARN_ON(1); > raw_spin_unlock(_lock); > > d) printk from ->pi_lock from semaphore up > printk() > console_unlock() > up() > try_to_wake_up() > raw_spin_lock_irqsave(>pi_lock, flags); > WARN_ON(1); > raw_spin_unlock_irqrestore(>pi_lock, flags); and e) anything from call_console_drivers() should be fine as well. console_unlock() big printing loop is covered by printk_safe, need to protect console_cont_flush()->call_console_drivers(). so printk "direct path" seems to be safe now: printk -> .. -> console_unlock() -> call_console_drivers() -> serial_console_write() anything that is "indirect" is not covered in this patch set. -ss
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On (10/19/16 00:40), Sergey Senozhatsky wrote: [..] > Deadlock scenarios that printk_safe can handle: > > a) printk recursion from logbuf_lock spin_lock section in printk() > printk() > raw_spin_lock(_lock); > WARN_ON(1); > raw_spin_unlock(_lock); > > b) printk from sem->lock spin_lock section > printk() > console_trylock() > down_trylock() > raw_spin_lock_irqsave(>lock, flags); > WARN_ON(1); > raw_spin_unlock_irqrestore(>lock, flags); > > c) printk from logbuf_lock spin_lock section in console_unlock() > printk() > console_unlock() > raw_spin_lock(_lock); > WARN_ON(1); > raw_spin_unlock(_lock); > > d) printk from ->pi_lock from semaphore up > printk() > console_unlock() > up() > try_to_wake_up() > raw_spin_lock_irqsave(>pi_lock, flags); > WARN_ON(1); > raw_spin_unlock_irqrestore(>pi_lock, flags); and e) anything from call_console_drivers() should be fine as well. console_unlock() big printing loop is covered by printk_safe, need to protect console_cont_flush()->call_console_drivers(). so printk "direct path" seems to be safe now: printk -> .. -> console_unlock() -> call_console_drivers() -> serial_console_write() anything that is "indirect" is not covered in this patch set. -ss
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On (10/18/16 19:07), Peter Zijlstra wrote: > > This patch set extends a lock-less NMI per-cpu buffers idea to > > handle recursive printk() calls. The basic mechanism is pretty much the > > same -- at the beginning of a deadlock-prone section we switch to lock-less > > printk callback, and return back to a default printk implementation at the > > end; the messages are getting flushed to a logbuf buffer from a safer > > context. > > So I think you're not taking this far enough. You've also missed an > entire class of deadlocks. yes. this patch set addresses only printk() recursion. printk deadlocks in general are much more complicated and basically we don't even know how many locks we are about to take every time we call printk() (there may be serial console locks, etc we never know in advance). > The first is that you still keep the logbuf. Having this global > serialized thing is a source of fail. It would be much better to only > keep per cpu stuff. _OR_ at the very least make the logbuf itself > lockfree. So generate the printk entry local (so we know its size) then > atomically reserve the logbuf entry and copy it over. logbuf spin_lock is not the only problematic lock. we _at least_ have console sem spin_lock and rq spin_lock: https://marc.info/?l=linux-kernel=145406977822244 https://marc.info/?l=linux-kernel=146853438804534=2 > The entire class of deadlocks you've missed is that console->write() is > a piece of crap too ;-) Even the bog standard 8250 serial console driver > can do wakeups. yes. well, I didn't exactly "miss" it. I just don't have any working solution for those locks yet. so things like SyS_ioctl do_vfs_ioctl tty_ioctl n_tty_ioctl tty_mode_ioctl set_termios tty_set_termios uart_set_termios uart_change_speed FOO_serial_set_termios spin_lock_irqsave(>lock) // lock the output port !! WARN() or pr_err() or printk() vprintk_emit() /* console_trylock() */ console_unlock() call_console_drivers() FOO_write() spin_lock_irqsave(>lock) // already locked are out of this patch set's scope. another class of deadlocks is when we call printk under one of the locks that printk() can take later in order to print the message. so in this particular patch set I make "printk() -> $FOO -> printk()" less damaging. which is just a first step. we are also looking at things like might_printk() https://marc.info/?l=linux-kernel=147607261116163 and even extending the lockdep https://marc.info/?l=linux-kernel=147617830511381 not to miss out a DEFERRED_WARN patch set... //hm, I can't find it online Subject: [RFC 0/5] printk: Implement WARN_*DEFERRED() Message-Id: <1474992135-14777-1-git-send-email-pmla...@suse.com> > See for example: > > https://marc.info/?l=linux-kernel=132446649823347=2 interesting idea. > I've entirely given up on printk(), I'll post the 3 patches I carry to > make things work for me. yes, printk is surprisingly fragile and, some would _probably_ even say, broken. a) we have numerous scenarios when it can deadlock b) we have numerous scenarios when it can soft-lockup/hard-lockup the system c) we don't have a bullet proof panic printk. zap_locks() resets logbuf and console_sem, but there may be serial console locks/etc. -ss
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On (10/18/16 19:07), Peter Zijlstra wrote: > > This patch set extends a lock-less NMI per-cpu buffers idea to > > handle recursive printk() calls. The basic mechanism is pretty much the > > same -- at the beginning of a deadlock-prone section we switch to lock-less > > printk callback, and return back to a default printk implementation at the > > end; the messages are getting flushed to a logbuf buffer from a safer > > context. > > So I think you're not taking this far enough. You've also missed an > entire class of deadlocks. yes. this patch set addresses only printk() recursion. printk deadlocks in general are much more complicated and basically we don't even know how many locks we are about to take every time we call printk() (there may be serial console locks, etc we never know in advance). > The first is that you still keep the logbuf. Having this global > serialized thing is a source of fail. It would be much better to only > keep per cpu stuff. _OR_ at the very least make the logbuf itself > lockfree. So generate the printk entry local (so we know its size) then > atomically reserve the logbuf entry and copy it over. logbuf spin_lock is not the only problematic lock. we _at least_ have console sem spin_lock and rq spin_lock: https://marc.info/?l=linux-kernel=145406977822244 https://marc.info/?l=linux-kernel=146853438804534=2 > The entire class of deadlocks you've missed is that console->write() is > a piece of crap too ;-) Even the bog standard 8250 serial console driver > can do wakeups. yes. well, I didn't exactly "miss" it. I just don't have any working solution for those locks yet. so things like SyS_ioctl do_vfs_ioctl tty_ioctl n_tty_ioctl tty_mode_ioctl set_termios tty_set_termios uart_set_termios uart_change_speed FOO_serial_set_termios spin_lock_irqsave(>lock) // lock the output port !! WARN() or pr_err() or printk() vprintk_emit() /* console_trylock() */ console_unlock() call_console_drivers() FOO_write() spin_lock_irqsave(>lock) // already locked are out of this patch set's scope. another class of deadlocks is when we call printk under one of the locks that printk() can take later in order to print the message. so in this particular patch set I make "printk() -> $FOO -> printk()" less damaging. which is just a first step. we are also looking at things like might_printk() https://marc.info/?l=linux-kernel=147607261116163 and even extending the lockdep https://marc.info/?l=linux-kernel=147617830511381 not to miss out a DEFERRED_WARN patch set... //hm, I can't find it online Subject: [RFC 0/5] printk: Implement WARN_*DEFERRED() Message-Id: <1474992135-14777-1-git-send-email-pmla...@suse.com> > See for example: > > https://marc.info/?l=linux-kernel=132446649823347=2 interesting idea. > I've entirely given up on printk(), I'll post the 3 patches I carry to > make things work for me. yes, printk is surprisingly fragile and, some would _probably_ even say, broken. a) we have numerous scenarios when it can deadlock b) we have numerous scenarios when it can soft-lockup/hard-lockup the system c) we don't have a bullet proof panic printk. zap_locks() resets logbuf and console_sem, but there may be serial console locks/etc. -ss
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On (10/18/16 09:45), Joe Perches wrote:
> On Wed, 2016-10-19 at 00:40 +0900, Sergey Senozhatsky wrote:
> > This patch set extends a lock-less NMI per-cpu buffers idea to
> > handle recursive printk() calls.
>
> trivia:
>
> recursive or reentrant?
a recursive one.
printk -> {foo} -> printk
reentrant printk() case was addressed by Petr Mladek's nmi patchset:
printk()
foo
---> NMI
printk()
-ss
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On (10/18/16 09:45), Joe Perches wrote:
> On Wed, 2016-10-19 at 00:40 +0900, Sergey Senozhatsky wrote:
> > This patch set extends a lock-less NMI per-cpu buffers idea to
> > handle recursive printk() calls.
>
> trivia:
>
> recursive or reentrant?
a recursive one.
printk -> {foo} -> printk
reentrant printk() case was addressed by Petr Mladek's nmi patchset:
printk()
foo
---> NMI
printk()
-ss
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On Wed, Oct 19, 2016 at 12:40:39AM +0900, Sergey Senozhatsky wrote: > Hello, > > RFC > > This patch set extends a lock-less NMI per-cpu buffers idea to > handle recursive printk() calls. The basic mechanism is pretty much the > same -- at the beginning of a deadlock-prone section we switch to lock-less > printk callback, and return back to a default printk implementation at the > end; the messages are getting flushed to a logbuf buffer from a safer > context. So I think you're not taking this far enough. You've also missed an entire class of deadlocks. The first is that you still keep the logbuf. Having this global serialized thing is a source of fail. It would be much better to only keep per cpu stuff. _OR_ at the very least make the logbuf itself lockfree. So generate the printk entry local (so we know its size) then atomically reserve the logbuf entry and copy it over. The entire class of deadlocks you've missed is that console->write() is a piece of crap too ;-) Even the bog standard 8250 serial console driver can do wakeups. See for example: https://marc.info/?l=linux-kernel=132446649823347=2 I've entirely given up on printk(), I'll post the 3 patches I carry to make things work for me.
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On Wed, Oct 19, 2016 at 12:40:39AM +0900, Sergey Senozhatsky wrote: > Hello, > > RFC > > This patch set extends a lock-less NMI per-cpu buffers idea to > handle recursive printk() calls. The basic mechanism is pretty much the > same -- at the beginning of a deadlock-prone section we switch to lock-less > printk callback, and return back to a default printk implementation at the > end; the messages are getting flushed to a logbuf buffer from a safer > context. So I think you're not taking this far enough. You've also missed an entire class of deadlocks. The first is that you still keep the logbuf. Having this global serialized thing is a source of fail. It would be much better to only keep per cpu stuff. _OR_ at the very least make the logbuf itself lockfree. So generate the printk entry local (so we know its size) then atomically reserve the logbuf entry and copy it over. The entire class of deadlocks you've missed is that console->write() is a piece of crap too ;-) Even the bog standard 8250 serial console driver can do wakeups. See for example: https://marc.info/?l=linux-kernel=132446649823347=2 I've entirely given up on printk(), I'll post the 3 patches I carry to make things work for me.
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On Wed, 2016-10-19 at 00:40 +0900, Sergey Senozhatsky wrote: > This patch set extends a lock-less NMI per-cpu buffers idea to > handle recursive printk() calls. trivia: recursive or reentrant?
Re: [RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
On Wed, 2016-10-19 at 00:40 +0900, Sergey Senozhatsky wrote: > This patch set extends a lock-less NMI per-cpu buffers idea to > handle recursive printk() calls. trivia: recursive or reentrant?
[RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
Hello,
RFC
This patch set extends a lock-less NMI per-cpu buffers idea to
handle recursive printk() calls. The basic mechanism is pretty much the
same -- at the beginning of a deadlock-prone section we switch to lock-less
printk callback, and return back to a default printk implementation at the
end; the messages are getting flushed to a logbuf buffer from a safer
context.
Deadlock scenarios that printk_safe can handle:
a) printk recursion from logbuf_lock spin_lock section in printk()
printk()
raw_spin_lock(_lock);
WARN_ON(1);
raw_spin_unlock(_lock);
b) printk from sem->lock spin_lock section
printk()
console_trylock()
down_trylock()
raw_spin_lock_irqsave(>lock, flags);
WARN_ON(1);
raw_spin_unlock_irqrestore(>lock, flags);
c) printk from logbuf_lock spin_lock section in console_unlock()
printk()
console_unlock()
raw_spin_lock(_lock);
WARN_ON(1);
raw_spin_unlock(_lock);
d) printk from ->pi_lock from semaphore up
printk()
console_unlock()
up()
try_to_wake_up()
raw_spin_lock_irqsave(>pi_lock, flags);
WARN_ON(1);
raw_spin_unlock_irqrestore(>pi_lock, flags);
[[against next-20161018]]
v3: (review by Petr)
-- renamed to printk_safe
-- !NMI config build fix
-- report lost messages for both printk_sae and printk_nmi
-- dropped recursion reporting patch
-- etc.
v2:
-- fix build error on !NMI configs, reported by Fengguang
-- reworked the series based on Petr's feedback
-- added a new patch to drop zap_locks()
Sergey Senozhatsky (6):
printk: use vprintk_func in vprintk()
printk: rename nmi.c and exported api
printk: introduce per-cpu safe_print seq buffer
printk: report lost messages in printk safe/nmi contexts
printk: use printk_safe buffers
printk: remove zap_locks() function
arch/arm/kernel/smp.c | 4 +-
include/linux/hardirq.h| 4 +-
include/linux/printk.h | 29 ++--
init/Kconfig | 16 ++-
init/main.c| 2 +-
kernel/kexec_core.c| 2 +-
kernel/panic.c | 4 +-
kernel/printk/Makefile | 3 +-
kernel/printk/internal.h | 65 +
kernel/printk/printk.c | 116
kernel/printk/{nmi.c => printk_safe.c} | 239 +
lib/nmi_backtrace.c| 2 +-
12 files changed, 277 insertions(+), 209 deletions(-)
rename kernel/printk/{nmi.c => printk_safe.c} (54%)
--
2.10.1.382.ga23ca1b
[RFC][PATCHv3 0/6] printk: use printk_safe to handle printk() recursive calls
Hello,
RFC
This patch set extends a lock-less NMI per-cpu buffers idea to
handle recursive printk() calls. The basic mechanism is pretty much the
same -- at the beginning of a deadlock-prone section we switch to lock-less
printk callback, and return back to a default printk implementation at the
end; the messages are getting flushed to a logbuf buffer from a safer
context.
Deadlock scenarios that printk_safe can handle:
a) printk recursion from logbuf_lock spin_lock section in printk()
printk()
raw_spin_lock(_lock);
WARN_ON(1);
raw_spin_unlock(_lock);
b) printk from sem->lock spin_lock section
printk()
console_trylock()
down_trylock()
raw_spin_lock_irqsave(>lock, flags);
WARN_ON(1);
raw_spin_unlock_irqrestore(>lock, flags);
c) printk from logbuf_lock spin_lock section in console_unlock()
printk()
console_unlock()
raw_spin_lock(_lock);
WARN_ON(1);
raw_spin_unlock(_lock);
d) printk from ->pi_lock from semaphore up
printk()
console_unlock()
up()
try_to_wake_up()
raw_spin_lock_irqsave(>pi_lock, flags);
WARN_ON(1);
raw_spin_unlock_irqrestore(>pi_lock, flags);
[[against next-20161018]]
v3: (review by Petr)
-- renamed to printk_safe
-- !NMI config build fix
-- report lost messages for both printk_sae and printk_nmi
-- dropped recursion reporting patch
-- etc.
v2:
-- fix build error on !NMI configs, reported by Fengguang
-- reworked the series based on Petr's feedback
-- added a new patch to drop zap_locks()
Sergey Senozhatsky (6):
printk: use vprintk_func in vprintk()
printk: rename nmi.c and exported api
printk: introduce per-cpu safe_print seq buffer
printk: report lost messages in printk safe/nmi contexts
printk: use printk_safe buffers
printk: remove zap_locks() function
arch/arm/kernel/smp.c | 4 +-
include/linux/hardirq.h| 4 +-
include/linux/printk.h | 29 ++--
init/Kconfig | 16 ++-
init/main.c| 2 +-
kernel/kexec_core.c| 2 +-
kernel/panic.c | 4 +-
kernel/printk/Makefile | 3 +-
kernel/printk/internal.h | 65 +
kernel/printk/printk.c | 116
kernel/printk/{nmi.c => printk_safe.c} | 239 +
lib/nmi_backtrace.c| 2 +-
12 files changed, 277 insertions(+), 209 deletions(-)
rename kernel/printk/{nmi.c => printk_safe.c} (54%)
--
2.10.1.382.ga23ca1b
