Re: [PATCH printk v2 2/5] printk: remove safe buffers
On Fri 2021-04-02 11:14:18, Sergey Senozhatsky wrote: > On (21/04/01 16:17), Petr Mladek wrote: > > > For the long term, we should introduce a printk-context API that allows > > > callers to perfectly pack their multi-line output into a single > > > entry. We discussed [0][1] this back in August 2020. > > > > We need a "short" term solution. There are currently 3 solutions: > > > > 1. Keep nmi_safe() and all the hacks around. > > > > 2. Serialize nmi_cpu_backtrace() by a spin lock and later by > >the special lock used also by atomic consoles. > > > > 3. Tell complaining people how to sort the messed logs. > > Are we talking about nmi_cpu_backtrace()->dump_stack() or some > other path? It is about serializing if (regs) show_regs(regs); else dump_stack(); in nmi_cpu_backtrace() when it is triggered on many(all) CPUs at the same time. > dump_stack() seems to be already serialized by `dump_lock`. Hmm, > show_regs() is not serialized, seems like it should be under the > same `dump_lock` as dump_stack(). Ah, I think that you already mentioned it in the past and I forget it. Yes, we would need to synchronize all these dump/show functions using the same lock. It is already the lock that might be taken recursively on the same CPU. In each case, we must not introduce another lock in nmi_cpu_backtrace() because it might cause deadlock with @dump_lock. Anyway, I would really like to keep the dumps serialized. So, we either need to use the same lock everywhere or we need to keep nmi_safe buffers for now. I would like to remove the nmi_safe buffers in the long term but I am fine with doing it later after the consoles rework. I'll leave the prioritization for John who is doing the work and might have some preferences. Best Regards, Petr ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH printk v2 2/5] printk: remove safe buffers
On 2021-04-01, Petr Mladek wrote: >> Caller-id solves this problem and is easy to sort for anyone with >> `grep'. Yes, it is a shame that `dmesg' does not show it, but >> directly using any of the printk interfaces does show it (kmsg_dump, >> /dev/kmsg, syslog, console). > > True but frankly, the current situation is _far_ from convenient: > >+ consoles do not show it by default >+ none userspace tool (dmesg, journalctl, crash) is able to show it >+ grep is a nightmare, especially if you have more than handful of CPUs > > Yes, everything is solvable but not easily. > >> > I get this with "echo l >/proc/sysrq-trigger" and this patchset: >> >> Of course. Without caller-id, it is a mess. But this has nothing to do >> with NMI. The same problem exists for WARN_ON() on multiple CPUs >> simultaneously. If the user is not using caller-id, they are >> lost. Caller-id is the current solution to the interlaced logs. > > Sure. But in reality, the risk of mixed WARN_ONs is small. While > this patch makes backtraces from all CPUs always unusable without > caller_id and non-trivial effort. I would prefer we solve the situation for non-NMI as well, not just for the sysrq "l" case. >> For the long term, we should introduce a printk-context API that allows >> callers to perfectly pack their multi-line output into a single >> entry. We discussed [0][1] this back in August 2020. > > We need a "short" term solution. There are currently 3 solutions: > > 1. Keep nmi_safe() and all the hacks around. > > 2. Serialize nmi_cpu_backtrace() by a spin lock and later by >the special lock used also by atomic consoles. > > 3. Tell complaining people how to sort the messed logs. Or we look into the long term solution now. If caller-id's cannot not be used as the solution (because nobody turns it on, nobody knows about it, and/or distros do not enable it), then we should look at how to make at least the backtraces contiguous. I have a few ideas here. John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH printk v2 2/5] printk: remove safe buffers
On (21/04/01 16:17), Petr Mladek wrote: > > For the long term, we should introduce a printk-context API that allows > > callers to perfectly pack their multi-line output into a single > > entry. We discussed [0][1] this back in August 2020. > > We need a "short" term solution. There are currently 3 solutions: > > 1. Keep nmi_safe() and all the hacks around. > > 2. Serialize nmi_cpu_backtrace() by a spin lock and later by >the special lock used also by atomic consoles. > > 3. Tell complaining people how to sort the messed logs. Are we talking about nmi_cpu_backtrace()->dump_stack() or some other path? dump_stack() seems to be already serialized by `dump_lock`. Hmm, show_regs() is not serialized, seems like it should be under the same `dump_lock` as dump_stack(). ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH printk v2 2/5] printk: remove safe buffers
On Thu 2021-04-01 15:19:52, John Ogness wrote:
> On 2021-04-01, Petr Mladek wrote:
> >> --- a/kernel/printk/printk.c
> >> +++ b/kernel/printk/printk.c
> >> @@ -1142,24 +1128,37 @@ void __init setup_log_buf(int early)
> >> new_descs, ilog2(new_descs_count),
> >> new_infos);
> >>
> >> - printk_safe_enter_irqsave(flags);
> >> + local_irq_save(flags);
> >
> > IMHO, we actually do not have to disable IRQ here. We already copy
> > messages that might appear in the small race window in NMI. It would
> > work the same way also for IRQ context.
>
> We do not have to, but why open up this window? We are still in early
> boot and interrupts have always been disabled here. I am not happy that
> this window even exists. I really prefer to keep it NMI-only.
Fair enough.
> >> --- a/lib/nmi_backtrace.c
> >> +++ b/lib/nmi_backtrace.c
> >> @@ -75,12 +75,6 @@ void nmi_trigger_cpumask_backtrace(const cpumask_t
> >> *mask,
> >>touch_softlockup_watchdog();
> >>}
> >>
> >> - /*
> >> - * Force flush any remote buffers that might be stuck in IRQ context
> >> - * and therefore could not run their irq_work.
> >> - */
> >> - printk_safe_flush();
> >
> > Sigh, this reminds me that the nmi_safe buffers serialized backtraces
> > from all CPUs.
> >
> > I am afraid that we have to put back the spinlock into
> > nmi_cpu_backtrace().
>
> Please no. That spinlock is a disaster. It can cause deadlocks with
> other cpu-locks (such as in kdb)
Could you please explain more the kdb case?
I am curious what locks might depend on each other here.
> and it will cause a major problem for atomic consoles.
AFAIK, you are going to add a special lock that would allow
nesting on the same CPU. It should possible and safe
to use is also for synchronizing the backtraces here.
> We need to be very careful about introducing locks
> where NMIs are waiting on other CPUs.
I agree.
> > It has been repeatedly added and removed depending
> > on whether the backtrace was printed into the main log buffer
> > or into the per-CPU buffers. Last time it was removed by
> > the commit 03fc7f9c99c1e7ae2925d ("printk/nmi: Prevent deadlock
> > when accessing the main log buffer in NMI").
> >
> > It should be safe because there should not be any other locks in the
> > code path. Note that only one backtrace might be triggered at the same
> > time, see @backtrace_flag in nmi_trigger_cpumask_backtrace().
>
> It is adding a lock around a lockless ringbuffer. For me that is a step
> backwards.
>
> > We _must_ serialize it somehow[*]. The lock in nmi_cpu_backtrace()
> > looks less evil than the nmi_safe machinery. nmi_safe() shrinks
> > too long backtraces, lose timestamps, needs to be explicitely
> > flushed here and there, is a non-trivial code.
> >
> > [*] Non-serialized bactraces are real mess. Caller-id is visible
> > only on consoles or via syslogd interface. And it is not much
> > convenient.
>
> Caller-id solves this problem and is easy to sort for anyone with
> `grep'. Yes, it is a shame that `dmesg' does not show it, but directly
> using any of the printk interfaces does show it (kmsg_dump, /dev/kmsg,
> syslog, console).
True but frankly, the current situation is _far_ from convenient:
+ consoles do not show it by default
+ none userspace tool (dmesg, journalctl, crash) is able to show it
+ grep is a nightmare, especially if you have more than handful of CPUs
Yes, everything is solvable but not easily.
> > I get this with "echo l >/proc/sysrq-trigger" and this patchset:
>
> Of course. Without caller-id, it is a mess. But this has nothing to do
> with NMI. The same problem exists for WARN_ON() on multiple CPUs
> simultaneously. If the user is not using caller-id, they are
> lost. Caller-id is the current solution to the interlaced logs.
Sure. But in reality, the risk of mixed WARN_ONs is small. While
this patch makes backtraces from all CPUs always unusable without
caller_id and non-trivial effort.
> For the long term, we should introduce a printk-context API that allows
> callers to perfectly pack their multi-line output into a single
> entry. We discussed [0][1] this back in August 2020.
We need a "short" term solution. There are currently 3 solutions:
1. Keep nmi_safe() and all the hacks around.
2. Serialize nmi_cpu_backtrace() by a spin lock and later by
the special lock used also by atomic consoles.
3. Tell complaining people how to sort the messed logs.
My preference:
I most prefer 2nd solution until I see a realistic scenario
of a possible deadlock with the current kernel code.
I would still prefer 1st solution over 3rd one until we improve
kernel/userspace support for sorting the log by the caller id.
Best Regards,
Petr
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH printk v2 2/5] printk: remove safe buffers
On 2021-04-01, Petr Mladek wrote:
>> --- a/kernel/printk/printk.c
>> +++ b/kernel/printk/printk.c
>> @@ -1142,24 +1128,37 @@ void __init setup_log_buf(int early)
>> new_descs, ilog2(new_descs_count),
>> new_infos);
>>
>> -printk_safe_enter_irqsave(flags);
>> +local_irq_save(flags);
>
> IMHO, we actually do not have to disable IRQ here. We already copy
> messages that might appear in the small race window in NMI. It would
> work the same way also for IRQ context.
We do not have to, but why open up this window? We are still in early
boot and interrupts have always been disabled here. I am not happy that
this window even exists. I really prefer to keep it NMI-only.
>> --- a/lib/nmi_backtrace.c
>> +++ b/lib/nmi_backtrace.c
>> @@ -75,12 +75,6 @@ void nmi_trigger_cpumask_backtrace(const cpumask_t *mask,
>> touch_softlockup_watchdog();
>> }
>>
>> -/*
>> - * Force flush any remote buffers that might be stuck in IRQ context
>> - * and therefore could not run their irq_work.
>> - */
>> -printk_safe_flush();
>
> Sigh, this reminds me that the nmi_safe buffers serialized backtraces
> from all CPUs.
>
> I am afraid that we have to put back the spinlock into
> nmi_cpu_backtrace().
Please no. That spinlock is a disaster. It can cause deadlocks with
other cpu-locks (such as in kdb) and it will cause a major problem for
atomic consoles. We need to be very careful about introducing locks
where NMIs are waiting on other CPUs.
> It has been repeatedly added and removed depending
> on whether the backtrace was printed into the main log buffer
> or into the per-CPU buffers. Last time it was removed by
> the commit 03fc7f9c99c1e7ae2925d ("printk/nmi: Prevent deadlock
> when accessing the main log buffer in NMI").
>
> It should be safe because there should not be any other locks in the
> code path. Note that only one backtrace might be triggered at the same
> time, see @backtrace_flag in nmi_trigger_cpumask_backtrace().
It is adding a lock around a lockless ringbuffer. For me that is a step
backwards.
> We _must_ serialize it somehow[*]. The lock in nmi_cpu_backtrace()
> looks less evil than the nmi_safe machinery. nmi_safe() shrinks
> too long backtraces, lose timestamps, needs to be explicitely
> flushed here and there, is a non-trivial code.
>
> [*] Non-serialized bactraces are real mess. Caller-id is visible
> only on consoles or via syslogd interface. And it is not much
> convenient.
Caller-id solves this problem and is easy to sort for anyone with
`grep'. Yes, it is a shame that `dmesg' does not show it, but directly
using any of the printk interfaces does show it (kmsg_dump, /dev/kmsg,
syslog, console).
> I get this with "echo l >/proc/sysrq-trigger" and this patchset:
Of course. Without caller-id, it is a mess. But this has nothing to do
with NMI. The same problem exists for WARN_ON() on multiple CPUs
simultaneously. If the user is not using caller-id, they are
lost. Caller-id is the current solution to the interlaced logs.
For the long term, we should introduce a printk-context API that allows
callers to perfectly pack their multi-line output into a single
entry. We discussed [0][1] this back in August 2020.
John Ogness
[0]
https://lore.kernel.org/lkml/472f2e553805b52d9834d64e4056db965edee329.ca...@perches.com
[1] offlist message-id: 87d03k9ymz@jogness.linutronix.de
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH printk v2 2/5] printk: remove safe buffers
On Tue 2021-03-30 17:35:09, John Ogness wrote:
> With @logbuf_lock removed, the high level printk functions for
> storing messages are lockless. Messages can be stored from any
> context, so there is no need for the NMI and safe buffers anymore.
> Remove the NMI and safe buffers.
>
> Although the safe buffers are removed, the NMI and safe context
> tracking is still in place. In these contexts, store the message
> immediately but still use irq_work to defer the console printing.
>
> Since printk recursion tracking is in place, safe context tracking
> for most of printk is not needed. Remove it. Only safe context
> tracking relating to the console lock is left in place. This is
> because the console lock is needed for the actual printing.
> --- a/kernel/printk/printk.c
> +++ b/kernel/printk/printk.c
> @@ -1142,24 +1128,37 @@ void __init setup_log_buf(int early)
>new_descs, ilog2(new_descs_count),
>new_infos);
>
> - printk_safe_enter_irqsave(flags);
> + local_irq_save(flags);
IMHO, we actually do not have to disable IRQ here. We already copy
messages that might appear in the small race window in NMI. It would work
the same way also for IRQ context.
> log_buf_len = new_log_buf_len;
> log_buf = new_log_buf;
> new_log_buf_len = 0;
>
> free = __LOG_BUF_LEN;
> - prb_for_each_record(0, _rb_static, seq, )
> - free -= add_to_rb(_rb_dynamic, );
> + prb_for_each_record(0, _rb_static, seq, ) {
> + text_size = add_to_rb(_rb_dynamic, );
> + if (text_size > free)
> + free = 0;
> + else
> + free -= text_size;
> + }
>
> - /*
> - * This is early enough that everything is still running on the
> - * boot CPU and interrupts are disabled. So no new messages will
> - * appear during the transition to the dynamic buffer.
> - */
> prb = _rb_dynamic;
>
> - printk_safe_exit_irqrestore(flags);
> + local_irq_restore(flags);
> +
> + /*
> + * Copy any remaining messages that might have appeared from
> + * NMI context after copying but before switching to the
> + * dynamic buffer.
The above comment would need to get updated if we keep also normal
IRQ enabled when copying the log buffers.
> + */
> + prb_for_each_record(seq, _rb_static, seq, ) {
> + text_size = add_to_rb(_rb_dynamic, );
> + if (text_size > free)
> + free = 0;
> + else
> + free -= text_size;
> + }
>
> if (seq != prb_next_seq(_rb_static)) {
> pr_err("dropped %llu messages\n",
> --- a/lib/nmi_backtrace.c
> +++ b/lib/nmi_backtrace.c
> @@ -75,12 +75,6 @@ void nmi_trigger_cpumask_backtrace(const cpumask_t *mask,
> touch_softlockup_watchdog();
> }
>
> - /*
> - * Force flush any remote buffers that might be stuck in IRQ context
> - * and therefore could not run their irq_work.
> - */
> - printk_safe_flush();
Sigh, this reminds me that the nmi_safe buffers serialized backtraces
from all CPUs.
I am afraid that we have to put back the spinlock into
nmi_cpu_backtrace(). It has been repeatedly added and removed depending
on whether the backtrace was printed into the main log buffer
or into the per-CPU buffers. Last time it was removed by
the commit 03fc7f9c99c1e7ae2925d ("printk/nmi: Prevent deadlock
when accessing the main log buffer in NMI").
It should be safe because there should not be any other locks in the
code path. Note that only one backtrace might be triggered at the same
time, see @backtrace_flag in nmi_trigger_cpumask_backtrace().
We _must_ serialize it somehow[*]. The lock in nmi_cpu_backtrace()
looks less evil than the nmi_safe machinery. nmi_safe() shrinks
too long backtraces, lose timestamps, needs to be explicitely
flushed here and there, is a non-trivial code.
[*] Non-serialized bactraces are real mess. Caller-id is visible
only on consoles or via syslogd interface. And it is not much
convenient.
I get this with "echo l >/proc/sysrq-trigger" and this patchset:
[ 95.642793] sysrq: Show backtrace of all active CPUs
[ 95.645202] NMI backtrace for cpu 3
[ 95.646778] CPU: 3 PID: 5095 Comm: bash Kdump: loaded Tainted: GW
5.11.0-default+ #231
[ 95.650397] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS
rel-1.12.0-59-gc9ba527-rebuilt.opensuse.org 04/01/2014
[ 95.656497] Call Trace:
[ 95.657937] dump_stack+0x88/0xab
[ 95.659888] nmi_cpu_backtrace+0xa4/0xc0
[ 95.661744] ? lapic_can_unplug_cpu+0xa0/0xa0
[ 95.663658] nmi_trigger_cpumask_backtrace+0xe6/0x120
[ 95.665657] arch_trigger_cpumask_backtrace+0x19/0x20
[ 95.667720] sysrq_handle_showallcpus+0x17/0x20
[ 95.670218] __handle_sysrq+0xe1/0x240
[ 95.672190] write_sysrq_trigger+0x51/0x60
[ 95.673993] proc_reg_write+0x62/0x90
[ 95.675319] vfs_write+0xed/0x380
[
Re: [PATCH printk v2 2/5] printk: remove safe buffers
On 2021-03-30, John Ogness wrote: > diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c > index e971c0a9ec9e..f090d6a1b39e 100644 > --- a/kernel/printk/printk.c > +++ b/kernel/printk/printk.c > @@ -1772,16 +1759,21 @@ static struct task_struct *console_owner; > static bool console_waiter; > > /** > - * console_lock_spinning_enable - mark beginning of code where another > + * console_lock_spinning_enable_irqsave - mark beginning of code where > another > * thread might safely busy wait > * > * This basically converts console_lock into a spinlock. This marks > * the section where the console_lock owner can not sleep, because > * there may be a waiter spinning (like a spinlock). Also it must be > * ready to hand over the lock at the end of the section. > + * > + * This disables interrupts because the hand over to a waiter must not be > + * interrupted until the hand over is completed (@console_waiter is cleared). > */ > -static void console_lock_spinning_enable(void) > +static void console_lock_spinning_enable_irqsave(unsigned long *flags) I missed the prototype change for the !CONFIG_PRINTK case, resulting in: linux/kernel/printk/printk.c:2707:3: error: implicit declaration of function ‘console_lock_spinning_enable_irqsave’; did you mean ‘console_lock_spinning_enable’? [-Werror=implicit-function-declaration] console_lock_spinning_enable_irqsave(); ^~~~ console_lock_spinning_enable Will be fixed for v3. (I have now officially added !CONFIG_PRINTK to my CI tests.) John Ogness ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
[PATCH printk v2 2/5] printk: remove safe buffers
With @logbuf_lock removed, the high level printk functions for
storing messages are lockless. Messages can be stored from any
context, so there is no need for the NMI and safe buffers anymore.
Remove the NMI and safe buffers.
Although the safe buffers are removed, the NMI and safe context
tracking is still in place. In these contexts, store the message
immediately but still use irq_work to defer the console printing.
Since printk recursion tracking is in place, safe context tracking
for most of printk is not needed. Remove it. Only safe context
tracking relating to the console lock is left in place. This is
because the console lock is needed for the actual printing.
Signed-off-by: John Ogness
---
Note: The follow-up patch removes the NMI tracking.
arch/powerpc/kernel/traps.c| 1 -
arch/powerpc/kernel/watchdog.c | 5 -
include/linux/printk.h | 10 -
kernel/kexec_core.c| 1 -
kernel/panic.c | 3 -
kernel/printk/internal.h | 17 --
kernel/printk/printk.c | 137 +-
kernel/printk/printk_safe.c| 333 +
lib/nmi_backtrace.c| 6 -
9 files changed, 56 insertions(+), 457 deletions(-)
diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c
index 3ec7b443fe6b..7d2b339afcb0 100644
--- a/arch/powerpc/kernel/traps.c
+++ b/arch/powerpc/kernel/traps.c
@@ -170,7 +170,6 @@ extern void panic_flush_kmsg_start(void)
extern void panic_flush_kmsg_end(void)
{
- printk_safe_flush_on_panic();
kmsg_dump(KMSG_DUMP_PANIC);
bust_spinlocks(0);
debug_locks_off();
diff --git a/arch/powerpc/kernel/watchdog.c b/arch/powerpc/kernel/watchdog.c
index af3c15a1d41e..8ae46c5945d0 100644
--- a/arch/powerpc/kernel/watchdog.c
+++ b/arch/powerpc/kernel/watchdog.c
@@ -181,11 +181,6 @@ static void watchdog_smp_panic(int cpu, u64 tb)
wd_smp_unlock();
- printk_safe_flush();
- /*
-* printk_safe_flush() seems to require another print
-* before anything actually goes out to console.
-*/
if (sysctl_hardlockup_all_cpu_backtrace)
trigger_allbutself_cpu_backtrace();
diff --git a/include/linux/printk.h b/include/linux/printk.h
index fe7eb2351610..2476796c1150 100644
--- a/include/linux/printk.h
+++ b/include/linux/printk.h
@@ -207,8 +207,6 @@ __printf(1, 2) void dump_stack_set_arch_desc(const char
*fmt, ...);
void dump_stack_print_info(const char *log_lvl);
void show_regs_print_info(const char *log_lvl);
extern asmlinkage void dump_stack(void) __cold;
-extern void printk_safe_flush(void);
-extern void printk_safe_flush_on_panic(void);
#else
static inline __printf(1, 0)
int vprintk(const char *s, va_list args)
@@ -272,14 +270,6 @@ static inline void show_regs_print_info(const char
*log_lvl)
static inline void dump_stack(void)
{
}
-
-static inline void printk_safe_flush(void)
-{
-}
-
-static inline void printk_safe_flush_on_panic(void)
-{
-}
#endif
extern int kptr_restrict;
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index a0b6780740c8..480d5f77ef4f 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -977,7 +977,6 @@ void crash_kexec(struct pt_regs *regs)
old_cpu = atomic_cmpxchg(_cpu, PANIC_CPU_INVALID, this_cpu);
if (old_cpu == PANIC_CPU_INVALID) {
/* This is the 1st CPU which comes here, so go ahead. */
- printk_safe_flush_on_panic();
__crash_kexec(regs);
/*
diff --git a/kernel/panic.c b/kernel/panic.c
index 332736a72a58..1f0df42f8d0c 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -247,7 +247,6 @@ void panic(const char *fmt, ...)
* Bypass the panic_cpu check and call __crash_kexec directly.
*/
if (!_crash_kexec_post_notifiers) {
- printk_safe_flush_on_panic();
__crash_kexec(NULL);
/*
@@ -271,8 +270,6 @@ void panic(const char *fmt, ...)
*/
atomic_notifier_call_chain(_notifier_list, 0, buf);
- /* Call flush even twice. It tries harder with a single online CPU */
- printk_safe_flush_on_panic();
kmsg_dump(KMSG_DUMP_PANIC);
/*
diff --git a/kernel/printk/internal.h b/kernel/printk/internal.h
index 51615c909b2f..6cc35c5de890 100644
--- a/kernel/printk/internal.h
+++ b/kernel/printk/internal.h
@@ -22,7 +22,6 @@ __printf(1, 0) int vprintk_deferred(const char *fmt, va_list
args);
void __printk_safe_enter(void);
void __printk_safe_exit(void);
-void printk_safe_init(void);
bool printk_percpu_data_ready(void);
#define printk_safe_enter_irqsave(flags) \
@@ -37,18 +36,6 @@ bool printk_percpu_data_ready(void);
local_irq_restore(flags); \
} while (0)
-#define printk_safe_enter_irq()\
- do {\
- local_irq_disable();\
- __printk_safe_enter();
