From: "Gautham R. Shenoy" <[email protected]> Introduce the notion of TIF_NOTIFY_IPI flag. When a processor in TIF_POLLING mode needs to process an IPI, the sender sets NEED_RESCHED bit in idle task's thread_info to pull the target out of idle and avoids sending an interrupt to the idle CPU. When NEED_RESCHED is set, the scheduler assumes that a new task has been queued on the idle CPU and calls schedule_idle(), however, it is not necessary that an IPI on an idle CPU will necessarily end up waking a task on the said CPU. To avoid spurious calls to schedule_idle() assuming an IPI on an idle CPU will always wake a task on the said CPU, TIF_NOTIFY_IPI will be used to pull a TIF_POLLING CPU out of idle.
Since the IPI handlers are processed before the call to schedule_idle(), schedule_idle() will be called only if one of the handlers have woken up a new task on the CPU and has set NEED_RESCHED. Add tif_notify_ipi() and current_clr_notify_ipi() helpers to test if TIF_NOTIFY_IPI is set in the current task's thread_info, and to clear it respectively. These interfaces will be used in subsequent patches as TIF_NOTIFY_IPI notion is integrated in the scheduler and in the idle path. [ prateek: Split the changes into a separate patch, add commit log ] Cc: Richard Henderson <[email protected]> Cc: Ivan Kokshaysky <[email protected]> Cc: Matt Turner <[email protected]> Cc: Russell King <[email protected]> Cc: Guo Ren <[email protected]> Cc: Michal Simek <[email protected]> Cc: Dinh Nguyen <[email protected]> Cc: Jonas Bonn <[email protected]> Cc: Stefan Kristiansson <[email protected]> Cc: Stafford Horne <[email protected]> Cc: "James E.J. Bottomley" <[email protected]> Cc: Helge Deller <[email protected]> Cc: Michael Ellerman <[email protected]> Cc: Nicholas Piggin <[email protected]> Cc: Christophe Leroy <[email protected]> Cc: "Aneesh Kumar K.V" <[email protected]> Cc: "Naveen N. Rao" <[email protected]> Cc: Yoshinori Sato <[email protected]> Cc: Rich Felker <[email protected]> Cc: John Paul Adrian Glaubitz <[email protected]> Cc: "David S. Miller" <[email protected]> Cc: Thomas Gleixner <[email protected]> Cc: Ingo Molnar <[email protected]> Cc: Borislav Petkov <[email protected]> Cc: Dave Hansen <[email protected]> Cc: "H. Peter Anvin" <[email protected]> Cc: "Rafael J. Wysocki" <[email protected]> Cc: Daniel Lezcano <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Juri Lelli <[email protected]> Cc: Vincent Guittot <[email protected]> Cc: Dietmar Eggemann <[email protected]> Cc: Steven Rostedt <[email protected]> Cc: Ben Segall <[email protected]> Cc: Mel Gorman <[email protected]> Cc: Daniel Bristot de Oliveira <[email protected]> Cc: Valentin Schneider <[email protected]> Cc: Al Viro <[email protected]> Cc: Linus Walleij <[email protected]> Cc: Ard Biesheuvel <[email protected]> Cc: Andrew Donnellan <[email protected]> Cc: Nicholas Miehlbradt <[email protected]> Cc: Andrew Morton <[email protected]> Cc: Arnd Bergmann <[email protected]> Cc: Josh Poimboeuf <[email protected]> Cc: "Kirill A. Shutemov" <[email protected]> Cc: Rick Edgecombe <[email protected]> Cc: Tony Battersby <[email protected]> Cc: Brian Gerst <[email protected]> Cc: Tim Chen <[email protected]> Cc: David Vernet <[email protected]> Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Signed-off-by: Gautham R. Shenoy <[email protected]> Co-developed-by: K Prateek Nayak <[email protected]> Signed-off-by: K Prateek Nayak <[email protected]> --- include/linux/thread_info.h | 43 +++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/include/linux/thread_info.h b/include/linux/thread_info.h index 9ea0b28068f4..1e10dd8c0227 100644 --- a/include/linux/thread_info.h +++ b/include/linux/thread_info.h @@ -195,6 +195,49 @@ static __always_inline bool tif_need_resched(void) #endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H */ +#ifdef TIF_NOTIFY_IPI + +#ifdef _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H + +static __always_inline bool tif_notify_ipi(void) +{ + return arch_test_bit(TIF_NOTIFY_IPI, + (unsigned long *)(¤t_thread_info()->flags)); +} + +static __always_inline void current_clr_notify_ipi(void) +{ + arch_clear_bit(TIF_NOTIFY_IPI, + (unsigned long *)(¤t_thread_info()->flags)); +} + +#else + +static __always_inline bool tif_notify_ipi(void) +{ + return test_bit(TIF_NOTIFY_IPI, + (unsigned long *)(¤t_thread_info()->flags)); +} + +static __always_inline void current_clr_notify_ipi(void) +{ + clear_bit(TIF_NOTIFY_IPI, + (unsigned long *)(¤t_thread_info()->flags)); +} + +#endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H */ + +#else /* !TIF_NOTIFY_IPI */ + +static __always_inline bool tif_notify_ipi(void) +{ + return false; +} + +static __always_inline void current_clr_notify_ipi(void) { } + +#endif /* TIF_NOTIFY_IPI */ + #ifndef CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES static inline int arch_within_stack_frames(const void * const stack, const void * const stackend, -- 2.34.1
