From: "Gautham R. Shenoy" <gautham.she...@amd.com> Introduce the notion of TIF_NOTIFY_IPI flag. When a processor in TIF_POLLING mode needs to process an IPI, the sender sets NEED_RESCHED bit in idle task's thread_info to pull the target out of idle and avoids sending an interrupt to the idle CPU. When NEED_RESCHED is set, the scheduler assumes that a new task has been queued on the idle CPU and calls schedule_idle(), however, it is not necessary that an IPI on an idle CPU will necessarily end up waking a task on the said CPU. To avoid spurious calls to schedule_idle() assuming an IPI on an idle CPU will always wake a task on the said CPU, TIF_NOTIFY_IPI will be used to pull a TIF_POLLING CPU out of idle.
Since the IPI handlers are processed before the call to schedule_idle(), schedule_idle() will be called only if one of the handlers have woken up a new task on the CPU and has set NEED_RESCHED. Add tif_notify_ipi() and current_clr_notify_ipi() helpers to test if TIF_NOTIFY_IPI is set in the current task's thread_info, and to clear it respectively. These interfaces will be used in subsequent patches as TIF_NOTIFY_IPI notion is integrated in the scheduler and in the idle path. [ prateek: Split the changes into a separate patch, add commit log ] Cc: Richard Henderson <richard.hender...@linaro.org> Cc: Ivan Kokshaysky <i...@jurassic.park.msu.ru> Cc: Matt Turner <matts...@gmail.com> Cc: Russell King <li...@armlinux.org.uk> Cc: Guo Ren <guo...@kernel.org> Cc: Michal Simek <mon...@monstr.eu> Cc: Dinh Nguyen <dingu...@kernel.org> Cc: Jonas Bonn <jo...@southpole.se> Cc: Stefan Kristiansson <stefan.kristians...@saunalahti.fi> Cc: Stafford Horne <sho...@gmail.com> Cc: "James E.J. Bottomley" <james.bottom...@hansenpartnership.com> Cc: Helge Deller <del...@gmx.de> Cc: Michael Ellerman <m...@ellerman.id.au> Cc: Nicholas Piggin <npig...@gmail.com> Cc: Christophe Leroy <christophe.le...@csgroup.eu> Cc: "Aneesh Kumar K.V" <aneesh.ku...@kernel.org> Cc: "Naveen N. Rao" <naveen.n....@linux.ibm.com> Cc: Yoshinori Sato <ys...@users.sourceforge.jp> Cc: Rich Felker <dal...@libc.org> Cc: John Paul Adrian Glaubitz <glaub...@physik.fu-berlin.de> Cc: "David S. Miller" <da...@davemloft.net> Cc: Thomas Gleixner <t...@linutronix.de> Cc: Ingo Molnar <mi...@redhat.com> Cc: Borislav Petkov <b...@alien8.de> Cc: Dave Hansen <dave.han...@linux.intel.com> Cc: "H. Peter Anvin" <h...@zytor.com> Cc: "Rafael J. Wysocki" <raf...@kernel.org> Cc: Daniel Lezcano <daniel.lezc...@linaro.org> Cc: Peter Zijlstra <pet...@infradead.org> Cc: Juri Lelli <juri.le...@redhat.com> Cc: Vincent Guittot <vincent.guit...@linaro.org> Cc: Dietmar Eggemann <dietmar.eggem...@arm.com> Cc: Steven Rostedt <rost...@goodmis.org> Cc: Ben Segall <bseg...@google.com> Cc: Mel Gorman <mgor...@suse.de> Cc: Daniel Bristot de Oliveira <bris...@redhat.com> Cc: Valentin Schneider <vschn...@redhat.com> Cc: Al Viro <v...@zeniv.linux.org.uk> Cc: Linus Walleij <linus.wall...@linaro.org> Cc: Ard Biesheuvel <a...@kernel.org> Cc: Andrew Donnellan <a...@linux.ibm.com> Cc: Nicholas Miehlbradt <nicho...@linux.ibm.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Arnd Bergmann <a...@arndb.de> Cc: Josh Poimboeuf <jpoim...@kernel.org> Cc: "Kirill A. Shutemov" <kirill.shute...@linux.intel.com> Cc: Rick Edgecombe <rick.p.edgeco...@intel.com> Cc: Tony Battersby <to...@cybernetics.com> Cc: Brian Gerst <brge...@gmail.com> Cc: Tim Chen <tim.c.c...@linux.intel.com> Cc: David Vernet <v...@manifault.com> Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Cc: linux-al...@vger.kernel.org Cc: linux-arm-ker...@lists.infradead.org Cc: linux-c...@vger.kernel.org Cc: linux-openr...@vger.kernel.org Cc: linux-par...@vger.kernel.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux...@vger.kernel.org Cc: sparcli...@vger.kernel.org Cc: linux...@vger.kernel.org Signed-off-by: Gautham R. Shenoy <gautham.she...@amd.com> Co-developed-by: K Prateek Nayak <kprateek.na...@amd.com> Signed-off-by: K Prateek Nayak <kprateek.na...@amd.com> --- include/linux/thread_info.h | 43 +++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/include/linux/thread_info.h b/include/linux/thread_info.h index 9ea0b28068f4..1e10dd8c0227 100644 --- a/include/linux/thread_info.h +++ b/include/linux/thread_info.h @@ -195,6 +195,49 @@ static __always_inline bool tif_need_resched(void) #endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H */ +#ifdef TIF_NOTIFY_IPI + +#ifdef _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H + +static __always_inline bool tif_notify_ipi(void) +{ + return arch_test_bit(TIF_NOTIFY_IPI, + (unsigned long *)(¤t_thread_info()->flags)); +} + +static __always_inline void current_clr_notify_ipi(void) +{ + arch_clear_bit(TIF_NOTIFY_IPI, + (unsigned long *)(¤t_thread_info()->flags)); +} + +#else + +static __always_inline bool tif_notify_ipi(void) +{ + return test_bit(TIF_NOTIFY_IPI, + (unsigned long *)(¤t_thread_info()->flags)); +} + +static __always_inline void current_clr_notify_ipi(void) +{ + clear_bit(TIF_NOTIFY_IPI, + (unsigned long *)(¤t_thread_info()->flags)); +} + +#endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H */ + +#else /* !TIF_NOTIFY_IPI */ + +static __always_inline bool tif_notify_ipi(void) +{ + return false; +} + +static __always_inline void current_clr_notify_ipi(void) { } + +#endif /* TIF_NOTIFY_IPI */ + #ifndef CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES static inline int arch_within_stack_frames(const void * const stack, const void * const stackend, -- 2.34.1