On Mon, 31 Aug 2026 at 04:19, Shrikanth Hegde <[email protected]> wrote: > > Hi Yury, > Thanks for going through and your suggestions!. > > On 8/30/26 1:01 AM, Yury Norov wrote: > > On Fri, Aug 28, 2026 at 09:51:47AM +0200, Dietmar Eggemann wrote: > >> On 25.08.26 12:38, Shrikanth Hegde wrote: > >>> When possible, try to choose a preferred CPU. > >>> > >>> This is essential to maintain user affinities when preferred > >>> CPUs change. A task pinned on a non-preferred CPU should continue > >>> to run there, since this is a non-user triggered event. > >>> > >>> If a CPU is non-preferred and the task can run on other CPUs which are > >>> currently preferred, then choose a preferred CPU instead. > >>> This is decided by checking if cpus_ptr and cpu_preferred_mask > >>> intersect or not. If yes, then the task has other preferred CPUs. > >>> > >>> The push task mechanism uses a stopper thread which calls > >>> select_fallback_rq() and uses this mechanism to pick a preferred CPU. > >>> > >>> This takes care of the wakeup path for FAIR tasks too. > >>> is_cpu_allowed() is called to ensure wakeups happen on preferred CPUs. > >>> With that, additional checks in available_idle_cpu() are not necessary. > >>> > >>> Ignore the preferred CPU state if a task's affinity is changing and > >>> its new mask no longer includes the CPU it is currently running on. > >>> This ensures migration_cpu_stop() does not abort, preventing the task > >>> from being stranded outside its allowed affinity. > >>> > >>> Account for tasks with architecture-specific CPU masks > >>> (e.g., 32-bit tasks on arm64). For such tasks, explicitly check against > >>> the arch-allowed CPUs to determine if any of the preferred CPUs are > >>> actually valid. > >>> > >>> For the majority of cases, this would still keep select_fallback_rq() > >>> as O(N). cpumask_intersects(), which is O(N), is called only if > >>> !cpu_preferred. The task running there is expected to move out. > >>> Subsequently, it should run on a preferred CPU. This becomes O(N**2) > >>> only for tasks pinned solely to non-preferred CPUs. That is a rare case. > >>> > >>> Overhead is minimal when the CPU is preferred. > >>> > >>> Signed-off-by: Shrikanth Hegde <[email protected]> > >>> --- > >>> kernel/sched/core.c | 41 +++++++++++++++++++++++++++++++++++++++-- > >>> 1 file changed, 39 insertions(+), 2 deletions(-) > >>> > >>> diff --git a/kernel/sched/core.c b/kernel/sched/core.c > >>> index a45f7c308329..f71317fe281d 100644 > >>> --- a/kernel/sched/core.c > >>> +++ b/kernel/sched/core.c > >>> @@ -2494,6 +2494,35 @@ static inline bool rq_has_pinned_tasks(struct rq > >>> *rq) > >>> return rq->nr_pinned; > >>> } > >>> > >>> +static inline bool task_can_sched_on_preferred(int cpu, struct > >>> task_struct *p) > >>> +{ > >>> + const struct cpumask *valid_mask; > >>> + int i; > >>> + > >>> + if (cpu_preferred(cpu)) > >>> + return false; > >>> + > >>> + /* Only FAIR tasks honor preferred CPU state */ > >>> + if (unlikely(p->sched_class != &fair_sched_class)) > >>> + return false; > >>> + > >>> + /* Ignore preferred state if task affinity is changing */ > >>> + if (unlikely(!cpumask_test_cpu(task_cpu(p), p->cpus_ptr))) > >>> + return false; > >>> + > >>> + valid_mask = task_cpu_possible_mask(p); > >>> + if (likely(valid_mask == cpu_possible_mask)) > >>> + return cpumask_intersects(p->cpus_ptr, cpu_preferred_mask); > >>> + > >>> + /* Tasks with arch-specific CPU masks. e.g. 32-bit tasks on arm64. */ > >>> + for_each_cpu_and(i, p->cpus_ptr, cpu_preferred_mask) { > >>> + if (cpumask_test_cpu(i, valid_mask)) > >>> + return true; > >>> + } > >> > >> Looking more into this, there might be a window in 64-32-bit execve() > >> for 32bit EL0 tasks on Arm64 (w/ allow_mismatched_32bit_el0 command line > >> option). > >> > >> The time before arch_setup_new_exec() calls > >> force_compatible_cpus_allowed_ptr() to restrict CPU affinity for those > >> tasks. > >> > >> Let me run more test on this ... > >> > >> Why not simply: > >> > >> - return cpumask_intersects(p->cpus_ptr, cpu_preferred_mask); > >> + return cpumask_first_and_and(p->cpus_ptr, cpu_preferred_mask, > >> + task_cpu_possible_mask(p)) < nr_cpu_ids; > >> > >> IMHO, you want to know whether there is at least one CPU that belongs to > >> all three CPU masks? > > > > Yeah, the cpumask_first_and_and() would replace the for-loop more > > effectively, but I'd suggest introducing the new helper: > > > > > > return cpumask_intersects_and(p->cpus_ptr, cpu_preferred_mask, > > task_cpu_possible_mask(p)); > > > > > Ack. However, we will only need this change if ARM64 wants to enable > this driver right now. If not, we can go back to the earlier > cpumask_intersects(), and the 3-way intersection can be added > later when the ARM ecosystem enables the feature. > > > Dietmar/Vincent, > Do you think it makes sense to enable the driver on ARM64 now? > Or you think it is better to delay it and once the feature is stable > ARM ecosystem can enable it?
It's always better to support all arch by default, unless something is missing which is not the case here. > > > This would also highlight the intention better - we're looking for > > intersection, and call the 'intersects' function. > > > > I don't like how this patch plays with likely() macro. > > Possible == task_possible condition is surely likely for x86, but is > > always unlikely for aarch64/el0-32 tasks. This would lead to suboptimal > > code generation on aarch64. > > > > The approach I've suggested also worsen performance because of a > > possibly unnecessary traversing of the cpu_possible_mask. > > > > If task_can_sched_on_preferred() is really a performance critical piece > > of code, we can invent arch_task_can_sched_on_preferred() to avoid it. > > > > It is called only on non-preferred CPU and unless pinned, tasks would have > moved > out of it. So we are okay here I guess. If it really pops up, then we can do > the > above idea. > > > -- > > > > On general side, the governor is really tested in 2 configurations: > > PPC+powervm and x86+kvm; and there's clearly an interest from XEN and > > ARM engineers. > > > > Maybe, to stay on safe side, we'd enable the feature where it's actually > > tested? I suggested it when DOM0 case was revealed, and now we've got > > the 2nd corner case from arm64 compat tasks. > > Indeed. It would safer and that way it would get more testing before > arch/hypervisor > enables it. > > The design is intended to work on architectures that provide steal-time > accounting, but each architecture and hypervisor combination should be > validated > before enabling it. > > > > > The advantages of this approach are: > > > > - faster adoption of the existing code for the tested architectures; > > - delegate arch/vm support to the domain professionals; > > - delay arch/vm support decisions to the later phase of adoption, when > > the API is better stabilized. > > > > Thanks, > > Yury > > So far, I have PPC_SPLPAR, S390, X86_64 in the Kconfig list. > I will wait to hear from Dietmar/Vincent about ARM.
