Threads currently have a _migration_lock_counter which, when positive, prevent the thread from being migrated to a different CPU. Using sched::thread::pin() can make this counter permanently positive, but we cannot distinguish it from a temporary increase due to migrate_disable(). In the future, to support pinning and re-pinning of other threads, we want to make such distinction.
So this patch adds a _pinned flag for a thread. When true, the thread was permanently pinned with pin(). _pinned being set to true also explains an increase of 1 in _migration_lock_counter. In the future, the "_pinned" boolean flag should be replaced by a bitmask of CPUs which this thread is allowed to be on, so that a thread can be pinned to a set of CPUs instead of just one. Signed-off-by: Nadav Har'El <[email protected]> --- include/osv/sched.hh | 7 +++++++ core/sched.cc | 16 ++++++++++------ 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/include/osv/sched.hh b/include/osv/sched.hh index 1db23af..e79d022 100644 --- a/include/osv/sched.hh +++ b/include/osv/sched.hh @@ -630,6 +630,13 @@ private: std::unique_ptr<detached_state> _detached_state; attr _attr; int _migration_lock_counter; + // _migration_lock_counter being set may be temporary, but if _pinned + // is true, it was permanently incremented by 1 by sched::thread::pin(). + // In the future, we should replace this boolean _pinned by a bitmask + // of allowed cpus for this thread (for full support of + // sched_setaffinity()), and the load balancer should consult this bitmask + // to decide to which cpus a thread may migrate. + bool _pinned; arch_thread _arch; unsigned int _id; std::atomic<bool> _interrupted; diff --git a/core/sched.cc b/core/sched.cc index e0fd264..344b188 100644 --- a/core/sched.cc +++ b/core/sched.cc @@ -482,18 +482,20 @@ unsigned cpu::load() void thread::pin(cpu *target_cpu) { thread &t = *current(); - // We want to wake this thread on the target CPU, but can't do this while - // it is still running on this CPU. So we need a different thread to - // complete the wakeup. We could re-used an existing thread (e.g., the - // load balancer thread) but a "good-enough" dirty solution is to - // temporarily create a new ad-hoc thread, "wakeme" - if (!t._migration_lock_counter) { + if (!t._pinned) { + // _pinned comes with a +1 increase to _migration_counter. migrate_disable(); + t._pinned = true; } cpu *source_cpu = cpu::current(); if (source_cpu == target_cpu) { return; } + // We want to wake this thread on the target CPU, but can't do this while + // it is still running on this CPU. So we need a different thread to + // complete the wakeup. We could re-used an existing thread (e.g., the + // load balancer thread) but a "good-enough" dirty solution is to + // temporarily create a new ad-hoc thread, "wakeme". bool do_wakeme = false; thread wakeme([&] () { wait_until([&] { return do_wakeme; }); @@ -764,6 +766,7 @@ thread::thread(std::function<void ()> func, attr attr, bool main, bool app) , _detached_state(new detached_state(this)) , _attr(attr) , _migration_lock_counter(0) + , _pinned(false) , _id(0) , _cleanup([this] { delete this; }) , _app(app) @@ -831,6 +834,7 @@ thread::thread(std::function<void ()> func, attr attr, bool main, bool app) if (_attr._pinned_cpu) { ++_migration_lock_counter; + _pinned = true; } if (main) { -- 2.5.5 -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
