A vCPU can get preempted at any time on a multi-tenant and oversubscribed
host. Phantom Tracker 1. monitors vCPU preemptions and runnable-task
counts on the host using eBPF, 2. computes and exposes phantom_average and
idle_average, two time-weighted and paravirtualized scheduling metrics
representing, respectively, the number of guest vCPUs that are unable to
make progress because of host-side scheduling decisions and the number of
physical CPUs (pCPUs) that are idle on the host during a given scheduler
tick, and 3. packs both metrics, together with a host timestamp, into a
single watermark and exposes it to guest libgomp through eBPF-compatible
paravirtualized shared memory exposed to target VMs using a QEMU ivshmem
PCI device.
This patch introduces phantom_average and idle_average to libgomp's
dynamic degree-of-parallelism (DoP) adaptation, implemented with the help
of GOMP_DYNAMIC_POLICY, a new libgomp environment variable, which when set
to "pvsched" uses Juunansei to dynamically resolve the DoP returned by
gomp_dynamic_max_threads() at the start of each parallel region.
With the default libgomp OMP_DYNAMIC heuristic, the DoP is resolved from
the number of online CPUs and the host's 15-minute load average. Linux
updates the load average approximately every 5 seconds, and the
15-minute average responds slowly to sudden changes in system load.
Moreover, inside a target VM, the guest's load average reflects only the
guest's own scheduling state and does not account for host-side
scheduling interference affecting its vCPUs. Consequently, the default
heuristic cannot properly resolve the DoP.
The Juunansei policy changes this behavior by using the latest Phantom
Tracker watermark to adapt the DoP to the host's scheduling state. When
phantom_average becomes positive, indicating that guest vCPUs are
experiencing host-side scheduling contention, Juunansei reduces the DoP
by the number of phantoms detected. When no phantoms are detected,
Juunansei keeps the DoP steady and considers increasing it by
min_idle_average, the minimum idle_average observed over a stability
window of consecutive scheduler ticks. If this increase causes phantoms
to appear, indicating that the additional parallelism has introduced
host-side scheduling contention, Juunansei dynamically doubles the
stability window before attempting another increase. Otherwise, the
stability window resets to two ticks after a successful increase.
Juunansei relies on the ivshmem-backed watermark channel. If
initialization or watermark reading fails, GOMP_DYNAMIC_POLICY falls
back to the default OMP_DYNAMIC heuristic for the remainder of the run.
libgomp/ChangeLog:
* Makefile.am (libgomp_la_SOURCES): Add pvsched.c.
* Makefile.in: Regenerate.
* config/linux/proc.c: Include pvsched.h and sys/sdt.h.
(pv_gomp_dynamic_max_threads): New function. Resolve the DoP
from Juunansei's decision state.
(gomp_dynamic_max_threads): Call pv_gomp_dynamic_max_threads
when gomp_dynamic_policy == 1. Fall back to the default
OMP_DYNAMIC heuristic and disable the policy on failure.
* env.c (gomp_dynamic_policy): New variable.
(parse_dynamic_policy): New function. Parse GOMP_DYNAMIC_POLICY
(pvsched/default).
(initialize_env): Call parse_dynamic_policy and initialize
Juunansei when pvsched is selected.
* libgomp.h (gomp_dynamic_policy): Declare.
* pvsched.c: New file. Implement Juunansei's DoP adaptation:
state initialization, the first and same-tick decisions, active
and current DoP resolution from phantom_average and
idle_average, and policy cleanup on failure.
(juunansei_min_idle_avg): Guard against a non-positive
stability_requirement before declaring the watermark buffer.
* pvsched.h: Define the juunansei_knobs, juunansei_decision,
juunansei_context, juunansei_timestamps and juunansei_status
structures, and declare Juunansei's decision-state globals and
interfaces.
(read_watermark_at_slot, read_n_lastest_watermarks): New
functions. Read a single watermark slot, or the last N
watermarks in history order, from the shared-memory ring.
(read_latest_phantom_average): Decode phantom_average from the
packed watermark instead of returning it raw.
[1] Himadri Chhaya-Shailesh, "Scheduler Guided OpenMP Execution in
Cloud VMs" (PhD thesis), https://inria.hal.science/tel-05438117v2
---
libgomp/Makefile.am | 1 +
libgomp/Makefile.in | 5 +-
libgomp/config/linux/proc.c | 70 +++++++-
libgomp/env.c | 68 ++++++-
libgomp/libgomp.h | 1 +
libgomp/pvsched.c | 348 ++++++++++++++++++++++++++++++++++++
libgomp/pvsched.h | 127 ++++++++++++-
7 files changed, 608 insertions(+), 12 deletions(-)
create mode 100644 libgomp/pvsched.c
diff --git a/libgomp/Makefile.am b/libgomp/Makefile.am
index d649e1727ad..cc3339c3b06 100644
--- a/libgomp/Makefile.am
+++ b/libgomp/Makefile.am
@@ -67,6 +67,7 @@ libgomp_la_SOURCES = alloc.c atomic.c barrier.c critical.c
env.c error.c \
icv.c icv-device.c iter.c iter_ull.c loop.c loop_ull.c ordered.c \
parallel.c scope.c sections.c single.c task.c team.c work.c lock.c \
mutex.c proc.c sem.c bar.c ptrlock.c time.c fortran.c affinity.c \
+ pvsched.c \
target.c splay-tree.c libgomp-plugin.c oacc-parallel.c oacc-host.c \
oacc-init.c oacc-mem.c oacc-async.c oacc-plugin.c oacc-cuda.c \
priority_queue.c affinity-fmt.c teams.c allocator.c oacc-profiling.c \
diff --git a/libgomp/Makefile.in b/libgomp/Makefile.in
index 92709f25319..1ef5e33ddb5 100644
--- a/libgomp/Makefile.in
+++ b/libgomp/Makefile.in
@@ -215,7 +215,7 @@ am_libgomp_la_OBJECTS = alloc.lo atomic.lo barrier.lo
critical.lo \
loop.lo loop_ull.lo ordered.lo parallel.lo scope.lo \
sections.lo single.lo task.lo team.lo work.lo lock.lo mutex.lo \
proc.lo sem.lo bar.lo ptrlock.lo time.lo fortran.lo \
- affinity.lo target.lo splay-tree.lo libgomp-plugin.lo \
+ affinity.lo pvsched.lo target.lo splay-tree.lo libgomp-plugin.lo \
oacc-parallel.lo oacc-host.lo oacc-init.lo oacc-mem.lo \
oacc-async.lo oacc-plugin.lo oacc-cuda.lo priority_queue.lo \
affinity-fmt.lo teams.lo allocator.lo oacc-profiling.lo \
@@ -550,7 +550,7 @@ libgomp_la_SOURCES = alloc.c atomic.c barrier.c critical.c
env.c \
error.c icv.c icv-device.c iter.c iter_ull.c loop.c loop_ull.c \
ordered.c parallel.c scope.c sections.c single.c task.c team.c \
work.c lock.c mutex.c proc.c sem.c bar.c ptrlock.c time.c \
- fortran.c affinity.c target.c splay-tree.c libgomp-plugin.c \
+ fortran.c affinity.c pvsched.c target.c splay-tree.c libgomp-plugin.c \
oacc-parallel.c oacc-host.c oacc-init.c oacc-mem.c \
oacc-async.c oacc-plugin.c oacc-cuda.c priority_queue.c \
affinity-fmt.c teams.c allocator.c oacc-profiling.c \
@@ -780,6 +780,7 @@ distclean-compile:
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/priority_queue.Plo@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/proc.Plo@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ptrlock.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/pvsched.Plo@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/scope.Plo@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/sections.Plo@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/sem.Plo@am__quote@
diff --git a/libgomp/config/linux/proc.c b/libgomp/config/linux/proc.c
index d9d7e0cca29..15de8ad7b7f 100644
--- a/libgomp/config/linux/proc.c
+++ b/libgomp/config/linux/proc.c
@@ -31,9 +31,11 @@
#endif
#include "libgomp.h"
#include "proc.h"
+#include "pvsched.h"
#include <errno.h>
#include <stdlib.h>
#include <unistd.h>
+#include <sys/sdt.h>
#ifdef HAVE_GETLOADAVG
# ifdef HAVE_SYS_LOADAVG_H
# include <sys/loadavg.h>
@@ -184,10 +186,71 @@ get_num_procs (void)
the load average. Here I use the number of processors online
minus the 15 minute load average. */
+unsigned
+pv_gomp_dynamic_max_threads(void)
+{
+ struct juunansei_context context;
+ struct juunansei_decision *d;
+ struct juunansei_decision decision_snapshot;
+ struct juunansei_status status_snapshot;
+ int ret;
+
+ ret = juunansei_get_latest_context (&context);
+ if (ret < 0)
+ return -1;
+
+ if (first_decision)
+ {
+ d = juunansei_first_decision (&context);
+ status_snapshot = status;
+ DTRACE_PROBE3 (gomp, dop_decision, d, &status_snapshot, &context);
+ return (unsigned) d->curr_dop;
+ }
+
+ if (decision.timestamp <= timestamps.last_decision)
+ {
+ unsigned same_tick_dop = (unsigned) juunansei_same_tick_decision ();
+ decision_snapshot = decision;
+ status_snapshot = status;
+ DTRACE_PROBE3 (gomp, dop_decision, &decision_snapshot, &status_snapshot,
&context);
+ return same_tick_dop;
+ }
+
+ decision.active_dop = juunansei_active_dop_resolution ();
+ decision.curr_dop = juunansei_curr_dop_resolution (&context);
+
+ update_juunansei ();
+
+ decision_snapshot = decision;
+ status_snapshot = status;
+ DTRACE_PROBE3 (gomp, dop_decision, &decision_snapshot, &status_snapshot,
&context);
+
+ return (unsigned) decision.curr_dop;
+}
+
unsigned
gomp_dynamic_max_threads (void)
{
+ int dop_res;
unsigned n_onln, loadavg, nthreads_var = gomp_icv (false)->nthreads_var;
+ unsigned result;
+
+ if ( gomp_dynamic_policy == 1)
+ {
+ dop_res = pv_gomp_dynamic_max_threads();
+ if(dop_res < 0)
+ {
+ gomp_debug (0, "pvsched: pv_gomp_dynamic_max_threads failed "
+ "(dop_res=%d), falling back to default heuristic\n",
+ dop_res);
+ juunanei_cleanup();
+ }
+ else
+ {
+ DTRACE_PROBE1 (gomp, dyn_max_threads, dop_res);
+ return dop_res;
+ }
+ }
n_onln = get_num_procs ();
if (n_onln > nthreads_var)
@@ -206,9 +269,12 @@ gomp_dynamic_max_threads (void)
#endif
if (loadavg >= n_onln)
- return 1;
+ result = 1;
else
- return n_onln - loadavg;
+ result = n_onln - loadavg;
+
+ DTRACE_PROBE1 (gomp, dyn_max_threads, result);
+ return result;
}
int
diff --git a/libgomp/env.c b/libgomp/env.c
index 2a539c0221b..ee5f398e449 100644
--- a/libgomp/env.c
+++ b/libgomp/env.c
@@ -135,6 +135,7 @@ int gomp_pv_wait_policy = -1;
int guest_ivshmem_fd = -1;
void *guest_ivshmem_h2g;
struct ivshmem_header *pvsched_ivshmem_header;
+int gomp_dynamic_policy = -1;
/* Parse the GOMP_WAIT_POLICY environment variable. */
static bool
@@ -180,6 +181,45 @@ parse_pv_wait_policy (const char *name, int *pvalue)
return false;
}
+/* Parse the GOMP_DYNAMIC_POLICY environment variable. */
+static bool
+parse_dynamic_policy (const char *name, int *pvalue)
+{
+ char *env;
+
+ env = getenv (name);
+ if (env == NULL)
+ return false;
+
+ while (isspace ((unsigned char) *env))
+ ++env;
+ if (*env == '\0')
+ goto invalid;
+ if (strncasecmp (env, "pvsched", 7) == 0)
+ {
+ *pvalue = 1;
+ env += 7;
+ }
+ else if (strncasecmp (env, "default", 7) == 0)
+ {
+ *pvalue = 0;
+ env += 7;
+ }
+ else
+ goto invalid;
+
+ while (isspace ((unsigned char) *env))
+ ++env;
+ if (*env != '\0')
+ goto invalid;
+ return true;
+
+ invalid:
+ gomp_error ("Invalid value for environment variable %s", name);
+ *pvalue = -1;
+ return false;
+}
+
static void
print_env_var_error (const char *env, const char *val)
{
@@ -2261,7 +2301,8 @@ gomp_pvsched_init (void)
if (guest_ivshmem_fd < 0)
{
gomp_error ("failed to open /dev/guest_ivshmem: %s; falling back to "
- "the default OMP_WAIT_POLICY", strerror (errno));
+ "the default OMP_WAIT_POLICY/OMP_DYNAMIC heuristics",
+ strerror (errno));
return -1;
}
@@ -2270,7 +2311,8 @@ gomp_pvsched_init (void)
if (header == MAP_FAILED)
{
gomp_error ("failed to read ivshmem header: %s; falling back to "
- "the default OMP_WAIT_POLICY", strerror (errno));
+ "the default OMP_WAIT_POLICY/OMP_DYNAMIC heuristics",
+ strerror (errno));
close (guest_ivshmem_fd);
guest_ivshmem_fd = -1;
return -1;
@@ -2279,8 +2321,9 @@ gomp_pvsched_init (void)
MAP_SHARED, guest_ivshmem_fd, header->h2g_page_offset);
if (base == MAP_FAILED)
{
- gomp_error ("failed to map guest_ivshmem H2G pages: %s; "
- "falling back to the default OMP_WAIT_POLICY", strerror
(errno));
+ gomp_error ("failed to map guest_ivshmem H2G pages: %s; falling "
+ "back to the default OMP_WAIT_POLICY/OMP_DYNAMIC "
+ "heuristics", strerror (errno));
munmap (header, sizeof (struct ivshmem_header));
close (guest_ivshmem_fd);
guest_ivshmem_fd = -1;
@@ -2519,9 +2562,24 @@ initialize_env (void)
if(!parse_pv_wait_policy("GOMP_WAIT_POLICY", &gomp_pv_wait_policy))
gomp_pv_wait_policy = -1;
- if (gomp_pv_wait_policy == 2)
+ if (!parse_dynamic_policy ("GOMP_DYNAMIC_POLICY", &gomp_dynamic_policy))
+ gomp_dynamic_policy = -1;
+
+ /* Both GOMP_WAIT_POLICY=pvsched and GOMP_DYNAMIC_POLICY=pvsched read
+ Juunansei's watermarks through the same guest_ivshmem mapping, so
+ either one on its own is enough to bring it up; run the mapping once
+ here rather than tying it to GOMP_WAIT_POLICY alone. */
+ if (gomp_pv_wait_policy == 2 || gomp_dynamic_policy == 1)
gomp_wait_policy_init = gomp_pvsched_init ();
+ if (gomp_dynamic_policy == 1)
+ {
+ if (gomp_wait_policy_init < 0)
+ gomp_dynamic_policy = 0;
+ else
+ juunansei_initialize ();
+ }
+
if (!parse_spincount ("GOMP_SPINCOUNT", &gomp_spin_count_var))
{
diff --git a/libgomp/libgomp.h b/libgomp/libgomp.h
index cb175fe9ee7..9d06cadcb75 100644
--- a/libgomp/libgomp.h
+++ b/libgomp/libgomp.h
@@ -622,6 +622,7 @@ extern char *goacc_device_type;
extern int goacc_default_dims[GOMP_DIM_MAX];
extern int gomp_reverse_offload_threads;
extern int gomp_pv_wait_policy;
+extern int gomp_dynamic_policy;
enum gomp_task_kind
{
diff --git a/libgomp/pvsched.c b/libgomp/pvsched.c
new file mode 100644
index 00000000000..28e06d090ac
--- /dev/null
+++ b/libgomp/pvsched.c
@@ -0,0 +1,348 @@
+/* Copyright (C) 2026 Free Software Foundation, Inc.
+ Contributed by NchangRoy <[email protected]>,
+ Himadri CS <[email protected]>,
+ Andrea Righi <[email protected]>,
+ Tobias Burnus <[email protected]>,
+ Thomas Schwinge <[email protected]>, as part of GSoC 2026.
+
+ This file is part of the GNU Offloading and Multi Processing Library
+ (libgomp).
+
+ Libgomp is free software; you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3, or (at your option)
+ any later version.
+
+ Libgomp is distributed in the hope that it will be useful, but WITHOUT ANY
+ WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ more details.
+
+ Under Section 7 of GPL version 3, you are granted additional
+ permissions described in the GCC Runtime Library Exception, version
+ 3.1, as published by the Free Software Foundation.
+
+ You should have received a copy of the GNU General Public License and
+ a copy of the GCC Runtime Library Exception along with this program;
+ see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
+ <http://www.gnu.org/licenses/>. */
+
+/* Juunansei degree-of-parallelism decision logic for pv-barrier-sync.
+
+ Policy inspired by: Himadri Chhaya-Shailesh, "Scheduler Guided OpenMP
+ Execution in Cloud VMs" (PhD thesis),
+ https://inria.hal.science/tel-05438117v2 */
+
+#include "libgomp.h"
+#include "pvsched.h"
+
+extern int omp_get_num_procs (void);
+
+struct juunansei_knobs juunansei_knobs;
+struct juunansei_decision decision;
+struct juunansei_status status;
+struct juunansei_timestamps timestamps;
+int first_decision = 1;
+
+void
+juunansei_initialize ()
+{
+ decision = (struct juunansei_decision) {0};
+ status = (struct juunansei_status) {0};
+ timestamps = (struct juunansei_timestamps) {0};
+
+ status.total_vcpus = omp_get_num_procs ();
+ juunansei_knobs.global_phantoms_detected = 0;
+ juunansei_knobs.global_phantoms_increasing = 0;
+ juunansei_knobs.consuming_idle = 0;
+ juunansei_knobs.stability_requirement = 2;
+}
+
+
+struct juunansei_decision *
+juunansei_first_decision (struct juunansei_context *context)
+{
+ first_decision = 0;
+
+ if (context->phantom_average > 0)
+ decision.curr_dop = status.total_vcpus;
+ else
+ decision.curr_dop = context->min_idle_average;
+ if (decision.curr_dop < 1)
+ decision.curr_dop = 1;
+ decision.active_dop = decision.curr_dop;
+ update_juunansei ();
+ return &decision;
+}
+
+static inline int
+juunansei_min_idle_avg (pvsched_u64 * idle_avg)
+{
+ int n = juunansei_knobs.stability_requirement;
+ int ret, min_idle_avg;
+
+ if (n <= 0)
+ return -ENODATA;
+
+ pvsched_u64 msgs[n];
+
+ ret = read_n_lastest_watermarks (n, msgs);
+ if (ret < n)
+ return ret < 0 ? ret : -ENODATA;
+
+ min_idle_avg = (int) (msgs[0] & 0xFFFF);
+ for (int i = 1; i < n; i++)
+ {
+ int cur = (int) (msgs[i] & 0xFFFF);
+ if (cur < min_idle_avg)
+ min_idle_avg = cur;
+ }
+
+ *idle_avg = min_idle_avg;
+ return 0;
+}
+
+void
+juunanei_cleanup()
+{
+ gomp_debug (0, "pvsched: disabling GOMP_DYNAMIC_POLICY=pvsched for the "
+ "rest of this process\n");
+ gomp_dynamic_policy = 0;
+}
+
+static inline int
+juunansei_nr_extra_blocked (int *extra_blocked)
+{
+ pvsched_u64 last_two_watermarks[2];
+ int ret,curr_phant_avg,prev_phant_avg;
+ ret = read_n_lastest_watermarks(2, last_two_watermarks);
+ if (ret < 2)
+ {
+ gomp_debug (0, "pvsched: nr_extra_blocked: got %d/2 watermarks "
+ "(ret=%d)\n", ret < 0 ? 0 : ret, ret);
+ juunanei_cleanup ();
+ return ret < 0 ? ret : -ENODATA;
+ }
+ curr_phant_avg = (last_two_watermarks[0] >> 16) & 0xFFFF;
+ prev_phant_avg = (last_two_watermarks[1] >> 16) & 0xFFFF;
+ *extra_blocked = curr_phant_avg - prev_phant_avg;
+ return 0;
+}
+
+static inline int
+juunansei_global_phantoms_increasing()
+{
+ int n = juunansei_knobs.stability_requirement;
+ pvsched_u64 watermarks[n];
+ int ret;
+ int32_t p_newest, p_oldest;
+ int64_t t_newest, t_oldest, total_time_diff;
+
+ ret = read_n_lastest_watermarks (n, watermarks);
+ if (ret < n)
+ {
+ gomp_debug (0, "pvsched: global_phantoms_increasing: got %d/%d "
+ "watermarks (ret=%d)\n", ret < 0 ? 0 : ret, n, ret);
+ juunanei_cleanup ();
+ return ret < 0 ? ret : -ENODATA;
+ }
+
+ p_newest = (int32_t) ((watermarks[0] >> 16) & 0xFFFF);
+ t_newest = (int64_t) (uint32_t) (watermarks[0] >> 32);
+
+ p_oldest = (int32_t) ((watermarks[n - 1] >> 16) & 0xFFFF);
+ t_oldest = (int64_t) (uint32_t) (watermarks[n - 1] >> 32);
+
+ total_time_diff = t_newest - t_oldest;
+ if (total_time_diff == 0)
+ return 0;
+
+ juunansei_knobs.global_phantoms_increasing =
+ ((p_newest - p_oldest) > 0) ? 1 : 0;
+
+ return juunansei_knobs.global_phantoms_increasing;
+}
+
+static inline int
+juunansei_global_phantoms_detected()
+{
+ int n = juunansei_knobs.stability_requirement;
+ pvsched_u64 watermarks[n];
+ int ret, detected = 0;
+
+ ret = read_n_lastest_watermarks (n, watermarks);
+ if (ret < n)
+ {
+ gomp_debug (0, "pvsched: global_phantoms_detected: got %d/%d "
+ "watermarks (ret=%d)\n", ret < 0 ? 0 : ret, n, ret);
+ juunanei_cleanup ();
+ return ret < 0 ? ret : -ENODATA;
+ }
+
+ for (int i = 0; i < n; i++)
+ {
+ int32_t p = (int32_t) ((watermarks[i] >> 16) & 0xFFFF);
+ if (p > 0)
+ {
+ detected = 1;
+ break;
+ }
+ }
+
+ return detected;
+}
+
+int
+juunansei_get_latest_context (struct juunansei_context *ctxt)
+{
+ pvsched_u64 watermark, min_idle;
+ int ret;
+
+ ret = read_watermark_at_slot (pvsched_ivshmem_header->latest_slot,
&watermark);
+ if (ret < 0)
+ {
+ gomp_debug (0, "pvsched: read_watermark_at_slot(%u) failed (ret=%d)\n",
+ pvsched_ivshmem_header->latest_slot, ret);
+ juunanei_cleanup ();
+ return ret;
+ }
+
+ decision.host_watermark = watermark;
+ decision.timestamp = watermark >> 32;
+
+ ctxt->phantom_average = (int32_t) ((watermark >> 16) & 0xFFFF);
+ decision.phantom_average = ctxt->phantom_average;
+ ctxt->idle_average = (int32_t) (watermark & 0xFFFF);
+
+ ret = juunansei_min_idle_avg (&min_idle);
+ if (ret < 0)
+ {
+ gomp_debug (0, "pvsched: juunansei_min_idle_avg failed (ret=%d, "
+ "need %ld watermarks)\n", ret,
+ (long) juunansei_knobs.stability_requirement);
+ juunanei_cleanup ();
+ return ret;
+ }
+
+ ctxt->min_idle_average = (int32_t) min_idle;
+
+ juunansei_knobs.global_phantoms_detected =
juunansei_global_phantoms_detected ();
+ juunansei_knobs.global_phantoms_increasing =
juunansei_global_phantoms_increasing ();
+
+ return 0;
+}
+
+int
+juunansei_active_dop_resolution ()
+{
+ if (juunansei_global_phantoms_detected ())
+ {
+ if (juunansei_global_phantoms_increasing ())
+ {
+ int extra_blocked;
+
+ if (juunansei_nr_extra_blocked (&extra_blocked) < 0)
+ decision.active_dop = status.active_dop;
+ else
+ decision.active_dop = status.active_dop - extra_blocked;
+
+ if (decision.active_dop < 1)
+ decision.active_dop = 1;
+ }
+ else
+ decision.active_dop = status.active_dop;
+ }
+ else
+ decision.active_dop = status.curr_dop;
+
+ if (decision.active_dop < 1)
+ decision.active_dop = 1;
+
+ return decision.active_dop;
+}
+
+int
+juunansei_curr_dop_resolution (struct juunansei_context *context)
+{
+ if (juunansei_knobs.consuming_idle)
+ {
+ if (status.dop_before_idle_consumption > decision.active_dop)
+ {
+ int64_t max_stability
+ = (pvsched_ivshmem_header->h2g_page_size
+ / sizeof (struct hg_message)) - 1;
+
+ juunansei_knobs.stability_requirement *= 2;
+ if (juunansei_knobs.stability_requirement > max_stability)
+ juunansei_knobs.stability_requirement = max_stability;
+ }
+ else
+ juunansei_knobs.stability_requirement = 2;
+ }
+
+ if (context->phantom_average == 0 && context->idle_average == 0)
+ {
+ juunansei_knobs.consuming_idle = 0;
+ decision.curr_dop = decision.active_dop;
+ }
+ else if (context->phantom_average > 0)
+ {
+ juunansei_knobs.consuming_idle = 0;
+ decision.curr_dop = decision.active_dop - context->phantom_average;
+ if (decision.curr_dop < 1)
+ decision.curr_dop = 1;
+ }
+ else if (context->idle_average > 0)
+ {
+ if (juunansei_global_phantoms_detected ())
+ {
+ juunansei_knobs.consuming_idle = 0;
+ decision.curr_dop = decision.active_dop;
+ }
+ else if (context->min_idle_average > 0)
+ {
+ juunansei_knobs.consuming_idle = 1;
+ decision.curr_dop = decision.active_dop + context->min_idle_average;
+ if (decision.curr_dop > status.total_vcpus)
+ decision.curr_dop = status.total_vcpus;
+ }
+ else
+ {
+ juunansei_knobs.consuming_idle = 0;
+ decision.curr_dop = decision.active_dop;
+ }
+ }
+
+ if (decision.curr_dop < 1)
+ decision.curr_dop = 1;
+
+ if (juunansei_knobs.consuming_idle)
+ status.dop_before_idle_consumption = status.curr_dop;
+ else
+ status.dop_before_idle_consumption = decision.curr_dop;
+
+ return decision.curr_dop;
+}
+
+void
+update_juunansei ()
+{
+ decision.prev_dop = status.curr_dop;
+ status.prev_dop = status.curr_dop;
+ status.curr_dop = decision.curr_dop;
+ status.active_dop = decision.active_dop;
+ status.watermark_used_at_decision = decision.host_watermark;
+ timestamps.last_decision = decision.timestamp;
+}
+
+int
+juunansei_same_tick_decision ()
+{
+ decision.curr_dop = status.curr_dop;
+ decision.prev_dop = status.prev_dop;
+ decision.active_dop = status.curr_dop;
+
+ update_juunansei ();
+
+ return decision.curr_dop;
+}
diff --git a/libgomp/pvsched.h b/libgomp/pvsched.h
index 9ceec7df7fd..27eeda0b4a9 100644
--- a/libgomp/pvsched.h
+++ b/libgomp/pvsched.h
@@ -67,15 +67,136 @@ extern int guest_ivshmem_fd;
extern void *guest_ivshmem_h2g;
extern struct ivshmem_header *pvsched_ivshmem_header;
+#define WAKE_CONTEXT 0
+#define WAIT_CONTEXT 1
+
+struct juunansei_knobs {
+ unsigned long long consuming_idle : 1;
+ unsigned long long : 63;
+ uint8_t global_phantoms_detected;
+ uint8_t global_phantoms_increasing;
+ uint8_t _rsvd0[6];
+ int64_t stability_requirement;
+ uint8_t _pad[40];
+} __attribute__((aligned(64)));
+
+struct juunansei_decision {
+ uint64_t host_watermark;
+ uint64_t timestamp;
+ int32_t phantom_average;
+ int32_t decision_nr;
+ int32_t curr_dop;
+ int32_t prev_dop;
+ int32_t active_dop;
+ uint8_t padding[28];
+} __attribute__((packed, aligned(64)));
+
+
+struct juunansei_context {
+ int32_t phantom_average;
+ int32_t idle_average;
+ int32_t min_idle_average;
+ uint8_t padding[52];
+} __attribute__ ((aligned(64)));
+
+struct juunansei_timestamps {
+ uint64_t last_decision;
+ uint8_t padding[56];
+} __attribute__((packed, aligned(64)));
+
+struct juunansei_status {
+ uint64_t watermark_used_at_decision;
+ int64_t total_vcpus;
+ int64_t active_dop;
+ int64_t prev_dop;
+ int64_t curr_dop;
+ int64_t decision_count;
+ int64_t dop_before_idle_consumption;
+ uint64_t padding;
+} __attribute__((packed, aligned(64)));
+
+extern struct juunansei_knobs juunansei_knobs;
+extern int first_decision;
+extern struct juunansei_decision decision;
+extern struct juunansei_status status;
+extern struct juunansei_timestamps timestamps;
+
+extern void juunansei_initialize (void);
+
+extern struct juunansei_decision *
+juunansei_first_decision (struct juunansei_context *context);
+
+extern void update_juunansei ();
+
+extern int juunansei_active_dop_resolution (void);
+
+extern int juunansei_curr_dop_resolution (struct juunansei_context *context);
+
+extern int juunansei_get_latest_context (struct juunansei_context *ctxt);
+
+extern int juunansei_same_tick_decision (void);
+
+extern void juunanei_cleanup (void);
+
static inline int
read_latest_phantom_average (pvsched_u64 *avg)
{
+ pvsched_u64 watermark;
if (guest_ivshmem_h2g == NULL)
return -ENODEV;
- *avg = __atomic_load_n (&((struct hg_message *)
guest_ivshmem_h2g)[pvsched_ivshmem_header->latest_slot].msg,
- MEMMODEL_RELAXED);
+ watermark = __atomic_load_n (&((struct hg_message *)
guest_ivshmem_h2g)[pvsched_ivshmem_header->latest_slot].msg,
+ MEMMODEL_RELAXED);
+ *avg = (watermark >> 16) & 0xffff;
DTRACE_PROBE1 (gomp, phantom_average, *avg);
return 0;
}
-#endif
\ No newline at end of file
+static inline int
+read_watermark_at_slot (uint32_t slot, pvsched_u64 * msg)
+{
+ if (guest_ivshmem_h2g == NULL)
+ return -ENODEV;
+ *msg = __atomic_load_n (&((struct hg_message *) guest_ivshmem_h2g)[slot].msg,
+ MEMMODEL_RELAXED);
+ return 0;
+}
+
+static inline int
+read_n_lastest_watermarks (uint32_t n, pvsched_u64 *msgs)
+{
+ uint32_t nr_slots, latest_slot_pos_in_his, best_ts, slot, i;
+ pvsched_u64 watermark;
+
+ if (guest_ivshmem_h2g == NULL)
+ return -ENODEV;
+
+ nr_slots = pvsched_ivshmem_header->h2g_page_size / sizeof (struct
hg_message);
+ read_watermark_at_slot (pvsched_ivshmem_header->latest_slot, &watermark);
+ latest_slot_pos_in_his = 1;
+ best_ts = (uint32_t) (watermark >> 32);
+
+ for (slot = 1; slot < nr_slots; slot++)
+ {
+ uint32_t ts;
+
+ read_watermark_at_slot (slot, &watermark);
+ ts = (uint32_t) (watermark >> 32);
+
+ if (ts == best_ts)
+ {
+ latest_slot_pos_in_his = slot;
+ break;
+ }
+ }
+
+ slot = latest_slot_pos_in_his;
+ for (i = 0; i < n; i++)
+ {
+ read_watermark_at_slot (slot, &msgs[i]);
+ slot = (slot == 1) ? nr_slots - 1 : slot - 1;
+ }
+
+ return (int) n;
+}
+
+#endif
--
2.34.1
