Libgomp is virtualization oblivious, which becomes a problem when
OpenMP applications runs inside an oversubscribed VM (more virtual
CPUs (vCPUs) than physical CPUs (pCPUs)). Recent research from Inria
[1] addresses this with the idea of "Phantom vCPUs". Inside QEMU/KVM
VMs, OpenMP threads run on the guest's vCPUs, and those vCPUs in
turn gets scheduled as ordinary tasks onto the host's pCPUs. The
guest scheduler decides which threads run on which vCPUs; the host
scheduler independently decides which vCPUs runs on which pCPUs and
for how long. Because these two levels of scheduling decisions happen
independently, a vCPU can get preempted by the host at an inconvenient
time for the guest application .The research defines problematic
vCPU preemptions and delayed vCPU wakeups on the host that hinder
the progress of the guest OpenMP applications as phantom vCPUs:
A vCPU is a phantom if it is currently waiting in the runqueue of a
host pCPU while the worker thread of the guest parallel application
running on it is stuck. The vCPUs that are able to successfully run
or are sleeping on the host are considered viable.

The same research proposes Phantom Tracker [2], which monitors
scheduling decisions made by both the host and guest schedulers and
aggregates them into two paravirtualized scheduling metrics:
phantom_average, which quantifies the impact of host contention on
the scheduling of a target VM's vCPUs at scheduler-tick granularity,
and idle_average, which represents the average number of idle pCPUs
on the host during the last scheduler tick.

My GSoC project [3] reimplements this algorithm using eBPF.

This RFC attempts to make Phantom Tracker usable in real-world
OpenMP cloud deployments by introducing two virtualization-aware
runtime policies to libgomp: pv-barrier-sync, which uses
phantom_average to decide whether to spin or block on a per-thread
and per-barrier basis at runtime; and Juunansei, which uses
phantom_average and idle_average to dynamically resolve the number
of threads spawned at the start of each parallel region.

[1] Himadri Chhaya-Shailesh, "Scheduler Guided OpenMP Execution in Cloud VMs" 
(PhD thesis), https://inria.hal.science/tel-05438117v2
[2] Phantom Tracker repo: https://github.com/himadrics/phantom-tracker
[3] GSoC Project Description: 
https://summerofcode.withgoogle.com/programs/2026/projects/iXbp42du

NchangRoy (4):
  libgomp: Implement GOMP_WAIT_POLICY to use pv-barrier-sync
  libgomp: Add tests and documentation for GOMP_WAIT_POLICY
  libgomp: Implement GOMP_DYNAMIC_POLICY to use Juunansei for dynamic
    DoP adaptation
  libgomp: Add tests and documentation for GOMP_DYNAMIC_POLICY

 gcc/doc/install.texi                          |   9 +
 libgomp/Makefile.am                           |   1 +
 libgomp/Makefile.in                           |   5 +-
 libgomp/config/linux/proc.c                   |  70 +++-
 libgomp/config/linux/wait.h                   |  35 +-
 libgomp/env.c                                 | 201 +++++++++-
 libgomp/libgomp.h                             |   2 +
 libgomp/libgomp.texi                          |  90 ++++-
 libgomp/pvsched.c                             | 348 ++++++++++++++++++
 libgomp/pvsched.h                             | 202 ++++++++++
 libgomp/team.c                                |   4 +
 .../libgomp.c/pvsched_dynamic_policy-1.c      |  20 +
 .../libgomp.c/pvsched_dynamic_policy-2.c      |  14 +
 .../libgomp.c/pvsched_dynamic_policy-3.c      |  19 +
 .../libgomp.c/pvsched_dynamic_policy-4.c      |  16 +
 .../libgomp.c/pvsched_wait_policy-1.c         |  20 +
 .../libgomp.c/pvsched_wait_policy-2.c         |  17 +
 .../libgomp.c/pvsched_wait_policy-3.c         |  23 ++
 .../libgomp.c/pvsched_wait_policy-4.c         |  25 ++
 .../libgomp.c/pvsched_wait_policy-5.c         |  21 ++
 20 files changed, 1130 insertions(+), 12 deletions(-)
 create mode 100644 libgomp/pvsched.c
 create mode 100644 libgomp/pvsched.h
 create mode 100644 libgomp/testsuite/libgomp.c/pvsched_dynamic_policy-1.c
 create mode 100644 libgomp/testsuite/libgomp.c/pvsched_dynamic_policy-2.c
 create mode 100644 libgomp/testsuite/libgomp.c/pvsched_dynamic_policy-3.c
 create mode 100644 libgomp/testsuite/libgomp.c/pvsched_dynamic_policy-4.c
 create mode 100644 libgomp/testsuite/libgomp.c/pvsched_wait_policy-1.c
 create mode 100644 libgomp/testsuite/libgomp.c/pvsched_wait_policy-2.c
 create mode 100644 libgomp/testsuite/libgomp.c/pvsched_wait_policy-3.c
 create mode 100644 libgomp/testsuite/libgomp.c/pvsched_wait_policy-4.c
 create mode 100644 libgomp/testsuite/libgomp.c/pvsched_wait_policy-5.c

-- 
2.34.1

Reply via email to