Hi,
On 2026-11-04 09:36:03 -0500, Andres Freund wrote:
> Peter, after all of this, I'd like to backpatch CI back to where we used to
> have CI support. One of the things that makes that slightly more painful is
> that there already are divergences between 19 and master. I'd like to
> backport
>
> 2026-10-25 4075409de08 ci: Enable PCH in mingw cross job
> 2026-10-25 3ca57291ff3 ci: Convert CompilerWarnings job to meson
>
> to 19, to keep the divergences smaller.
> I think
> 2026-10-19 0f3b42ce806 Add missing dependencies to meson headerscheck target
>
> ought to be in 19 regardless of anything, given that meson headerscheck was
> added to 19?
I'll do those backpatches in the next hour, unless somebody protests soon.
> > @@ -154,7 +148,8 @@ jobs:
> > warn-if-not-opted-in:
> > name: Report if not opted into CI
> > if: ${{vars.PG_CI_ENABLED != '1'}}
> > - runs-on: ubuntu-slim
> > + # See the setup task for an explanation
> > + runs-on: ${{ case(vars.PG_CI_RUNS_ON_SETUP != '',
> > vars.PG_CI_RUNS_ON_SETUP, 'ubuntu-slim') }}
> > steps:
> > - name: Warn
> > env:
> > ...
> > + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '',
> > vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }}
>
> I'd love if there were a neater way to do this, but I don't know of any :(.
>
> I'll play around with it to see if I can come up with something neater,
> particularly the repetition of the variable name is annoying.
It turns out one can just do ${{ vars.PG_CI_RUNS_ON_LINUX || 'ubuntu-24.04' }}.
Not super well documented, but looks to be fairly widely used. Think that's a
good bit neater.
Also worked on making the image: simpler. There's now one format() template
(still a long line), but the individual tasks now are
- image: ${{ needs.setup.outputs.container_linux_ci_base
}}/linux-autoconf:${{ needs.setup.outputs.container_tag }}
+ image: ${{ format(needs.setup.outputs.container_image, 'linux-autoconf')
}}
which seems a lot easier to read to me.
Here's a mildly polished version:
- Adoped the simpler runs-on: and image: formulations
- An AI review raised the following, which I addressed:
- location: ${{github.workspace}} is suboptimal, because it means that the
directory will be searched for logs in case of a failure. Testing shows
that setup-msys2 will pick a sane directory, so I think just not
specifying a directory makes sense
- CHERE_INVOKING is now obsolete
- The socket dir could end up with a too long path. I don't entirely remember
how I ended up with the previous formulation, I think it had to do with
issues around the paths in the container being different than outside.
The new formulation of just creating it in the container seemed better
- added the discussion links, dropped references, as mentioned in my earlier
email
What do you think?
Greetings,
Andres Freund
>From f27a5cb938824bc974714b00856ee2b4a6fd522b Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Wed, 4 Nov 2026 11:52:22 -0500
Subject: [PATCH v15a 1/5] ci: Get build/test concurrency from environment
Previously we hardcoded the amount of concurrency in the CI definition. This
practice IIRC originated from a) using make, where the concurrency is not
sourced from the current system b) early cirrus-ci macos runners, which slowed
down a lot with full concurrency.
Hardcoding the concurrency is problematic as e.g. private repositories have
lower concurrency and larger github runners have more cores.
For most of the jobs we can rely on just using meson's and ninja's logic for
getting the current core count. For the autoconf build we can use the nproc
helper.
To make this at least somewhat understandable, I extended the existing sysinfo
step to show the number of cores (and made it a bit easier to understand the
information already printed).
Author: Andres Freund <[email protected]>
Co-authored-by: Nazir Bilal Yavuz <[email protected]>
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
Discussion: https://postgr.es/m/can55fz1bcwbswgcck2gqkyn6tzdxcw2hvk2m3qeiwanxunj...@mail.gmail.com
Backpatch-through: 19
---
.github/workflows/pg-ci.yml | 43 ++++++++++++++++++++++---------------
1 file changed, 26 insertions(+), 17 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index e8eb49b5f15..bf32729c703 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -40,14 +40,6 @@ env:
# concurrent jobs and retrying older runs have a chance of working.
CLONE_DEPTH: 500
- # At the moment all jobs use 4vcore runners, and none seems to benefit from
- # increasing concurrency further.
- BUILD_JOBS: 4
-
- # It's possible that some jobs benefit from an increased test concurrency,
- # but a default of 4 is a safe bet. Individual jobs can override.
- TEST_JOBS: 4
-
CCACHE_MAXSIZE: "250M"
CCACHE_DIR: ${{ github.workspace }}/ccache_dir
@@ -215,10 +207,18 @@ jobs:
- &nix_sysinfo_step
name: sysinfo
run: |
- id
- uname -a
+ echo "Number of cores: " $(nproc || sysctl hw.physicalcpu hw.logicalcpu)
+
+ echo "id: $(id)"
+ echo "uname: $(uname -a)"
+
+ echo ::group::ulimit
ulimit -a -H && ulimit -a -S
+ echo ::endgroup::
+
+ echo ::group::env
env
+ echo ::endgroup::
- name: Parse ci-os-only
id: os
@@ -352,7 +352,7 @@ jobs:
- name: Build
shell: *su_postgres_shell
run: &ninja_build_cmd |
- ninja -C build -j${{env.BUILD_JOBS}} ${{env.MBUILD_TARGET}}
+ ninja -C build ${{env.MBUILD_TARGET}}
ninja -C build -t missingdeps
# Decide if it's worth uploading a new version of the ccache cache. If
@@ -405,7 +405,7 @@ jobs:
meson test ${{env.MTEST_ARGS}} --suite setup --logbase setup || exit 1
echo ::endgroup::
- meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} --no-suite setup ${{env.MTEST_TARGET}}
+ meson test ${{env.MTEST_ARGS}} --no-suite setup ${{env.MTEST_TARGET}}
- &linux_collect_cores_step
name: Core backtraces
@@ -511,7 +511,7 @@ jobs:
- name: Build
shell: *su_postgres_shell
run: |
- make -s -j${BUILD_JOBS} world-bin
+ make -s -j$(nproc) world-bin
- *ccache_decide_save_step
- *ccache_save_step
@@ -519,7 +519,7 @@ jobs:
- name: Test world
shell: *su_postgres_shell
run: |
- make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS}
+ make -s ${CHECK} ${CHECKFLAGS} -j$(nproc)
- *linux_collect_cores_step
- *upload_logs_step
@@ -611,7 +611,7 @@ jobs:
build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start
# Run the tests supporting running against an already running
- meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} --setup running
+ meson test ${{env.MTEST_ARGS}} --setup running
build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop
@@ -893,9 +893,18 @@ jobs:
- name: Sysinfo
run: |
+ echo Number of logical processors: %NUMBER_OF_PROCESSORS%
+
+ echo codepage:
chcp
+
+ echo ::group::systeminfo
systeminfo
+ echo ::endgroup::
+
+ echo ::group::env
set
+ echo ::endgroup::
# The TAP tests build an initdb template under build/tmp_install and
# then `robocopy` it into per-test data directories. Robocopy with the
@@ -1203,7 +1212,7 @@ jobs:
build
echo "::endgroup::"
- ninja -C build --quiet -j${{env.BUILD_JOBS}} ${{env.MBUILD_TARGET}}
+ ninja -C build --quiet ${{env.MBUILD_TARGET}}
# gcc, cassert on, dtrace off
- name: gcc warnings + (cassert)
@@ -1281,7 +1290,7 @@ jobs:
build
echo "::endgroup::"
- ninja -C build --quiet -j${{env.BUILD_JOBS}} -k 0 \
+ ninja -C build --quiet -k 0 \
headerscheck cpluspluscheck
- *ccache_decide_save_step
base-commit: 8e997cb8c9ecc27e55e38bb42f1cefd5ef74c9a0
--
2.56.0.307.g50fe5cc63f
>From 08c7692eef887185f6818951528f4dc9a03b985c Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Wed, 4 Nov 2026 11:52:22 -0500
Subject: [PATCH v15a 2/5] ci: Make msys2 install smaller
We only needed git and diffutils installed via pacman because the already
installed tools were hidden by the login script resetting PATH. "Fix" that
instead by telling msys to leave the old PATH around.
Also don't install libbacktrace, it is not needed at the moment.
Author: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
Discussion: https://postgr.es/m/can55fz1bcwbswgcck2gqkyn6tzdxcw2hvk2m3qeiwanxunj...@mail.gmail.com
Backpatch-through: 19
---
.github/workflows/pg-ci.yml | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index bf32729c703..9d62db95188 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -1065,6 +1065,11 @@ jobs:
CCACHE_SLOPPINESS: pch_defines,time_macros
CCACHE_DEPEND: 1
+ # We don't want using an msys bash to "hide" all the other already
+ # installed tools, that would require us to install tools into msys that
+ # are already available otherwise.
+ MSYS2_PATH_TYPE: inherit
+
defaults:
run:
shell: 'D:\msys64\usr\bin\bash.exe --login -eo pipefail "{0}"'
@@ -1095,11 +1100,10 @@ jobs:
# MSYS2. It dynamically expands to the correct prefix for the active
# shell environment.
pacman -S --noconfirm --needed --asdeps \
- git bison flex diffutils \
+ bison flex \
${MINGW_PACKAGE_PREFIX}-ccache \
${MINGW_PACKAGE_PREFIX}-gcc \
${MINGW_PACKAGE_PREFIX}-icu \
- ${MINGW_PACKAGE_PREFIX}-libbacktrace \
${MINGW_PACKAGE_PREFIX}-libxml2 \
${MINGW_PACKAGE_PREFIX}-libxslt \
${MINGW_PACKAGE_PREFIX}-lz4 \
--
2.56.0.307.g50fe5cc63f
>From 28270d038bc30be7d0beaf5cf55c7e972c17088a Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Wed, 4 Nov 2026 11:52:22 -0500
Subject: [PATCH v15a 3/5] ci: Use setup-msys2 instead of relocating
preinstalled msys2
Previously we relocated the existing install, for performance reasons. However
that has three disadvantages:
1) moving the install and then installing the packages we need is slower than
just creating a new install
2) It hardcodes that D: is the fast drive, but that turns out to depend on the
type of runner used
3) We were not using any caching and therefore downloaded the same files over
and over. We could have added caching in the previous approach too, but
it'd have been extra work.
I previously prototyped handrolling the install, instead of using
msys2/setup-msys2, and that does turn out to be a bit faster, but it doesn't
include caching. By the time we add that, the overall complexity seems like
it'd be too big.
The other alternative would be to generate a downloadable release in the
pg-vm-images repo. That'd likely be even faster.
This removes the explicit use of CHERE_INVOKING as it is implied by using
setup-msys2.
Author: Andres Freund <[email protected]>
Co-authored-by: Nazir Bilal Yavuz <[email protected]>
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
Discussion: https://postgr.es/m/can55fz1bcwbswgcck2gqkyn6tzdxcw2hvk2m3qeiwanxunj...@mail.gmail.com
Backpatch-through: 19
---
.github/workflows/pg-ci.yml | 78 +++++++++++++++++--------------------
1 file changed, 36 insertions(+), 42 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index 9d62db95188..ac6709a9519 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -1053,8 +1053,6 @@ jobs:
TAR: "c:/windows/system32/tar.exe"
MSYS: winjitdebug
- CHERE_INVOKING: 1
- MSYSTEM: UCRT64
# Keep -Dnls explicitly disabled, as the number of files it creates
# causes a noticeable slowdown.
@@ -1065,55 +1063,51 @@ jobs:
CCACHE_SLOPPINESS: pch_defines,time_macros
CCACHE_DEPEND: 1
- # We don't want using an msys bash to "hide" all the other already
- # installed tools, that would require us to install tools into msys that
- # are already available otherwise.
- MSYS2_PATH_TYPE: inherit
-
defaults:
run:
- shell: 'D:\msys64\usr\bin\bash.exe --login -eo pipefail "{0}"'
+ shell: msys2 {0}
steps:
- *windows_disable_defender_step
- *window_setup_hosts_step
- *checkout_step
- # Relocate the preinstalled MSYS2 tree from C:\ (slow system disk) to
- # D:\ (faster ephemeral data disk). Every subsequent MSYS2 step uses
- # D:\msys64\usr\bin\bash.exe via the job's `defaults.run.shell`.
+ # The pre-existing msys install is on C:\, often a rather slow system
+ # disk, whereas the workspace is often on a faster, ephemeral disk.
+ # Using the pre-existing msys would often increase the total runtime of
+ # this task by ~15 minutes.
#
- # This reduces the total runtime of this task by ~15 minutes.
- #
- # robocopy returns 0-7 on success (with various "files copied" bits
- # set) and 8+ on real failure, so we have to translate its exit code.
- - name: Relocate MSYS2 to D
- shell: pwsh
- run: |
- robocopy C:\msys64 D:\msys64 /E /NJS /NJH /NFL /NDL /NP
- if ($LASTEXITCODE -ge 8) { exit $LASTEXITCODE }
- exit 0
-
- - name: Setup MSYS2
- run: |
- # ${MINGW_PACKAGE_PREFIX} is an environment variable used in the
- # MSYS2. It dynamically expands to the correct prefix for the active
- # shell environment.
- pacman -S --noconfirm --needed --asdeps \
- bison flex \
- ${MINGW_PACKAGE_PREFIX}-ccache \
- ${MINGW_PACKAGE_PREFIX}-gcc \
- ${MINGW_PACKAGE_PREFIX}-icu \
- ${MINGW_PACKAGE_PREFIX}-libxml2 \
- ${MINGW_PACKAGE_PREFIX}-libxslt \
- ${MINGW_PACKAGE_PREFIX}-lz4 \
- ${MINGW_PACKAGE_PREFIX}-make \
- ${MINGW_PACKAGE_PREFIX}-meson \
- ${MINGW_PACKAGE_PREFIX}-perl \
- ${MINGW_PACKAGE_PREFIX}-pkgconf \
- ${MINGW_PACKAGE_PREFIX}-readline \
- ${MINGW_PACKAGE_PREFIX}-zlib \
- ${MINGW_PACKAGE_PREFIX}-zstd
+ # Instead we use msys2/setup-msys2 to set up our own install. That's
+ # faster than moving the install from C:\, and also works when the
+ # workspace is not on a separate disk. setup-msys2 automatically chooses
+ # a path relative to the temp dir, which is not on C:\ for GitHub hosted
+ # runners.
+ - name: Install MSYS2
+ uses: msys2/setup-msys2@ec48f7c5447b3140e2b088413ae3a55687bccb6e # v2.33.0
+ with:
+ msystem: UCRT64
+ # We don't want using an msys bash to "hide" all the other already
+ # installed tools, that would require us to install tools into msys
+ # that are already available otherwise.
+ path-type: inherit
+ update: true
+ install: >-
+ bison flex
+ # name:p means MINGW_PACKAGE_PREFIX-only
+ pacboy: >-
+ ccache:p
+ gcc:p
+ icu:p
+ libxml2:p
+ libxslt:p
+ lz4:p
+ make:p
+ meson:p
+ perl:p
+ pkgconf:p
+ readline:p
+ zlib:p
+ zstd:p
- *nix_sysinfo_step
--
2.56.0.307.g50fe5cc63f
>From 51c2486088c0c8e7fcb87b6f5ea85f086fdac61e Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Wed, 4 Nov 2026 11:52:22 -0500
Subject: [PATCH v15a 4/5] ci: Make runs-on overridable
Previously what runners CI ran on was hardcoded. Unfortunately that makes it
harder to use faster or self hosted runners (e.g. in a private repository
where the runners are very slow, or self hosted runners for cfbot).
Fix that by allowing to override the runner types via repository/organization
variables. How to do that is documented in src/tools/ci/README.
Testing faster github hosted runners showed that we have some dependencies on
specific locations, fix most of those.
As part of this the "Determine enabled jobs" job moves to ubuntu-slim, which
the setup job already did. Normally this won't make a difference, but seems
more consistent.
Author: Andres Freund <[email protected]>
Co-authored-by: Nazir Bilal Yavuz <[email protected]>
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
Discussion: https://postgr.es/m/can55fz1bcwbswgcck2gqkyn6tzdxcw2hvk2m3qeiwanxunj...@mail.gmail.com
Backpatch-through: 19
---
.github/workflows/pg-ci.yml | 72 +++++++++++++++++++++++--------------
src/tools/ci/README | 21 +++++++++++
2 files changed, 66 insertions(+), 27 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index ac6709a9519..4446365ea40 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -106,12 +106,6 @@ env:
--with-uuid=ossp
--with-zstd
- # Centrally define the version of linux runners, to make it easier to
- # update. We don't just want to use ubuntu-latest, as it's not implausible
- # there will be breakage when that switches to the next ubuntu version.
- _LINUX_RUNS_ON: &linux_runs_on |
- ubuntu-24.04
-
# Debian Trixie containers used by all Linux jobs. Built by
# 'https://github.com/anarazel/pg-vm-images/'.
CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main
@@ -154,7 +148,13 @@ jobs:
warn-if-not-opted-in:
name: Report if not opted into CI
if: ${{vars.PG_CI_ENABLED != '1'}}
- runs-on: ubuntu-slim
+
+ # Can't use the output from the setup task here, so this has to be done
+ # differently. To avoid having the same logic it in the setup job (which
+ # also can't use its own outputs), have it use a yaml reference.
+ runs-on: &setup_runs_on |
+ ${{ vars.PG_CI_RUNS_ON_SETUP || 'ubuntu-slim' }}
+
steps:
- name: Warn
env:
@@ -184,15 +184,33 @@ jobs:
# Only run CI if repo owner opted in. If this task is skipped due to the
# if, none of it's depending tasks (i.e. the actual CI tasks) run either.
if: ${{vars.PG_CI_ENABLED == '1'}}
- runs-on: *linux_runs_on
+ runs-on: *setup_runs_on
timeout-minutes: 1
outputs:
+ # Are certain tasks enabled?
linux: ${{ steps.os.outputs.linux }}
macos: ${{ steps.os.outputs.macos }}
windows: ${{ steps.os.outputs.windows }}
mingw: ${{ steps.os.outputs.mingw }}
compilerwarnings: ${{ steps.os.outputs.compilerwarnings }}
sanitycheck: ${{ steps.os.outputs.sanitycheck }}
+
+ # What runners to run on:
+ #
+ # To allow over-riding what runners jobs use, we allow setting
+ # repository / org level variables influencing that choice on a
+ # per-host-operating-system basis.
+ #
+ # This also makes it easier to change the version of linux / windows
+ # used centrally. We don't just want to use ubuntu-latest, as it's not
+ # implausible there will be breakage when that switches to the next
+ # ubuntu version.
+ runs_on_linux: ${{ vars.PG_CI_RUNS_ON_LINUX || 'ubuntu-24.04' }}
+ runs_on_windows: ${{ vars.PG_CI_RUNS_ON_WINDOWS || 'windows-2022' }}
+ runs_on_macos: ${{ vars.PG_CI_RUNS_ON_MACOS || 'macos-15' }}
+
+ # Exports:
+ #
# Re-export workflow-level env vars that other jobs need to reference
# from contexts (e.g. `jobs.<id>.container.image`) where the `env`
# context is not available.
@@ -257,7 +275,7 @@ jobs:
if: |
!cancelled() &&
needs.setup.outputs.sanitycheck == 'true'
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
timeout-minutes: 15
container: &linux_ci_container
image: ${{ needs.setup.outputs.container_linux_ci }}
@@ -445,7 +463,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.linux == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
container: *linux_ci_container
timeout-minutes: 60
@@ -539,7 +557,7 @@ jobs:
name: Linux - Meson (32-bit)
needs: [setup, sanity-check]
if: *linux_job_if
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
container: *linux_ci_container
timeout-minutes: 60
env: *linux_env
@@ -631,7 +649,7 @@ jobs:
name: Linux - Meson (64-bit)
needs: [setup, sanity-check]
if: *linux_job_if
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
container: *linux_ci_container
timeout-minutes: 60
env: *linux_env
@@ -687,7 +705,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.macos == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: macos-15
+ runs-on: ${{ needs.setup.outputs.runs_on_macos }}
timeout-minutes: 60
env:
MACPORTS_CACHE: ${{ github.workspace }}/macports-cache
@@ -745,10 +763,10 @@ jobs:
macos_major=$(sw_vers -productVersion | sed 's/\..*//')
pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q)
script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh)
- echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT"
+ echo "key=macports-${macos_major}-${{runner.arch}}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT"
# It's faster to start with a partial cache for the same macos
- # version than from scratch
- echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT"
+ # version and architecture than from scratch
+ echo "restore-key=macports-${macos_major}-${{runner.arch}}-" >> "$GITHUB_OUTPUT"
- name: "MacPorts: Restore cache"
id: mp-restore
@@ -844,7 +862,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.windows == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: windows-2022
+ runs-on: ${{ needs.setup.outputs.runs_on_windows }}
timeout-minutes: 60
# As described at the top of the task, split the tests across two runners
@@ -860,7 +878,6 @@ jobs:
env:
# Avoid port conflicts between concurrent tap tests
PG_TEST_USE_UNIX_SOCKETS: 1
- PG_REGRESS_SOCK_DIR: 'd:\pgsock'
TAR: "c:/windows/system32/tar.exe"
MESON_FEATURES: >-
@@ -965,7 +982,8 @@ jobs:
- name: Install dependencies
shell: pwsh
run: |
- # meson is not preinstalled on windows-2022. Install via pip
+ # meson is not preinstalled, at least on windows-2022. Install via
+ # pip
echo ::group::pip
python -m pip install --upgrade meson
if (!$?) { throw 'cmdfail' }
@@ -995,9 +1013,12 @@ jobs:
Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.2 pg-loadbalancetest"
Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.3 pg-loadbalancetest"
- - name: Setup socket directory
+ - &windows_setup_socket_dir_step
+ name: Setup socket directory
shell: cmd
- run: mkdir ${{env.PG_REGRESS_SOCK_DIR}}
+ run: |
+ mkdir "%RUNNER_TEMP%\pgsock"
+ echo PG_REGRESS_SOCK_DIR=%RUNNER_TEMP%\pgsock>>"%GITHUB_ENV%"
- &windows_setup_debugger_step
name: Setup Windows debugger
@@ -1044,12 +1065,11 @@ jobs:
!cancelled() &&
needs.setup.outputs.mingw == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: windows-2022
+ runs-on: ${{ needs.setup.outputs.runs_on_windows }}
timeout-minutes: 60
env:
# Avoid port conflicts between concurrent tap tests
PG_TEST_USE_UNIX_SOCKETS: 1
- PG_REGRESS_SOCK_DIR: 'd:\pgsock'
TAR: "c:/windows/system32/tar.exe"
MSYS: winjitdebug
@@ -1121,9 +1141,7 @@ jobs:
perl -mIPC::Run -e 1
echo ::endgroup::
- - name: Setup socket directory
- shell: cmd
- run: mkdir ${{env.PG_REGRESS_SOCK_DIR}}
+ - *windows_setup_socket_dir_step
- *windows_setup_debugger_step
@@ -1165,7 +1183,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.compilerwarnings == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
timeout-minutes: 60
container:
image: ${{ needs.setup.outputs.container_linux_ci_docs }}
diff --git a/src/tools/ci/README b/src/tools/ci/README
index 642e518c296..9276d4a654c 100644
--- a/src/tools/ci/README
+++ b/src/tools/ci/README
@@ -75,3 +75,24 @@ messages. Currently the following controls are available:
Only runs CI on operating systems specified. This can be useful when
addressing portability issues affecting only a subset of platforms.
+
+
+Controlling which runners CI uses
+=================================
+
+By default each job runs on a GitHub-hosted runner appropriate for its
+operating system. This can be overridden on a per-operating-system basis by
+creating repository or organization variables, at
+https://github.com/<username>/<reponame>/settings/variables/actions
+
+This is useful to run on faster GitHub-hosted runners, or on self-hosted
+runners.
+
+The following variables are recognized. If a variable is unset or empty, a
+default runner suitable for that operating system is used:
+
+- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings
+- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs
+- PG_CI_RUNS_ON_MACOS: the macOS job
+- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and
+ job setup)
--
2.56.0.307.g50fe5cc63f
>From f9961681eb78fbfa06ffab4cadbc57f061280037 Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Wed, 4 Nov 2026 11:52:22 -0500
Subject: [PATCH v15a 5/5] ci: Move to per-job naming for containers
Previously we only had two types of containers, "ci" for everything but the
CompilerWarnings job and ci_docs that had the tools necessary to build the
docs as part of CompilerWarnings. But that was unnecessarily heavy for several
of the jobs. As the pull time is a noticeable performance factor, that's not
great.
Instead use containers that are named by their job. Today some of those are
the same, but after this change we can optimize the containers for their jobs
without a problem.
It probably would make sense to allow over-riding CONTAINER_REPO on a
per-repo/org basis, but that seems better done as a separate change.
Author: Andres Freund <[email protected]>
Co-authored-by: Nazir Bilal Yavuz <[email protected]>
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
Discussion: https://postgr.es/m/can55fz1bcwbswgcck2gqkyn6tzdxcw2hvk2m3qeiwanxunj...@mail.gmail.com
Backpatch-through: 19
---
.github/workflows/pg-ci.yml | 33 ++++++++++++++++++++++-----------
1 file changed, 22 insertions(+), 11 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index 4446365ea40..9e55f0fbc45 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -108,9 +108,12 @@ env:
# Debian Trixie containers used by all Linux jobs. Built by
# 'https://github.com/anarazel/pg-vm-images/'.
+ #
+ # The containers are named
+ # ghcr.io/anarazel/pg-vm-images/<branch>/linux_debian_ci/<debian version>/<ci-job>:<CONTAINER_TAG>
CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main
- CONTAINER_LINUX_CI: linux_debian_trixie_ci:latest
- CONTAINER_LINUX_CI_DOCS: linux_debian_trixie_ci_docs:latest
+ CONTAINER_LINUX_CI_NAME: linux_debian_ci/trixie
+ CONTAINER_TAG: latest
# The full set of OS / job selectors recognized by the `ci-os-only:`
# commit-message directive parsed in the `setup` job below.
@@ -199,7 +202,7 @@ jobs:
#
# To allow over-riding what runners jobs use, we allow setting
# repository / org level variables influencing that choice on a
- # per-host-operating-system basis.
+ # per-host-operating-system basis. See also src/tools/ci/README
#
# This also makes it easier to change the version of linux / windows
# used centrally. We don't just want to use ubuntu-latest, as it's not
@@ -214,8 +217,10 @@ jobs:
# Re-export workflow-level env vars that other jobs need to reference
# from contexts (e.g. `jobs.<id>.container.image`) where the `env`
# context is not available.
- container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }}
- container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }}
+ #
+ # For brevity, container_image is a format() template, {0} is the task
+ # specific image name.
+ container_image: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_NAME }}/{0}:${{ env.CONTAINER_TAG }}
steps:
# Anchor reused by other jobs further down. GitHub Actions supports YAML
@@ -277,8 +282,8 @@ jobs:
needs.setup.outputs.sanitycheck == 'true'
runs-on: ${{ needs.setup.outputs.runs_on_linux }}
timeout-minutes: 15
- container: &linux_ci_container
- image: ${{ needs.setup.outputs.container_linux_ci }}
+ container:
+ image: ${{ format(needs.setup.outputs.container_image, 'sanity-check') }}
# Options passed to all linux containers. Not all of the jobs need
# all of them, but it's easier to just define them centrally.
@@ -464,7 +469,9 @@ jobs:
needs.setup.outputs.linux == 'true' &&
needs.sanity-check.result != 'failure'
runs-on: ${{ needs.setup.outputs.runs_on_linux }}
- container: *linux_ci_container
+ container:
+ image: ${{ format(needs.setup.outputs.container_image, 'linux-autoconf') }}
+ options: *linux_container_options
timeout-minutes: 60
env: &linux_env
@@ -558,7 +565,9 @@ jobs:
needs: [setup, sanity-check]
if: *linux_job_if
runs-on: ${{ needs.setup.outputs.runs_on_linux }}
- container: *linux_ci_container
+ container:
+ image: ${{ format(needs.setup.outputs.container_image, 'linux-meson-32') }}
+ options: *linux_container_options
timeout-minutes: 60
env: *linux_env
@@ -650,7 +659,9 @@ jobs:
needs: [setup, sanity-check]
if: *linux_job_if
runs-on: ${{ needs.setup.outputs.runs_on_linux }}
- container: *linux_ci_container
+ container:
+ image: ${{ format(needs.setup.outputs.container_image, 'linux-meson-64') }}
+ options: *linux_container_options
timeout-minutes: 60
env: *linux_env
@@ -1186,7 +1197,7 @@ jobs:
runs-on: ${{ needs.setup.outputs.runs_on_linux }}
timeout-minutes: 60
container:
- image: ${{ needs.setup.outputs.container_linux_ci_docs }}
+ image: ${{ format(needs.setup.outputs.container_image, 'compiler-warnings') }}
env:
# Use larger ccache cache as this job compiles with multiple
# compilers / flag combinations.
--
2.56.0.307.g50fe5cc63f
