Hi, Thank you for looking into this!
On Wed, 27 May 2026 at 21:10, Andres Freund <[email protected]> wrote: > > > Here is the v2, I took Jelte's patch and reviewed & merged it with my > > patch. Updates and questions are: > > > > 1- I continued to use Jelte's container method (Linux tasks only for > > now, BSD tasks will be included in the future) because I think that is > > the future-proof way since we might want to generate our container > > images in the future. Also, up-to-date Debian images can be tested > > with this way; otherwise we would need to use Ubuntu 24.04. > > Good. > > > > 2- io_uring tests work on the Linux Meson task. > > Is there a reason to not just do that for all the tasks? I might word it incorrectly. I meant that Linux meson tests use: PG_TEST_INITDB_EXTRA_OPTS: >- -c io_method=io_uring and that wasn't working before, now it works. I guess we have this only on Linux because we wanted to test io_method=worker in the other tasks. > > 3- I didn't put commands to helper scripts for now. I think it is a > > good thing to have a helper script but it would be better to have this > > helper script after the first version is committed since it can extend > > the timeline. Also, I found that having all commands in one file makes > > debugging easier. > > Hm. I'm a bit worried about this getting pretty unmaintainable, due to the > repetition. I think at least we need to use yaml anchors to deduplicate some > steps. Github Actions added support of yaml anchors last year but unfortunately they don't support merge keys. Related information: [1]. > > 4- FreeBSD task has these options: > > > > PG_TEST_INITDB_EXTRA_OPTS: >- > > -c debug_copy_parse_plan_trees=on > > -c debug_write_read_parse_plan_trees=on > > -c debug_raw_expression_coverage_test=on > > -c debug_parallel_query=regress > > > > Since we won't have FreeBSD for the first version. I put these options > > to the MacOS task but I couldn't decide where to put > > 'PG_TEST_PG_UPGRADE_MODE: --link'. > > Makes sense. > > > > Also, I am planning to work on back patches when we agree on the > > upstream one. Does that sound good? > > Yep. > > > > > diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml > > new file mode 100644 > > index 00000000000..6d20068727c > > --- /dev/null > > +++ b/.github/workflows/ci.yml > > @@ -0,0 +1,1125 @@ > > +# GitHub Actions CI configuration for PostgreSQL > > + > > +name: Github Actions CI > > + > > +on: > > + push: > > + branches: [ "*" ] > > + > > +# Default to the minimum privilege the jobs need (just reading the repo > > +# contents during checkout). Individual jobs override this when they need > > +# more, e.g. `cancel-previous` needs `actions: write` to cancel runs. > > +permissions: > > + contents: read > > I'm not sure I like that we ever need more than that. I'd expect that > postgresql-cfbot will explicitly disable write permissions for runs. Done. Updated the comment and removed the 'Cancel previous runs' step. > > +# NB: intentionally NO workflow-level `concurrency:` block. The native > > +# concurrency mechanism makes a new run wait for the previous one to fully > > +# cancel before it starts — which can take a while. Instead the > > +# `cancel-previous` job below fires a cancel API call asynchronously, > > +# so the new run gets going immediately. On master the cancel job is > > skipped, > > +# so every push runs to completion. > > Is this really worth having our own code? Seems like it'd not be that frequent > to push if there are already running runs? What kind of delays are we talking > about? Jelte already answered this in [2]. 'Cancel previous runs' step is removed and concurrency is used instead. > > + # To avoid unnecessarily spinning up a lot of VMs / containers for > > entirely > > + # broken commits, have a minimal task that all others depend on. > > + # > > + # SPECIAL: > > + # - Builds with --auto-features=disabled and thus almost no enabled > > + # dependencies > > + sanity-check: > > + name: SanityCheck > > + needs: setup > > + if: needs.setup.outputs.sanitycheck == 'true' > > + runs-on: ubuntu-latest > > + timeout-minutes: 15 > > + container: > > + image: ${{ needs.setup.outputs.linux_ci_image }} > > + env: > > + BUILD_JOBS: 8 > > + TEST_JOBS: 8 > > + CCACHE_DIR: ${{ github.workspace }}/ccache_dir > > + # no options enabled, should be small > > + CCACHE_MAXSIZE: "150M" > > + steps: > > + - uses: actions/checkout@v6 > > + with: > > + fetch-depth: ${{ env.CLONE_DEPTH }} > > + > > + - name: Restore ccache > > + uses: actions/cache@v5 > > Seems like this is used by every task. Can we move this into a yaml anchor or > such, by using a variable representing the job name? Github Actions doesn't support merge keys. So we can't really duplicate them. I used yaml anchors for the checkout step since it is exactly for all jobs. > > + with: > > + path: ${{ env.CCACHE_DIR }} > > + key: ccache-sanitycheck-${{ github.run_id }} > > + restore-keys: ccache-sanitycheck- > > Why is the key here the run id? Doesn't that mean that we will never have a > precise cache match and that we will keep multiple versions of the cache > around? That seems like a waste of cache space? > > For efficiency, particularly on cfbot, it seems like it could be useful to > populate the cache of branches with the cache of the master branch. For that > we'd need the branch name in the key. Which I think would also good for > postgres/postgres, as we currently have a lot of interference between runs on > the main and the REL_XY_STABLE branches. I think that is the default way. If the cache has the exact hit, it doesn't refresh the cache. So, having ${{ github.run_id }} makes sure we won't have exact hits and the cache will always be refreshed. This sounds bad but that is what I understood :( I can implement something like this: - name: Restore ccache uses: actions/cache/restore@v5 with: path: ${{ env.CCACHE_DIR }} key: ccache-sanitycheck-master restore-keys: | ccache-sanitycheck-${{ github.ref_name }} ccache-sanitycheck- - name: Save ccache if: always() uses: actions/cache/save@v5 with: path: ${{ env.CCACHE_DIR }} key: ccache-sanitycheck-${{ github.ref_name }}-${{ github.run_id }} So, it will first look for master's cache, then current branch's cache and lastly whatever cache is available. Do you prefer that? > > + - name: Prepare workspace > > + run: | > > + whoami > > + useradd -m postgres > > + chown -R postgres:postgres . > > + mkdir -p "$CCACHE_DIR" > > + chown -R postgres:postgres "$CCACHE_DIR" > > + # Can't change the container's kernel.core_pattern; the postgres > > + # user can't write to / normally. Make / writable. > > + chown root:postgres / > > + chmod g+rwx / > > Why not just always use a privileged container? Done. > > + - name: Configure > > + run: | > > + su postgres <<-'EOF' > > + set -e > > + meson setup \ > > + --buildtype=debug \ > > + --auto-features=disabled \ > > + -Ddefault_library=shared \ > > + -Dtap_tests=enabled \ > > + build > > + EOF > > + > > + - name: Build > > + run: | > > + su postgres <<EOF > > + set -e > > + ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} > > + EOF > > Should we have an explicit cache upload step here? Or are upload steps run > unconditionally? Like I explained above, that is done by having ${{ github.run_id }} in the cache key. > > + # Run a minimal set of tests. The main regression tests take too long > > + # for this purpose. For now this is a random quick pg_regress style > > + # test, and a tap test that exercises both a frontend binary and the > > + # backend. > > + - name: Test > > + run: | > > + su postgres <<EOF > > + set -e > > + ulimit -c unlimited > > + meson test ${MTEST_ARGS} --suite setup > > + meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS} \ > > + cube/regress pg_ctl/001_start_stop > > + EOF > > + > > + - name: Core backtraces > > + if: failure() > > + run: | > > + mkdir -m 770 /tmp/cores > > + find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' > > /tmp/cores/ \; > > + src/tools/ci/cores_backtrace.sh linux /tmp/cores > > + > > + - name: Upload logs > > + if: failure() > > + uses: actions/upload-artifact@v7 > > + with: > > + name: sanitycheck-logs-${{ github.run_id }} > > + path: | > > + build*/testrun/**/*.log > > + build*/testrun/**/*.diffs > > + build*/testrun/**/regress_log_* > > + build*/meson-logs/*.txt > > + if-no-files-found: ignore > > I think this really should be in a yaml anchor, we have a few somewhat > different versions of this now. Same thing, we can't have yaml anchors because merge keys are not supported. I created this variable: _LOG_PATHS: &log_paths | build*/testrun/**/*.log build*/testrun/**/*.diffs build*/testrun/**/regress_log_* build*/meson-logs/*.txt and used it in the Upload logs' path. > It's pretty annoying that the output of the failures isn't visible in the UI. > Maybe we ought to print a few of the failures out or something? We already have '--print-errorlogs', do you mean something different? > > + > > + # SPECIAL: > > + # - Uses address sanitizer (sanitizer failures are typically printed in > > + # the server log) > > + # - Configures postgres with a small segment size > > + # > > + # Enable a reasonable set of sanitizers. Use the linux task for that, as > > + # it's one of the fastest tasks (without sanitizers). Also several of the > > + # sanitizers work best on linux. > > + # > > + # The overhead of alignment sanitizer is low, undefined behaviour has > > + # moderate overhead. Test alignment sanitizer in the meson task, as it > > + # does both 32 and 64 bit builds and is thus more likely to expose > > + # alignment bugs. > > + # > > + # Address sanitizer in contrast is somewhat expensive. Enable it in the > > + # autoconf task, as the meson task tests both 32 and 64bit. > > I wonder if we should split the meson task into two, one for 32bit and one for > 64bit. The concurrency limits for public repos are high enough for that to > seem like a reasonable tradeoff? There's no work, other than the repo > checkout, shared between them. Done. > > + # disable_coredump=0, abort_on_error=1: for useful backtraces in case of > > crashes > > + # print_stacktraces=1,verbosity=2, duh > > + # detect_leaks=0: too many uninteresting leak errors in short-lived > > binaries > > + linux-autoconf: > > + name: Linux - Debian Trixie - Autoconf > > + needs: [setup, sanity-check] > > + if: | > > + !cancelled() && > > + needs.setup.outputs.linux == 'true' && > > + needs.sanity-check.result != 'failure' > > + runs-on: ubuntu-latest > > + timeout-minutes: 60 > > + container: > > + image: ${{ needs.setup.outputs.linux_ci_image }} > > + # Share the host PID + IPC namespaces. 017_shm.pl rapidly creates, > > + # kill9's, and restarts postgres; with the container's small PID > > + # space a new postgres can recycle the dead postmaster's PID before > > + # pg_ctl's postmaster.pid check notices, producing spurious "node X > > + # is already running" failures. SysV shm in the test also relies on > > + # host-like IPC behavior. > > + # > > + # --ulimit raises memlock and core dump size. Memlock is needed for > > + # running the AIO tests. > > + # > > + # --privileged is needed so the prepare step can write to sysctls > > + # under /proc/sys (it's mounted read-only without it). We use it to > > + # set kernel.core_pattern. > > + options: --pid=host --ipc=host --ulimit memlock=-1:-1 --privileged > > + env: > > + BUILD_JOBS: 4 > > + TEST_JOBS: 8 > > + CCACHE_DIR: /tmp/ccache_dir > > + DEBUGINFOD_URLS: "https://debuginfod.debian.net"; > > + > > + SANITIZER_FLAGS: -fsanitize=address > > + UBSAN_OPTIONS: > > print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 > > + ASAN_OPTIONS: > > print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0:detect_stack_use_after_return=0 > > + CFLAGS: -Og -ggdb -fno-sanitize-recover=all -fsanitize=address > > + CXXFLAGS: -Og -ggdb -fno-sanitize-recover=all -fsanitize=address > > + LDFLAGS: -fsanitize=address > > + CC: ccache gcc > > + CXX: ccache g++ > > There's a fair bit of stuff shared between the meson/autoconf linux > tasks. Previously they used a matrix to reduce that a *bit*. But now it's > entirely duplicated, including stuff that doesn't apply to the current job > (e.g. UBSAN_OPTIONS/ASAN_OPTIONS). And blocks like the following: > > > > + - name: Prepare workspace > > + run: | > > + useradd -m postgres > > + chown -R postgres:postgres . > > + mkdir -p "$CCACHE_DIR" > > + chown -R postgres:postgres "$CCACHE_DIR" > > + mkdir -m 770 /tmp/cores > > + chown root:postgres /tmp/cores > > + sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' > > + > > + # Hosts for the load balance test > > + cat >> /etc/hosts <<-EOF > > + 127.0.0.1 pg-loadbalancetest > > + 127.0.0.2 pg-loadbalancetest > > + 127.0.0.3 pg-loadbalancetest > > + EOF I found we can use matrices and merged all linux tasks. I am not sure that is better since it is a bit harder to read now. > > + # Install dependencies via Homebrew rather than Macports. On stock > > + # GH runners macports requires a heavy bootstrap, and the relevant > > + # Postgres deps are all available in brew. > > What does "heavy bootstrap" mean? I used MacPorts on my first version. It took ~10 minutes to download MacPorts. I think that if we could use caching like we did in the Cirrus, it makes sense to use MacPorts. I will spend some time on that. And after spending some time, I am able to make it work. Now the first run's dependencies install takes ~10 minutes since there is no MacPorts cache but subsequent runs' install only take ~5 seconds. > > + - name: Install dependencies > > + run: | > > + brew update > > + brew install \ > > + ccache meson openldap [email protected] tcl-tk > > + # IPC::Run via cpanm (system perl) > > + sudo cpan -T -i IPC::Run IO::Tty > > We do spend ~95s on this every run, that's not nothing. And it puts a bunch of > load onto the brew's mirrors to do that every run. You are right. MacPorts is used now. > > + - name: Test world > > + run: | > > + ulimit -c unlimited > > + ulimit -n 1024 > > + meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS} > > I'd re-add the comments that were in .cirrus.yml about this. Done. > > + windows-vs: > > + name: Windows - Server 2022, VS 2022 - Meson & ninja > > + needs: [setup, sanity-check] > > + if: | > > + !cancelled() && > > + needs.setup.outputs.windows == 'true' && > > + needs.sanity-check.result != 'failure' > > + runs-on: windows-2022 > > + timeout-minutes: 60 > > + env: > > + TEST_JOBS: 8 > > + # Avoid port conflicts between concurrent tap tests > > + PG_TEST_USE_UNIX_SOCKETS: 1 > > + PG_REGRESS_SOCK_DIR: 'c:\pgsock\' > > At least my editor gets confused by the \', thinking it's escaping the '. As > everything just works without the trailing \, I'd go that way. Done. > > + # The TAP tests build an initdb template under build/tmp_install and > > + # then `robocopy` it into per-test data directories. Robocopy with > > the > > + # default /COPY:DAT flag doesn't copy ACLs — destinations inherit > > from > > + # their parent dir. On GitHub-hosted Windows runners the workspace's > > + # inherited ACL grants Administrators:(F) and Users:(RX) but does NOT > > + # grant the runner user (runneradmin) directly. That matters because > > + # pg_ctl on Windows uses CreateRestrictedProcess to drop admin > > + # privileges from postmaster, so the postmaster process has the user > > + # SID in its token but no longer the Administrators group — leaving > > it > > + # with only "Users:(RX)" on pg_control and friends, which causes > > + # "PANIC: could not open file global/pg_control: Permission denied". > > + # > > + # Fix it once on the workspace dir with (OI)(CI) inheritance flags so > > + # every file/dir created underneath gets an explicit grant for the > > + # current user. > > + - name: Grant workspace ACL to runner user > > + shell: pwsh > > + run: | > > + icacls "${{ github.workspace }}" /grant > > "${env:USERNAME}:(OI)(CI)F" /Q | Out-Null > > + Write-Host "Granted Full Control to $env:USERNAME on ${{ > > github.workspace }}" > > Perhaps this would be better to fix by changing the robocopy flags? I couldn't fix this by using robocopy flags. I used /COPYALL and /SECFIX together but they didn't work. > > + # postgres' plpython3u loads python3.dll (the stable-ABI forwarder) > > + # which in turn loads whichever python3NN.dll the Windows loader > > finds > > + # first on PATH. On windows-2022 `C:\Program Files\Mercurial\` ships > > + # its own python3.dll + python39.dll and appears on PATH *before* the > > + # hostedtoolcache Python 3.12 — so without intervention the backend > > + # ends up running Python 3.9 while postgres' stdlib search uses 3.12, > > + # producing `ImportError: cannot import name 'text_encoding' from > > + # 'io'` (the 3.12 `io.py` calling into 3.9's `_io`). > > + # > > + # Pin PYTHONHOME to the Python 3.12 prefix, and prepend that prefix > > + # to PATH so its python3.dll wins the DLL search. > > + - name: Pin Python prefix on PATH and PYTHONHOME > > + shell: pwsh > > + run: | > > + $prefix = (python -c "import sys; print(sys.prefix)").Trim() > > + Add-Content $env:GITHUB_ENV "PYTHONHOME=$prefix" > > + Add-Content $env:GITHUB_PATH $prefix > > + Write-Host "PYTHONHOME=$prefix" > > + Write-Host "Prepended $prefix to PATH" > > GRJGJKLJKJDFJKDF. I re-checked this since Jelte wasn't completely sure about this [2] but this is unfortunately correct :( > > + - name: Install dependencies > > + shell: pwsh > > + run: | > > + choco install -y --no-progress --limitoutput diffutils > > winflexbison > > + # meson + ninja aren't preinstalled on windows-2022. Install via > > pip > > + python -m pip install --upgrade meson ninja > > + > > + # OpenSSL 1.1 via the slproweb installer (pinned to match the > > + # version used elsewhere in postgres CI). > > + curl.exe -fsSL -o openssl-setup.exe > > https://slproweb.com/download/Win64OpenSSL-1_1_1w.exe > > + Start-Process -Wait -FilePath ./openssl-setup.exe ` > > + -ArgumentList '/DIR=c:\openssl\1.1\ /VERYSILENT /SP- > > /SUPPRESSMSGBOXES' > > + # The slproweb installer puts libcrypto-1_1-x64.dll / > > libssl-1_1-x64.dll > > + # in c:\openssl\1.1\bin\ and updates the system PATH. GH Actions > > + # snapshots PATH at job start though, so the running job won't > > + # see those DLLs and initdb.exe would crash silently at runtime. > > + # Push the bin dir onto GITHUB_PATH so it persists for later > > steps. > > + Add-Content $env:GITHUB_PATH "c:\openssl\1.1\bin" > > I don't like that much, but I'm not sure we have a better alternative > short-term. Making chocolatey would be a nice alternative. You already said sometimes chocolatey takes too much time. I am planning to spend time on it unless we are planning to use our own Windows containers. > > + windows-mingw: > > + name: Windows - Server 2022, MinGW64 - Meson > > + needs: [setup, sanity-check] > > + if: | > > + !cancelled() && > > + needs.setup.outputs.mingw == 'true' && > > + needs.sanity-check.result != 'failure' > > + runs-on: windows-2022 > > + timeout-minutes: 60 > > + env: > > + TEST_JOBS: 4 # higher concurrency causes occasional failures > > + PG_TEST_USE_UNIX_SOCKETS: 1 > > + PG_REGRESS_SOCK_DIR: 'c:\pgsock\' > > + TAR: "c:/windows/system32/tar.exe" > > + # for mingw plpython to find its installation > > + PYTHONHOME: D:/a/_temp/msys64/ucrt64 > > + > > + MSYS: winjitdebug > > + CHERE_INVOKING: 1 > > + MESON_FEATURES: >- > > + -Dnls=disabled > > Missing comments from .cirrus.tasks.yml Done. v3 is attached. Just a quick note, v3 includes Zsolt [3] And Peter's [4] reviews & feedback too. I will reply to them after sending this. GA run after v3 is applied: https://github.com/nbyavuz/postgres/actions/runs/26587973538 [1] https://github.com/actions/runner/issues/1182 https://github.com/orgs/community/discussions/185877 [2] https://postgr.es/m/CAGECzQQBCF%3DHSk4eCc1fEYTpCt59rgpcwWp47%2B6M-CDMYEaM2A%40mail.gmail.com [3] https://postgr.es/m/CAN4CZFO4usEzFQoYzEywvOgoagW%3DU4yhpB4Oq-a7bUCR53djHA%40mail.gmail.com [4] https://postgr.es/m/3daa29a4-6a08-41c1-8a6a-53ba8cd3c7fb%40eisentraut.org -- Regards, Nazir Bilal Yavuz Microsoft
From b6b0c0b6b0b3846c81cf5fa599d32167f1beb4b7 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 28 May 2026 19:31:34 +0300 Subject: [PATCH v3] Add GitHub Actions yaml file Cirrus CI is shutting down. This is an initial attempt to get a GitHub Actions CI working. --- .github/workflows/postgresql-ci.yml | 1013 ++++++++++++++++++++++++++ src/tools/ci/ci_macports_packages.sh | 19 +- 2 files changed, 1029 insertions(+), 3 deletions(-) create mode 100644 .github/workflows/postgresql-ci.yml diff --git a/.github/workflows/postgresql-ci.yml b/.github/workflows/postgresql-ci.yml new file mode 100644 index 00000000000..b0130e868ac --- /dev/null +++ b/.github/workflows/postgresql-ci.yml @@ -0,0 +1,1013 @@ +# GitHub Actions CI configuration for PostgreSQL + +name: Github Actions CI + +on: + push: + +# Restrict GITHUB_TOKEN to the minimum the jobs need: reading repo +# contents during checkout. +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + # Never cancel in-progress runs on master to ensure all commits are tested. + cancel-in-progress: ${{ github.ref != 'refs/heads/master' }} + +env: + # The lower depth accelerates git clone. Use a bit of depth so that + # concurrent jobs and retrying older runs have a chance of working. + CLONE_DEPTH: 500 + + CCACHE_MAXSIZE: "250M" + + # check target for the autoconf builds + CHECK: check-world PROVE_FLAGS=--timer + CHECKFLAGS: -Otarget + + # Build test dependencies as part of the build step, to see compiler + # errors/warnings in one place. + MBUILD_TARGET: all testprep + MTEST_ARGS: --print-errorlogs --no-rebuild -C build + PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests + TEMP_CONFIG: ${{ github.workspace }}/src/tools/ci/pg_ci_base.conf + PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth + + # Postgres config args for the meson builds, shared between all meson tasks + # except the 'SanityCheck' task + MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true + + # Meson feature flags shared by all meson tasks, except: + # SanityCheck: uses almost no dependencies. + # Windows - VS: has fewer dependencies than listed here, so defines its own. + # Linux: uses the 'auto' feature option to test meson feature autodetection. + MESON_COMMON_FEATURES: >- + -Dauto_features=disabled + -Dldap=enabled + -Dssl=openssl + -Dtap_tests=enabled + -Dplperl=enabled + -Dplpython=enabled + -Ddocs=enabled + -Dicu=enabled + -Dlibxml=enabled + -Dlibxslt=enabled + -Dlz4=enabled + -Dpltcl=enabled + -Dreadline=enabled + -Dzlib=enabled + -Dzstd=enabled + + # Shared between the Linux autoconf job and the CompilerWarnings jobs + LINUX_CONFIGURE_FEATURES: >- + --with-gssapi + --with-icu + --with-ldap + --with-libcurl + --with-libxml + --with-libxslt + --with-llvm + --with-lz4 + --with-pam + --with-perl + --with-python + --with-selinux + --with-ssl=openssl + --with-systemd + --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ + --with-uuid=ossp + --with-zstd + + # Debian Trixie container image used by all Linux jobs. Built by + # 'https://github.com/anarazel/pg-vm-images/'. + LINUX_CI_IMAGE: us-docker.pkg.dev/pg-ci-images/ci/linux_debian_trixie_ci:latest + + # The full set of OS / job selectors recognized by the `ci-os-only:` + # commit-message directive parsed in the `setup` job below. + CI_OS_ONLY_JOBS: "linux macos windows mingw compilerwarnings sanitycheck" + + _LOG_PATHS: &log_paths | + build*/testrun/**/*.log + build*/testrun/**/*.diffs + build*/testrun/**/regress_log_* + build*/meson-logs/*.txt + + +jobs: + # Parse "ci-os-only: ..." from the commit message and expose flags + # consumed by the jobs' `if:` conditions. + setup: + name: Determine enabled jobs + runs-on: ubuntu-latest + timeout-minutes: 1 + outputs: + linux: ${{ steps.os.outputs.linux }} + macos: ${{ steps.os.outputs.macos }} + windows: ${{ steps.os.outputs.windows }} + mingw: ${{ steps.os.outputs.mingw }} + compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} + sanitycheck: ${{ steps.os.outputs.sanitycheck }} + # Re-export workflow-level env vars that other jobs need to reference + # from contexts (e.g. `jobs.<id>.container.image`) where the `env` + # context is not available. + linux_ci_image: ${{ env.LINUX_CI_IMAGE }} + steps: + - id: os + env: + MSG: ${{ github.event.head_commit.message }} + shell: bash + run: | + set -e + all_os="${CI_OS_ONLY_JOBS}" + if printf '%s\n' "$MSG" | grep -qE '^ci-os-only: '; then + sel=$(printf '%s\n' "$MSG" | grep -E '^ci-os-only: ' | head -1 | sed 's/^ci-os-only: //') + echo "ci-os-only selection: $sel" + else + sel="$all_os" + fi + for o in $all_os; do + if echo " $sel " | grep -qE "[ ,]$o[ ,]"; then + echo "$o=true" >> "$GITHUB_OUTPUT" + else + echo "$o=false" >> "$GITHUB_OUTPUT" + fi + done + cat "$GITHUB_OUTPUT" + + + # To avoid unnecessarily spinning up a lot of VMs / containers for entirely + # broken commits, have a minimal task that all others depend on. + # + # SPECIAL: + # - Builds with --auto-features=disabled and thus almost no enabled + # dependencies + sanity-check: + name: SanityCheck + needs: setup + if: needs.setup.outputs.sanitycheck == 'true' + runs-on: ubuntu-latest + timeout-minutes: 15 + container: + image: ${{ needs.setup.outputs.linux_ci_image }} + # --privileged is needed so the prepare step can write to sysctls + # under /proc/sys (it's mounted read-only without it). We use it to + # set kernel.core_pattern. + options: --privileged + env: + BUILD_JOBS: 8 + TEST_JOBS: 8 + CCACHE_DIR: ${{ github.workspace }}/ccache_dir + # no options enabled, should be small + CCACHE_MAXSIZE: "150M" + steps: + # Anchor reused by other jobs further down. GitHub Actions supports + # YAML anchors/aliases but not merge keys, so the alias copies the + # whole step verbatim. The anchor is resolved at YAML parse time, so the + # alias keeps working even if this job is skipped at runtime. + - &checkout_step + uses: actions/checkout@v6 + with: + fetch-depth: ${{ env.CLONE_DEPTH }} + + - name: Restore ccache + uses: actions/cache@v5 + with: + path: ${{ env.CCACHE_DIR }} + key: ccache-sanitycheck-${{ github.ref_name }}-${{ github.run_id }} + restore-keys: | + ccache-sanitycheck-${{ github.ref_name }}- + ccache-sanitycheck- + + - name: Prepare workspace + run: | + whoami + useradd -m postgres + chown -R postgres:postgres . + mkdir -p "$CCACHE_DIR" + chown -R postgres:postgres "$CCACHE_DIR" + + - name: Configure + run: | + su postgres <<-'EOF' + set -e + meson setup \ + --buildtype=debug \ + --auto-features=disabled \ + -Ddefault_library=shared \ + -Dtap_tests=enabled \ + build + EOF + + - name: Build + run: | + su postgres <<EOF + set -e + ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} + EOF + + # Run a minimal set of tests. The main regression tests take too long + # for this purpose. For now this is a random quick pg_regress style + # test, and a tap test that exercises both a frontend binary and the + # backend. + - name: Test + run: | + su postgres <<EOF + set -e + ulimit -c unlimited + meson test ${MTEST_ARGS} --suite setup + meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS} \ + cube/regress pg_ctl/001_start_stop + EOF + + - name: Core backtraces + if: failure() + run: | + mkdir -m 770 /tmp/cores + find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; + src/tools/ci/cores_backtrace.sh linux /tmp/cores + + - name: Upload logs + if: failure() + uses: actions/upload-artifact@v7 + with: + name: sanitycheck-logs-${{ github.run_id }} + path: *log_paths + if-no-files-found: ignore + + + # Build & test postgres on Linux in three configurations. + # + # Autoconf: + # - Uses address sanitizer (sanitizer failures are typically printed in + # the server log) + # - Configures postgres with a small segment size + # - Uses PG_TEST_PG_COMBINEBACKUP_MODE=--copy-file-range + # + # Meson: + # - Test both 64 and 32 bit builds + # - Uses undefined behaviour and alignment sanitizers, (sanitizer failures + # are typically printed in the server log) + # - Uses io_method=io_uring + # - Uses meson feature autodetection + # - 32 bit build tests with LANG=C to give ICU some buildfarm-uncovered + # coverage. Also, newer Python insists on changing LC_CTYPE away from C, + # prevent that with PYTHONCOERCECLOCALE. + # + # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes + # print_stacktraces=1,verbosity=2, duh + # detect_leaks=0: too many uninteresting leak errors in short-lived binaries + linux: + name: Linux - ${{ matrix.name }} + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.linux == 'true' && + needs.sanity-check.result != 'failure' + runs-on: ubuntu-latest + timeout-minutes: 60 + strategy: + fail-fast: false + matrix: + include: + - name: Autoconf + slug: autoconf + cc: ccache gcc + cxx: ccache g++ + sanitizer_flags: -fsanitize=address + pg_test_pg_combinebackup_mode: '--copy-file-range' + configure: | + ./configure \ + --enable-cassert --enable-injection-points --enable-debug \ + --enable-tap-tests --enable-nls \ + --with-segsize-blocks=6 \ + --with-libnuma \ + --with-liburing \ + ${LINUX_CONFIGURE_FEATURES} \ + CLANG="ccache clang" + build: | + make -s -j${BUILD_JOBS} world-bin + test: | + make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} + logs_paths: | + **/*.log + **/*.diffs + **/regress_log_* + + - name: Meson (64bit) + slug: meson-64 + cc: ccache gcc + cxx: ccache g++ + sanitizer_flags: -fsanitize=alignment,undefined + pg_test_initdb_extra_opts: '-c io_method=io_uring' + configure: | + meson setup \ + ${MESON_COMMON_PG_CONFIG_ARGS} \ + -Duuid=e2fs \ + --buildtype=debug \ + -Dllvm=enabled \ + build + build: | + ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} + ninja -C build -t missingdeps + test: | + meson test ${MTEST_ARGS} -C build --num-processes ${TEST_JOBS} + logs_paths: *log_paths + + - name: Meson (32bit) + slug: meson-32 + cc: ccache gcc -m32 + cxx: ccache g++ -m32 + sanitizer_flags: -fsanitize=alignment,undefined + pg_test_initdb_extra_opts: '-c io_method=io_uring' + configure: | + meson setup \ + ${MESON_COMMON_PG_CONFIG_ARGS} \ + -Duuid=e2fs \ + --buildtype=debug \ + --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ + -DPERL=perl5.40-i386-linux-gnu \ + -Dlibnuma=disabled \ + build + build: | + ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} + ninja -C build -t missingdeps + test: | + PYTHONCOERCECLOCALE=0 LANG=C \ + meson test ${MTEST_ARGS} -C build --num-processes ${TEST_JOBS} + logs_paths: *log_paths + container: + image: ${{ needs.setup.outputs.linux_ci_image }} + # Share the host PID + IPC namespaces. 017_shm.pl rapidly creates, + # kill9's, and restarts postgres; with the container's small PID + # space a new postgres can recycle the dead postmaster's PID before + # pg_ctl's postmaster.pid check notices, producing spurious "node X + # is already running" failures. SysV shm in the test also relies on + # host-like IPC behavior. + # + # --ulimit raises memlock and core dump size. Memlock is needed for + # running the AIO tests. + # + # --privileged is needed so the prepare step can write to sysctls + # under /proc/sys (it's mounted read-only without it). We use it to + # set kernel.core_pattern and (for the meson entries) to flip + # kernel.io_uring_disabled (default 2 on recent GH runner kernels). + options: --pid=host --ipc=host --ulimit memlock=-1:-1 --privileged + env: + BUILD_JOBS: 4 + TEST_JOBS: 8 + CCACHE_DIR: /tmp/ccache_dir + DEBUGINFOD_URLS: "https://debuginfod.debian.net"; + + UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 + ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0:detect_stack_use_after_return=0 + CFLAGS: -Og -ggdb -fno-sanitize-recover=all ${{ matrix.sanitizer_flags }} + CXXFLAGS: -Og -ggdb -fno-sanitize-recover=all ${{ matrix.sanitizer_flags }} + LDFLAGS: ${{ matrix.sanitizer_flags }} + CC: ${{ matrix.cc }} + CXX: ${{ matrix.cxx }} + + PG_TEST_INITDB_EXTRA_OPTS: ${{ matrix.pg_test_initdb_extra_opts }} + PG_TEST_PG_COMBINEBACKUP_MODE: ${{ matrix.pg_test_pg_combinebackup_mode }} + steps: + - *checkout_step + + - name: Restore ccache + uses: actions/cache@v5 + with: + path: ${{ env.CCACHE_DIR }} + key: ccache-linux-${{ matrix.slug }}-${{ github.ref_name }}-${{ github.run_id }} + restore-keys: | + ccache-linux-${{ matrix.slug }}-${{ github.ref_name }}- + ccache-linux-${{ matrix.slug }}- + + - name: Prepare workspace + run: | + useradd -m postgres + chown -R postgres:postgres . + mkdir -p "$CCACHE_DIR" + chown -R postgres:postgres "$CCACHE_DIR" + mkdir -m 770 /tmp/cores + chown root:postgres /tmp/cores + sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' + # This is only needed on Linux Meson but it doesn't harm to have + # this enabled. + sysctl -w kernel.io_uring_disabled=0 + + cat >> /etc/hosts <<-EOF + 127.0.0.1 pg-loadbalancetest + 127.0.0.2 pg-loadbalancetest + 127.0.0.3 pg-loadbalancetest + EOF + + - name: Configure + run: | + su postgres <<EOF + set -e + ${{ matrix.configure }} + EOF + + - name: Build + run: | + su postgres <<EOF + set -e + ${{ matrix.build }} + EOF + + - name: Test world + run: | + su postgres <<EOF + set -e + ulimit -c unlimited + ${{ matrix.test }} + EOF + + - name: Core backtraces + if: failure() + run: src/tools/ci/cores_backtrace.sh linux /tmp/cores + + - name: Upload logs + if: failure() + uses: actions/upload-artifact@v7 + with: + name: linux-${{ matrix.slug }}-logs-${{ github.run_id }} + path: ${{ matrix.logs_paths }} + if-no-files-found: ignore + + + # SPECIAL: + # - Enables --clone for pg_upgrade and pg_combinebackup + # - Specifies configuration options that test reading/writing/copying of node trees + # - Specifies debug_parallel_query=regress, to catch related issues during CI + macos: + name: macOS - Meson + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.macos == 'true' && + needs.sanity-check.result != 'failure' + runs-on: macos-15 + timeout-minutes: 60 + env: + BUILD_JOBS: 4 + # Test performance regresses noticeably when using all cores. 8 works OK. + # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de + # Fix: Needs to be re-tested for Github Actions. + TEST_JOBS: 8 + + CCACHE_DIR: ${{ github.workspace }}/ccache_dir + MACPORTS_CACHE: ${{ github.workspace }}/macports-cache + + MESON_FEATURES: >- + -Dbonjour=enabled + -Ddtrace=enabled + -Dgssapi=enabled + -Dlibcurl=enabled + -Dnls=enabled + -Duuid=e2fs + + MACOS_PACKAGE_LIST: >- + ccache + icu + kerberos5 + lz4 + meson + openldap + openssl + p5.34-io-tty + p5.34-ipc-run + python312 + tcl + zstd + + CC: ccache cc + CXX: ccache c++ + CFLAGS: -Og -ggdb + CXXFLAGS: -Og -ggdb + PG_TEST_PG_UPGRADE_MODE: --clone + PG_TEST_PG_COMBINEBACKUP_MODE: --clone + + # Several buildfarm animals enable these options. Without testing them + # during CI, it would be easy to cause breakage on the buildfarm with CI + # passing. + PG_TEST_INITDB_EXTRA_OPTS: >- + -c debug_copy_parse_plan_trees=on + -c debug_write_read_parse_plan_trees=on + -c debug_raw_expression_coverage_test=on + -c debug_parallel_query=regress + + steps: + - *checkout_step + + - name: Sysinfo + run: | + id + uname -a + ulimit -a -H && ulimit -a -S + env + + - name: Setup core files + run: | + mkdir -p $HOME/cores + sudo sysctl kern.corefile="$HOME/cores/core.%P" + + - name: Restore ccache + uses: actions/cache@v5 + with: + path: ${{ env.CCACHE_DIR }} + key: ccache-macos-${{ github.ref_name }}-${{ github.run_id }} + restore-keys: | + ccache-macos-${{ github.ref_name }}- + ccache-macos- + + - name: Compute MacPorts cache key + id: mpkey + run: | + macos_major=$(sw_vers -productVersion | sed 's/\..*//') + pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) + script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) + echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}-${GITHUB_RUN_ID}" >> "$GITHUB_OUTPUT" + echo "restore-key=macports-${macos_major}-${pkglist_hash}-${script_hash}-" >> "$GITHUB_OUTPUT" + + - name: Restore MacPorts cache + uses: actions/cache@v5 + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mpkey.outputs.key }} + restore-keys: ${{ steps.mpkey.outputs.restore-key }} + + # Use macports, even though homebrew is installed. The installation + # of the additional packages we need would take quite a while with + # homebrew, even if we cache the downloads. We can't cache all of + # homebrew, because it's already large. So we use macports. To cache + # the installation we create a .dmg file that we mount if it already + # exists. + # XXX: The reason for the direct p5.34* references is that we'd need + # the large macport tree around to figure out that p5-io-tty is + # actually p5.34-io-tty. Using the unversioned name works, but + # updates macports every time. + - name: Install dependencies (MacPorts) + env: + # Pass token so the script's GitHub API call to list MacPorts + # releases isn't subject to the 60/hr/IP unauthenticated rate + # limit (shared across all jobs on the runner's IP). + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST + # system python doesn't provide headers + sudo /opt/local/bin/port select python3 python312 + # Make macports install visible to subsequent steps + echo /opt/local/sbin >> "$GITHUB_PATH" + echo /opt/local/bin >> "$GITHUB_PATH" + + - name: Configure + run: | + export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" + meson setup \ + ${MESON_COMMON_PG_CONFIG_ARGS} \ + --buildtype=debug \ + -Dextra_include_dirs=/opt/local/include \ + -Dextra_lib_dirs=/opt/local/lib \ + -Ddarwin_sysroot=none \ + ${MESON_COMMON_FEATURES} \ + ${MESON_FEATURES} \ + build + + - name: Build + run: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} + + - name: Test world + run: | + ulimit -c unlimited # default is 0 + ulimit -n 1024 # default is 256, pretty low + meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS} + + - name: Core backtraces + if: failure() + run: src/tools/ci/cores_backtrace.sh macos "$HOME/cores" + + - name: Upload logs + if: failure() + uses: actions/upload-artifact@v7 + with: + name: macos-logs-${{ github.run_id }} + path: *log_paths + if-no-files-found: ignore + + + windows-vs: + name: Windows - VS - Meson & ninja + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.windows == 'true' && + needs.sanity-check.result != 'failure' + runs-on: windows-2022 + timeout-minutes: 60 + env: + TEST_JOBS: 8 + # Avoid port conflicts between concurrent tap tests + PG_TEST_USE_UNIX_SOCKETS: 1 + PG_REGRESS_SOCK_DIR: 'c:\pgsock' + + MESON_FEATURES: >- + -Dcpp_args=/std:c++20 + -Dauto_features=disabled + -Dtap_tests=enabled + -Dldap=enabled + -Dssl=openssl + -Dplperl=enabled + -Dplpython=enabled + TAR: "c:/windows/system32/tar.exe" + + defaults: + run: + shell: cmd + steps: + - name: Disable Windows Defender + shell: powershell + run: | + Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable + # Verify Defender status + $status = Get-MpComputerStatus -ErrorAction SilentlyContinue + if ($status) { + Write-Host "RealTimeProtectionEnabled: $($status.RealTimeProtectionEnabled)" + Write-Host "AntivirusEnabled: $($status.AntivirusEnabled)" + } + + - *checkout_step + + - name: Sysinfo + run: | + chcp + systeminfo + set + + # The TAP tests build an initdb template under build/tmp_install and + # then `robocopy` it into per-test data directories. Robocopy with the + # default /COPY:DAT flag doesn't copy ACLs — destinations inherit from + # their parent dir. On GitHub-hosted Windows runners the workspace's + # inherited ACL grants Administrators:(F) and Users:(RX) but does NOT + # grant the runner user (runneradmin) directly. That matters because + # pg_ctl on Windows uses CreateRestrictedProcess to drop admin + # privileges from postmaster, so the postmaster process has the user + # SID in its token but no longer the Administrators group — leaving it + # with only "Users:(RX)" on pg_control and friends, which causes + # "PANIC: could not open file global/pg_control: Permission denied". + # + # Fix it once on the workspace dir with (OI)(CI) inheritance flags so + # every file/dir created underneath gets an explicit grant for the + # current user. + - name: Grant workspace ACL to runner user + shell: pwsh + run: | + icacls "${{ github.workspace }}" /grant "${env:USERNAME}:(OI)(CI)F" /Q | Out-Null + Write-Host "Granted Full Control to $env:USERNAME on ${{ github.workspace }}" + + # postgres' plpython3u loads python3.dll (the stable-ABI forwarder) + # which in turn loads whichever python3NN.dll the Windows loader finds + # first on PATH. On windows-2022 `C:\Program Files\Mercurial\` ships + # its own python3.dll + python39.dll and appears on PATH *before* the + # hostedtoolcache Python 3.12 — so without intervention the backend + # ends up running Python 3.9 while postgres' stdlib search uses 3.12, + # producing `ImportError: cannot import name 'text_encoding' from + # 'io'` (the 3.12 `io.py` calling into 3.9's `_io`). + # + # Pin PYTHONHOME to the Python 3.12 prefix, and prepend that prefix + # to PATH so its python3.dll wins the DLL search. + - name: Pin Python prefix on PATH and PYTHONHOME + shell: pwsh + run: | + $prefix = (python -c "import sys; print(sys.prefix)").Trim() + Add-Content $env:GITHUB_ENV "PYTHONHOME=$prefix" + Add-Content $env:GITHUB_PATH $prefix + Write-Host "PYTHONHOME=$prefix" + Write-Host "Prepended $prefix to PATH" + + - name: Install dependencies + shell: pwsh + run: | + choco install -y --no-progress --limitoutput diffutils winflexbison3 + # meson + ninja aren't preinstalled on windows-2022. Install via pip + python -m pip install --upgrade meson ninja + + # OpenSSL 1.1 via the slproweb installer (pinned to match the + # version used elsewhere in postgres CI). + curl.exe -fsSL -o openssl-setup.exe https://slproweb.com/download/Win64OpenSSL-1_1_1w.exe + Start-Process -Wait -FilePath ./openssl-setup.exe ` + -ArgumentList '/DIR=c:\openssl\1.1\ /VERYSILENT /SP- /SUPPRESSMSGBOXES' + # The slproweb installer puts libcrypto-1_1-x64.dll / libssl-1_1-x64.dll + # in c:\openssl\1.1\bin\ and updates the system PATH. GH Actions + # snapshots PATH at job start though, so the running job won't + # see those DLLs and initdb.exe would crash silently at runtime. + # Push the bin dir onto GITHUB_PATH so it persists for later steps. + Add-Content $env:GITHUB_PATH "c:\openssl\1.1\bin" + + # Install IPC::Run. + # - recommends_policy=0 keeps cpan from pulling in IO::Tty / IO::Pty, + # which don't build on Windows ("This module requires a POSIX + # compliant system to work"). + # - Pin to NJM/IPC-Run-20250809.0 because TODDR/IPC-Run-20260322.0 + # broke postgres tap tests on Windows (changed pipe stdio + # handling). See upstream pg-vm-images commit ff5238afa3 and + # the thread at + # https://postgr.es/m/CAN55FZ06xanSbJdHe-CurjX_qNuBWZDEvS1kAk36L38YCtZXnw%40mail.gmail.com + "o conf recommends_policy 0`no conf commit`nnotest install NJM/IPC-Run-20250809.0.tar.gz" | cpan + perl -mIPC::Run -e 1 + + - name: Setup hosts file + shell: pwsh + run: | + Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.1 pg-loadbalancetest" + Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.2 pg-loadbalancetest" + Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.3 pg-loadbalancetest" + + - name: Setup socket directory + shell: cmd + run: mkdir %PG_REGRESS_SOCK_DIR% + + - name: Configure + run: | + call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 + meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% %MESON_FEATURES% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% build + + - name: Build + run: | + call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 + ninja -C build %MBUILD_TARGET% + ninja -C build -t missingdeps + + - name: Test world + run: | + call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 + meson test %MTEST_ARGS% --num-processes %TEST_JOBS% + + - name: Upload logs + if: failure() + uses: actions/upload-artifact@v7 + with: + name: windows-vs-logs-${{ github.run_id }} + path: | + ${{ env._LOG_PATHS }} + crashlog-*.txt + if-no-files-found: ignore + + + windows-mingw: + name: Windows - MinGW - Meson + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.mingw == 'true' && + needs.sanity-check.result != 'failure' + runs-on: windows-2022 + timeout-minutes: 60 + env: + TEST_JOBS: 4 # higher concurrency causes occasional failures + PG_TEST_USE_UNIX_SOCKETS: 1 + PG_REGRESS_SOCK_DIR: 'c:\pgsock\' + TAR: "c:/windows/system32/tar.exe" + # for mingw plpython to find its installation + PYTHONHOME: D:/a/_temp/msys64/ucrt64 + + MSYS: winjitdebug + CHERE_INVOKING: 1 + + # Keep -Dnls explicitly disabled, as the number of files it creates + # causes a noticeable slowdown. + MESON_FEATURES: >- + -Dnls=disabled + + CCACHE_DIR: D:/a/ccache + CCACHE_MAXSIZE: "500M" + CCACHE_SLOPPINESS: pch_defines,time_macros + CCACHE_DEPEND: 1 + + steps: + - name: Disable Windows Defender + shell: powershell + run: | + Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable + # Verify Defender status + $status = Get-MpComputerStatus -ErrorAction SilentlyContinue + if ($status) { + Write-Host "RealTimeProtectionEnabled: $($status.RealTimeProtectionEnabled)" + Write-Host "AntivirusEnabled: $($status.AntivirusEnabled)" + } + + - *checkout_step + + - name: Setup MSYS2 + uses: msys2/setup-msys2@v2 + with: + msystem: UCRT64 + update: true + install: >- + git bison flex make diffutils + mingw-w64-ucrt-x86_64-ccache + mingw-w64-ucrt-x86_64-gcc + mingw-w64-ucrt-x86_64-icu + mingw-w64-ucrt-x86_64-libbacktrace + mingw-w64-ucrt-x86_64-libxml2 + mingw-w64-ucrt-x86_64-libxslt + mingw-w64-ucrt-x86_64-lz4 + mingw-w64-ucrt-x86_64-make + mingw-w64-ucrt-x86_64-meson + mingw-w64-ucrt-x86_64-perl + mingw-w64-ucrt-x86_64-pkg-config + mingw-w64-ucrt-x86_64-readline + mingw-w64-ucrt-x86_64-zlib + + - name: Install additional dependencies + shell: msys2 {0} + run: | + # Pin IPC::Run to NJM/IPC-Run-20250809.0; TODDR/IPC-Run-20260322.0 + # broke postgres tap tests on Windows (pipe stdio handling). + # See pg-vm-images commit ff5238afa3. + (echo; echo o conf recommends_policy 0; echo notest install NJM/IPC-Run-20250809.0.tar.gz) | cpan + perl -mIPC::Run -e 1 + + - name: Setup socket directory + shell: cmd + run: mkdir %PG_REGRESS_SOCK_DIR% + + - name: Restore ccache + uses: actions/cache@v5 + with: + path: ${{ env.CCACHE_DIR }} + key: ccache-mingw-${{ github.ref_name }}-${{ github.run_id }} + restore-keys: | + ccache-mingw-${{ github.ref_name }}- + ccache-mingw- + + - name: Configure + shell: msys2 {0} + run: | + meson setup \ + ${MESON_COMMON_PG_CONFIG_ARGS} \ + -Ddebug=true -Doptimization=g -Db_pch=true \ + ${MESON_COMMON_FEATURES} \ + ${MESON_FEATURES} \ + -DTAR=${TAR} \ + build + + - name: Build + shell: msys2 {0} + run: ninja -C build ${MBUILD_TARGET} + + - name: Test world + shell: msys2 {0} + run: meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS} + + - name: Upload logs + if: failure() + uses: actions/upload-artifact@v7 + with: + name: windows-mingw-logs-${{ github.run_id }} + path: | + ${{ env._LOG_PATHS }} + crashlog-*.txt + if-no-files-found: ignore + + # Test that code can be built with both gcc and clang without warnings, + # with various combinations of cassert/dtrace flags. Trace probes have + # a history of getting accidentally broken; the matrix is there to + # catch that. + # + # The autoconf cache files (gcc.cache / clang.cache) are intentionally + # reused across the matrix entries that share a compiler, so we don't + # pay for full feature detection on every entry. + compiler-warnings: + name: CompilerWarnings + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.compilerwarnings == 'true' && + needs.sanity-check.result != 'failure' + runs-on: ubuntu-latest + timeout-minutes: 60 + container: + image: ${{ needs.setup.outputs.linux_ci_image }} + env: + BUILD_JOBS: 4 + CCACHE_DIR: /tmp/ccache_dir + # Use larger ccache cache as this job compiles with multiple + # compilers / flag combinations. + CCACHE_MAXSIZE: "1G" + steps: + - *checkout_step + + - name: Restore ccache + uses: actions/cache@v5 + with: + path: ${{ env.CCACHE_DIR }} + key: ccache-compiler-warnings-${{ github.ref_name }}-${{ github.run_id }} + restore-keys: | + ccache-compiler-warnings-${{ github.ref_name }}- + ccache-compiler-warnings- + + - name: Sysinfo + run: | + id + uname -a + cat /proc/cmdline + ulimit -a -H && ulimit -a -S + gcc -v + clang -v + env + + - name: Setup workspace + run: | + echo "COPT=-Werror" > src/Makefile.custom + mkdir -p "$CCACHE_DIR" + + # gcc, cassert off, dtrace on + - name: gcc warnings + (dtrace) + if: always() + run: | + ./configure \ + --cache gcc.cache \ + --enable-dtrace \ + ${LINUX_CONFIGURE_FEATURES} \ + CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} world-bin + + # gcc, cassert on, dtrace off + - name: gcc warnings + (cassert) + if: always() + run: | + ./configure \ + --cache gcc.cache \ + --enable-cassert \ + ${LINUX_CONFIGURE_FEATURES} \ + CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} world-bin + + # clang, cassert off, dtrace off + - name: clang warnings + if: always() + run: | + ./configure \ + --cache clang.cache \ + ${LINUX_CONFIGURE_FEATURES} \ + CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} world-bin + + # clang, cassert on, dtrace on + - name: clang warnings + (cassert + dtrace) + if: always() + run: | + ./configure \ + --cache clang.cache \ + --enable-cassert \ + --enable-dtrace \ + ${LINUX_CONFIGURE_FEATURES} \ + CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} world-bin + + - name: mingw warnings (cross compilation) + if: always() + run: | + ./configure \ + --host=x86_64-w64-mingw32ucrt \ + --enable-cassert \ + --without-icu \ + CC="ccache x86_64-w64-mingw32ucrt-gcc" \ + CXX="ccache x86_64-w64-mingw32ucrt-g++" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} world-bin + + ### + # Verify docs can be built + ### + # XXX: Only do this if there have been changes in doc/ since last build + - name: Build documentation + if: always() + run: | + ./configure \ + --cache gcc.cache \ + CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} -C doc + + ### + # Verify headerscheck / cpluspluscheck succeed + # + # - Run both in same script to increase parallelism, use -k to get + # result of both + # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose + ### + - name: headerscheck + cpluspluscheck + if: always() + run: | + ./configure \ + ${LINUX_CONFIGURE_FEATURES} \ + --cache gcc.cache \ + --quiet \ + CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' diff --git a/src/tools/ci/ci_macports_packages.sh b/src/tools/ci/ci_macports_packages.sh index 63e97b37c78..4c79f90fed0 100755 --- a/src/tools/ci/ci_macports_packages.sh +++ b/src/tools/ci/ci_macports_packages.sh @@ -20,13 +20,26 @@ echo "macOS major version: $macos_major_version" # macOS release. macports_release_list_url="https://api.github.com/repos/macports/macports-base/releases"; macports_version_pattern="2\.10\.1" -macports_url="$( curl -s $macports_release_list_url | grep "\"https://github.com/macports/macports-base/releases/download/v$macports_version_pattern/MacPorts-$macports_version_pattern-$macos_major_version-[A-Za-z]*\.pkg\""; | sed 's/.*: "//;s/".*//' | head -1 )" +# Authenticate the GitHub API request when a token is available (e.g. on +# GitHub Actions). Unauthenticated requests share a 60/hr/IP rate limit +# with every other job on the runner's IP and frequently return an error +# JSON, leaving $macports_url empty and breaking the subsequent curl. +auth_header="" +if [ -n "$GITHUB_TOKEN" ]; then + auth_header="Authorization: Bearer $GITHUB_TOKEN" +fi +macports_url="$( curl -fsSL ${auth_header:+-H "$auth_header"} "$macports_release_list_url" | grep "\"https://github.com/macports/macports-base/releases/download/v$macports_version_pattern/MacPorts-$macports_version_pattern-$macos_major_version-[A-Za-z]*\.pkg\""; | sed 's/.*: "//;s/".*//' | head -1 )" echo "MacPorts package URL: $macports_url" +if [ -z "$macports_url" ]; then + echo "error: could not determine MacPorts package URL for macOS $macos_major_version (version pattern: $macports_version_pattern)" 1>&2 + exit 1 +fi + cache_dmg="macports.hfs.dmg" -if [ "$CIRRUS_CI" != "true" ]; then - echo "expect to be called within cirrus-ci" 1>2 +if [ "$CIRRUS_CI" != "true" ] && [ "$GITHUB_ACTIONS" != "true" ]; then + echo "expect to be called within cirrus-ci or github actions" 1>2 exit 1 fi -- 2.47.3
