Hi hackers,
I'm looking for more horsepower for testing commitfest entries
automatically, and today I tried out $SUBJECT. The attached is a
rudimentary first attempt, for show-and-tell. If you have a Github
account, you just have to push it to a branch there and look at the
Actions tab on the web page for the results. Does anyone else have
.github files and want to share, to see if we can combine efforts
here?
The reason for creating three separate "workflows" for Linux, Windows
and macOS rather than three separate "jobs" inside one workflow is so
that cfbot.cputube.org could potentially get separate pass/fail
results for each OS out of the API rather than one combined result. I
rather like that feature of cfbot's results. (I could be wrong about
needing to do that, this is the first time I've ever looked at this
stuff.)
The Windows test actually fails right now, exactly as reported by
Ranier[1]. It is a release build on a recent MSVC, so I guess that is
expected and off-topic for this thread. But generally,
.github/workflows/ci-windows.yml is the weakest part of this. It'd be
great to get a debug/assertion build, show backtraces when it crashes,
run more of the tests, etc etc, but I don't know nearly enough about
Windows to do that myself. Another thing is that it uses Choco for
flex and bison; it'd be better to find those on the image, if
possible. Also, for all 3 OSes, it's not currently attempting to
cache build results or anything like that.
I'm a bit sad that GH doesn't have FreeBSD build runners. Those are
now popping up on other CIs, but I'm not sure if their free/open
source tiers have enough resources for cfbot.
[1]
https://www.postgresql.org/message-id/flat/CAEudQArhn8bH836OB%2B3SboiaeEcgOtrJS58Bki4%3D5yeVqToxgw%40mail.gmail.com
From 424bcae7f1fcb1ada0e7046bfc5e0c5254c6f439 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.mu...@gmail.com>
Date: Thu, 3 Dec 2020 10:58:16 +1300
Subject: [PATCH] Github CI WIP
---
.github/workflows/ci-linux.yml | 43 ++++++++++++++++++++++
.github/workflows/ci-macos.yml | 38 +++++++++++++++++++
.github/workflows/ci-windows-buildsetup.pl | 35 ++++++++++++++++++
.github/workflows/ci-windows-dumpregr.pl | 22 +++++++++++
.github/workflows/ci-windows.yml | 32 ++++++++++++++++
5 files changed, 170 insertions(+)
create mode 100644 .github/workflows/ci-linux.yml
create mode 100644 .github/workflows/ci-macos.yml
create mode 100644 .github/workflows/ci-windows-buildsetup.pl
create mode 100644 .github/workflows/ci-windows-dumpregr.pl
create mode 100644 .github/workflows/ci-windows.yml
diff --git a/.github/workflows/ci-linux.yml b/.github/workflows/ci-linux.yml
new file mode 100644
index 0000000000..8ee32770cd
--- /dev/null
+++ b/.github/workflows/ci-linux.yml
@@ -0,0 +1,43 @@
+name: ci-linux
+on: [push]
+jobs:
+ ci-linux:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ - name: Install packages
+ run: |
+ sudo apt-get --yes update
+ sudo apt-get --yes install gcc libreadline-dev flex bison make perl libipc-run-perl clang llvm-dev libperl-dev libpython-dev tcl-dev libldap2-dev libicu-dev docbook-xml docbook-xsl fop libxml2-utils xsltproc krb5-admin-server krb5-kdc krb5-user slapd ldap-utils libssl-dev pkg-config locales-all gdb
+ env:
+ DEBIAN_FRONTEND: noninteractive
+ - name: Configure
+ run: ./configure --enable-cassert --enable-debug --enable-tap-tests --with-tcl --with-python --with-perl --with-ldap --with-openssl --with-icu --with-llvm
+ - name: Build
+ run: |
+ echo "COPT=-Wall -Werror" > src/Makefile.custom
+ make -s -j3
+ - name: Check world
+ run: |
+ echo '/tmp/%e-%s-%p.core' | sudo tee /proc/sys/kernel/core_pattern
+ ulimit -c unlimited
+ make -s check-world
+ env:
+ PG_TEST_EXTRA: "ssl kerberos"
+ #PG_TEST_EXTRA: "ssl ldap kerberos" why does slapd fail to start?
+ - name: Look for clues
+ if: ${{ failure() }}
+ run: |
+ # dump useful logs
+ for F in ` find . -name initdb.log -o -name regression.diffs ` ; do
+ echo === $F ===
+ head -1000 $F
+ done
+ # look for core files and spit out backtraces
+ for corefile in $(find /tmp/ -name '*.core' 2>/dev/null) ; do
+ binary=$(gdb -quiet -core $corefile -batch -ex 'info auxv' | grep AT_EXECFN | perl -pe "s/^.*\"(.*)\"\$/\$1/g")
+ echo dumping $corefile for $binary
+ gdb --batch --quiet -ex "thread apply all bt full" -ex "quit" $binary $corefile
+ done
+ - name: Documentation
+ run: make -s docs
diff --git a/.github/workflows/ci-macos.yml b/.github/workflows/ci-macos.yml
new file mode 100644
index 0000000000..2c20a5a279
--- /dev/null
+++ b/.github/workflows/ci-macos.yml
@@ -0,0 +1,38 @@
+name: ci-macos
+on: [push]
+jobs:
+ ci-macos:
+ runs-on: macos-latest
+ steps:
+ - uses: actions/checkout@v2
+ - name: Install packages
+ run: |
+ cpan -T IPC::Run
+ - name: Configure
+ run: ./configure --enable-cassert --enable-debug --enable-tap-tests --without-readline
+ env:
+ PERL5LIB: ~/perl5/lib/perl5
+ - name: Build
+ run: |
+ echo "COPT=-Wall -Werror" > src/Makefile.custom
+ make -s -j3
+ env:
+ PERL5LIB: ~/perl5/lib/perl5
+ - name: Check world
+ run: |
+ ulimit -c unlimited
+ make -s check-world
+ env:
+ PERL5LIB: ~/perl5/lib/perl5
+ - name: Look for clues
+ if: ${{ failure() }}
+ run: |
+ # dump useful logs
+ for F in ` find . -name initdb.log -o -name regression.diffs ` ; do
+ echo === $F ===
+ head -1000 $F
+ done
+ # look for core files and spit out backtraces
+ for corefile in $(find /cores/ -name 'core.*' 2>/dev/null) ; do
+ lldb -c $corefile --batch -o 'thread backtrace all' -o 'quit'
+ done
diff --git a/.github/workflows/ci-windows-buildsetup.pl b/.github/workflows/ci-windows-buildsetup.pl
new file mode 100644
index 0000000000..a27ded84e0
--- /dev/null
+++ b/.github/workflows/ci-windows-buildsetup.pl
@@ -0,0 +1,35 @@
+# first part of postgres build.pl, just doesn't run msbuild
+use strict;
+
+BEGIN
+{
+ chdir("../../..") if (-d "../msvc" && -d "../../../src");
+}
+
+use lib "src/tools/msvc";
+
+use Cwd;
+
+use Mkvcbuild;
+
+# buildenv.pl is for specifying the build environment settings
+# it should contain lines like:
+# $ENV{PATH} = "c:/path/to/bison/bin;$ENV{PATH}";
+
+if (-e "src/tools/msvc/buildenv.pl")
+{
+ do "src/tools/msvc/buildenv.pl";
+}
+elsif (-e "./buildenv.pl")
+{
+ do "./buildenv.pl";
+}
+
+# set up the project
+our $config;
+do "config_default.pl";
+do "config.pl" if (-f "src/tools/msvc/config.pl");
+
+# print "PATH: $_\n" foreach (split(';',$ENV{PATH}));
+
+Mkvcbuild::mkvcbuild($config);
diff --git a/.github/workflows/ci-windows-dumpregr.pl b/.github/workflows/ci-windows-dumpregr.pl
new file mode 100644
index 0000000000..116caa2427
--- /dev/null
+++ b/.github/workflows/ci-windows-dumpregr.pl
@@ -0,0 +1,22 @@
+use strict;
+use warnings FATAL => qw(all);
+
+use File::Find;
+
+my $Target = "regression.diffs";
+
+find(\&dump, "src");
+
+sub dump {
+ if ($_ eq $Target) {
+ my $path = $File::Find::name;
+ print "=== \$path ===\\n";
+ open(my $fh, "<", $_) || die "wtf";
+ for (1..1000) {
+ my $line = <$fh>;
+ last unless defined $line;
+ print $line;
+ }
+ }
+}
+
diff --git a/.github/workflows/ci-windows.yml b/.github/workflows/ci-windows.yml
new file mode 100644
index 0000000000..f3c8703bce
--- /dev/null
+++ b/.github/workflows/ci-windows.yml
@@ -0,0 +1,32 @@
+name: ci-windows
+on: [push]
+jobs:
+ ci-windows:
+ runs-on: windows-latest
+ steps:
+ - uses: actions/checkout@v2
+ - name: Install packages
+ shell: cmd
+ run: |
+ choco install -y winflexbison activeperl diffutils
+ rename "c:\ProgramData\chocolatey\bin\win_flex.exe" flex.exe
+ rename "c:\ProgramData\chocolatey\bin\win_bison.exe" bison.exe
+ - name: Build
+ shell: cmd
+ run: |
+ SET PATH=%PATH%;C:\Perl64\bin
+ call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
+ move .github\workflows\ci-windows-buildsetup.pl buildsetup.pl
+ move .github\workflows\ci-windows-dumpregr.pl dumpregr.pl
+ perl buildsetup.pl
+ msbuild pgsql.sln
+ - name: Check
+ shell: cmd
+ run: |
+ SET PATH=%PATH%;C:\Perl64\bin
+ cd src\tools\msvc && vcregress check
+ - name: Look for clues
+ if: ${{ failure() }}
+ shell: cmd
+ run: |
+ perl dumpregr.pl
--
2.20.1
