This is an automated email from the ASF dual-hosted git repository.
vatamane pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb-ci.git
The following commit(s) were added to refs/heads/main by this push:
new 5dfa685 Update to latest OTP 25 path. Add remote builds info
5dfa685 is described below
commit 5dfa685daa29c3b44534d49aa5066e92d53c4c63
Author: Nick Vatamaniuc <[email protected]>
AuthorDate: Sat Mar 8 11:58:02 2025 -0500
Update to latest OTP 25 path. Add remote builds info
* Update to latest OTP 25
* Use Node 20 instead of 18
* Update Elixir to latest 17 patch
* Add build instructions for remote builders
* Remove any Eralang Solution traces, their hosts are dead and fail
the build
* Replace some `yum`s with `dnf`s since that is the new thing. Yum
will still keep working if they are any instances left.
* Remove unsupport version checks (CentOS 6 and 7)
---
README.md | 58 +++++++++++++++++++++++++++++++++++++++++----
bin/apt-erlang.sh | 10 +-------
bin/install-dependencies.sh | 6 ++---
bin/install-elixir.sh | 2 +-
bin/yum-dependencies.sh | 52 +++++++++++++++++++---------------------
bin/yum-erlang.sh | 7 ------
dockerfiles/almalinux-8 | 4 ++--
dockerfiles/almalinux-9 | 4 ++--
dockerfiles/debian-bookworm | 4 ++--
dockerfiles/debian-bullseye | 4 ++--
dockerfiles/ubuntu-focal | 4 ++--
dockerfiles/ubuntu-jammy | 4 ++--
dockerfiles/ubuntu-noble | 4 ++--
13 files changed, 97 insertions(+), 66 deletions(-)
diff --git a/README.md b/README.md
index 4c178d0..1dcac9b 100644
--- a/README.md
+++ b/README.md
@@ -49,7 +49,7 @@ On the other hand, some OSes won't run older Erlangs because
of library changes,
Just specify on the command line any of the `ERLANGVERSION`, `NODEVERSION`, or
`ELIXIRVERSION` environment variables:
```
-NODEVERSION=18 ELIXIRVERSION=v1.17.2 ERLANGVERSION=25.3.2.15 ./build.sh
platform debian-bullseye
+NODEVERSION=20 ELIXIRVERSION=v1.17.3 ERLANGVERSION=25.3.2.18 ./build.sh
platform debian-bookworm
```
## Building images for other architectures
@@ -58,14 +58,64 @@ NODEVERSION=18 ELIXIRVERSION=v1.17.2
ERLANGVERSION=25.3.2.15 ./build.sh platform
Use Docker's
[Buildx](https://docs.docker.com/buildx/working-with-buildx/) plugin to
generate
-multi-architecture container images with a single command invocation. Docker
-Desktop ships with buildx support, but you'll need to create a new builder to
-use it:
+multi-architecture container images with a single command invocation.
+
+Multi-arch images can be built locally, with archtectures emulated
+with qemu, or using remote workers, if we have ssh acccess to hosts
+running on each of the supported architectures. Aside from the initial
+setup the rest of the commands are exactly the same.
+
+##### Local setup
```
docker buildx create --name apache-couchdb --use
```
+##### Remote setup
+
+This requires your ssh key to be installed on the remote workers. See
+the `couchdb-infra-cm` repo for the list of current keys.
+
+To use shorter names for the servers can include the `ssh.cfg` file
+from `couchdb-infra-cm` in your `ssh_config`.
+
+Ensure ssh connection to each build server works. There maybe be a
+different user for each one (root, ubuntu, linux1, etc), the `ssh.cfg`
+file, if you're using that already handles that transparently.
+
+If the user is not root, ensure the user can run `docker ps`. For instance:
+```
+sudo usermod -aG docker linux1
+```
+
+Test that `docker ps` run for each server:
+
+```
+ssh linux1 docker ps
+ssh ubuntu-nc-arm64-12 docker ps
+ssh ubuntu docker ps
+ssh ubuntu-fra1-10 docker ps
+```
+
+Setup the `multiarch` xbuild context:
+
+```
+docker buildx rm multiarch || true
+docker buildx create --name multiarch --driver docker-container --platform
linux/arm64 ssh://ubuntu-nc-arm64-12
+docker buildx create --append --name multiarch --driver docker-container
--platform linux/amd64 ssh://ubuntu-fra1-10
+docker buildx create --append --name multiarch --driver docker-container
--platform linux/s390x ssh://linux1
+docker buildx create --append --name multiarch --driver docker-container
--platform linux/ppc64le ssh://ubuntu
+```
+
+Before building, set `docker buildx` to `use` the new context as the new
default.
+When done, set to `default`, or whatever it was before:
+
+```
+docker buildx use multiarch
+```
+
+#### Building
+
The `build.sh` script has `buildx-base` and `buildx-platform` targets that will
will build **and upload** a new multi-arch container image to the registry. For
example:
diff --git a/bin/apt-erlang.sh b/bin/apt-erlang.sh
index faaa6f4..110e563 100755
--- a/bin/apt-erlang.sh
+++ b/bin/apt-erlang.sh
@@ -65,20 +65,12 @@ apt-get update || true
if [ "${ERLANGVERSION}" = "default" ]
then
apt-get update && apt-get install --no-install-recommends -y erlang-nox
erlang-dev erlang erlang-eunit erlang-dialyzer
-elif [ "${ARCH}" = x86_64 ]
-then
- wget https://packages.erlang-solutions.com/erlang-solutions_2.0_all.deb
- dpkg -i erlang-solutions_*_all.deb || true
- rm erlang-solutions_*_all.deb
- apt-get update || true
- apt-get install --no-install-recommends -y esl-erlang=1:${ERLANGVERSION} ||
true
fi
# fallback to source install if all else fails
if [ ! -x /usr/bin/erl -a ! -x /usr/local/bin/erl ]
then
- # remove any trailing -### in version used for erlang solutions packages
- apt-get purge -y erlang-solutions && apt-get update
+ apt-get update
export ERLANGVERSION=$(echo ${ERLANGVERSION} | cut -d- -f 1)
${SCRIPTPATH}/source-erlang.sh
for pkg in \
diff --git a/bin/install-dependencies.sh b/bin/install-dependencies.sh
index 37876e6..deda5d7 100755
--- a/bin/install-dependencies.sh
+++ b/bin/install-dependencies.sh
@@ -29,9 +29,9 @@
set -e
# Defaults updated 2023-03-20
-NODEVERSION=${NODEVERSION:-18}
-ERLANGVERSION=${ERLANGVERSION:-25.3.2.15}
-ELIXIRVERSION=${ELIXIRVERSION:-v1.17.2}
+NODEVERSION=${NODEVERSION:-20}
+ERLANGVERSION=${ERLANGVERSION:-25.3.2.18}
+ELIXIRVERSION=${ELIXIRVERSION:-v1.17.3}
# This works if we're not called through a symlink
diff --git a/bin/install-elixir.sh b/bin/install-elixir.sh
index 0ed9fa9..f749eb4 100755
--- a/bin/install-elixir.sh
+++ b/bin/install-elixir.sh
@@ -28,7 +28,7 @@
set -e
-ELIXIR_VSN=${ELIXIRVERSION:-v1.17.2}
+ELIXIR_VSN=${ELIXIRVERSION:-v1.17.3}
ERLANG_VSN=`erl -noshell -eval 'io:fwrite(erlang:system_info(otp_release)),
halt().'`
# See https://github.com/hexpm/bob
URL=https://repo.hex.pm/builds/elixir/${ELIXIR_VSN}-otp-${ERLANG_VSN}.zip
diff --git a/bin/yum-dependencies.sh b/bin/yum-dependencies.sh
index 00bb4c8..1e5a303 100755
--- a/bin/yum-dependencies.sh
+++ b/bin/yum-dependencies.sh
@@ -84,29 +84,29 @@ echo "Detected RedHat/Centos/Fedora version: ${VERSION_ID}
arch: ${ARCH}"
# TODO: Do the Right Things(tm) for Fedora
+dnf update -y
+
# Enable EPEL
-yum install -y
https://dl.fedoraproject.org/pub/epel/epel-release-latest-${VERSION_ID}.noarch.rpm
|| true
+dnf install -y
https://dl.fedoraproject.org/pub/epel/epel-release-latest-${VERSION_ID}.noarch.rpm
|| true
# PowerTools for Alma 8
if [[ ${VERSION_ID} -eq 8 ]]; then
dnf install -y 'dnf-command(config-manager)'
dnf config-manager --set-enabled powertools
- yum update -y
+ dnf update -y
fi
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY*
-if [[ ${VERSION_ID} -ne 6 ]]; then
- # more for RHEL than CentOS...
- yum install subscription-manager -y
- subscription-manager repos \
- --enable "rhel-*-optional-rpms" \
- --enable "rhel-*-extras-rpms" \
- >/dev/null 2>&1 || true
-fi
+# more for RHEL than CentOS...
+dnf install subscription-manager -y
+subscription-manager repos \
+ --enable "rhel-*-optional-rpms" \
+ --enable "rhel-*-extras-rpms" \
+ >/dev/null 2>&1 || true
# Upgrade all packages
-yum upgrade -y
+dnf upgrade -y
# Install auxiliary packages
-yum groupinstall -y 'Development Tools'
+dnf groupinstall -y 'Development Tools'
# Dependencies for make couch, except erlang and package building stuff.
# help2man is for docs
@@ -116,7 +116,7 @@ yum install -y sudo git wget which autoconf
autoconf-archive automake curl-devel
if [[ ${VERSION_ID} -eq 9 ]]; then
dnf --enablerepo=crb install -y help2man
else
- yum install -y help2man
+ dnf install -y help2man
fi
# Node.js
@@ -137,22 +137,18 @@ if [ $? -ne 0 ]; then
fake-rpm nodejs ${NODEVERSION}.0.0
else
set -e
- yum install -y nodejs
+ dnf install -y nodejs
fi
rm setup_${NODEVERSION}.x
npm install npm@latest -g
popd
# python for testing and documentaiton
-if [[ ${VERSION_ID} -eq 7 ]]; then
- yum install -y python36 python36-pip python-virtualenv
- PIP=pip3.6
- ln -s /usr/bin/python3.6 /usr/local/bin/python3
-elif [[ ${VERSION_ID} -eq 8 ]]; then
- yum install -y python3-pip python3-virtualenv
+if [[ ${VERSION_ID} -eq 8 ]]; then
+ dnf install -y python3-pip python3-virtualenv
PIP=pip3
else
- yum install -y python3-pip
+ dnf install -y python3-pip
PIP=pip3
fi
@@ -166,24 +162,24 @@ fi
if [[ $1 != "nojs" ]]; then
if [[ ${VERSION_ID} -lt 8 ]]; then
# config the CouchDB repo & install the JS packages
- yum install -y yum-utils
+ dnf install -y yum-utils
yum-config-manager --add-repo https://couchdb.apache.org/repo/couchdb.repo
# install the JS packages
- yum install -y couch-js-devel
+ dnf install -y couch-js-devel
elif [[ ${VERSION_ID} -eq 8 ]]; then
- yum install -y mozjs60-devel
+ dnf install -y mozjs60-devel
else
- yum install -y mozjs78-devel
+ dnf install -y mozjs78-devel
fi
else
# install js build-time dependencies only
# we can't add the CouchDB repo here because the plat may not exist yet
- yum install -y libffi-devel
+ dnf install -y libffi-devel
fi
# remove openjdk8 and jna, java 21 is installed and should be the default
# and clouseau installs it's own JRE 8 in /opt via a docker layer
-yum remove -y java-1.8.0-openjdk-headless jna
+dnf remove -y java-1.8.0-openjdk-headless jna
# clean up
-yum clean all -y
+dnf clean all -y
diff --git a/bin/yum-erlang.sh b/bin/yum-erlang.sh
index c3c2290..acd2a5d 100755
--- a/bin/yum-erlang.sh
+++ b/bin/yum-erlang.sh
@@ -78,17 +78,10 @@ EOF
# TODO: Do the Right Things(tm) for Fedora
if [[ ${ERLANGVERSION} == "default" ]]; then
yum install -y erlang
-elif [ ${ARCH} == x86_64 ]; then
- wget https://packages.erlang-solutions.com/erlang-solutions-2.0-1.noarch.rpm
- rpm -Uvh erlang-solutions-2.0-1.noarch.rpm
- yum install -y esl-erlang-${ERLANGVERSION} || true
- rm erlang-solutions-2.0-1.noarch.rpm
fi
# fallback to source install if all else fails
if [ ! -x /usr/bin/erl -a ! -x /usr/local/bin/erl ]; then
- # remove any trailing -### in version used for erlang solutions packages
- yum remove -y erlang-solutions
export ERLANGVERSION=$(echo ${ERLANGVERSION} | cut -d- -f 1)
${SCRIPTPATH}/source-erlang.sh
fake-rpm esl-erlang $(date "+%Y%m%d%H%M%S")
diff --git a/dockerfiles/almalinux-8 b/dockerfiles/almalinux-8
index cacd42f..247f82c 100644
--- a/dockerfiles/almalinux-8
+++ b/dockerfiles/almalinux-8
@@ -32,8 +32,8 @@ ARG js=js
ARG erlang=erlang
# Select version of Node, Erlang and Elixir to install
ARG erlangversion=25.3.2.15
-ARG elixirversion=v1.17.2
-ARG nodeversion=18
+ARG elixirversion=v1.17.3
+ARG nodeversion=20
# Create Jenkins user and group
RUN groupadd --gid 910 jenkins; \
diff --git a/dockerfiles/almalinux-9 b/dockerfiles/almalinux-9
index 17b1157..b9b307f 100644
--- a/dockerfiles/almalinux-9
+++ b/dockerfiles/almalinux-9
@@ -32,8 +32,8 @@ ARG js=js
ARG erlang=erlang
# Select version of Node, Erlang and Elixir to install
ARG erlangversion=25.3.2.15
-ARG elixirversion=v1.17.2
-ARG nodeversion=18
+ARG elixirversion=v1.17.3
+ARG nodeversion=20
# Create Jenkins user and group
RUN groupadd --gid 910 jenkins; \
diff --git a/dockerfiles/debian-bookworm b/dockerfiles/debian-bookworm
index 8b2856c..d126a9d 100644
--- a/dockerfiles/debian-bookworm
+++ b/dockerfiles/debian-bookworm
@@ -37,8 +37,8 @@ ARG js=js
ARG erlang=erlang
# Select version of Node, Erlang and Elixir
ARG erlangversion=25.3.2.15
-ARG elixirversion=v1.17.2
-ARG nodeversion=18
+ARG elixirversion=v1.17.3
+ARG nodeversion=20
# Create Jenkins user and group
RUN groupadd --gid 910 jenkins; \
diff --git a/dockerfiles/debian-bullseye b/dockerfiles/debian-bullseye
index b64b96d..2a88596 100644
--- a/dockerfiles/debian-bullseye
+++ b/dockerfiles/debian-bullseye
@@ -37,8 +37,8 @@ ARG js=js
ARG erlang=erlang
# Select version of Node, Erlang and Elixir
ARG erlangversion=25.3.2.15
-ARG elixirversion=v1.17.2
-ARG nodeversion=18
+ARG elixirversion=v1.17.3
+ARG nodeversion=20
# Create Jenkins user and group
RUN groupadd --gid 910 jenkins; \
diff --git a/dockerfiles/ubuntu-focal b/dockerfiles/ubuntu-focal
index e0303b3..4f7e059 100644
--- a/dockerfiles/ubuntu-focal
+++ b/dockerfiles/ubuntu-focal
@@ -35,8 +35,8 @@ ARG js=js
ARG erlang=erlang
# Select version of Node, Erlang and Elixir to install
ARG erlangversion=25.3.2.15
-ARG elixirversion=v1.17.2
-ARG nodeversion=18
+ARG elixirversion=v1.17.3
+ARG nodeversion=20
# Create Jenkins user and group
RUN groupadd --gid 910 jenkins; \
diff --git a/dockerfiles/ubuntu-jammy b/dockerfiles/ubuntu-jammy
index ad68d61..5305416 100644
--- a/dockerfiles/ubuntu-jammy
+++ b/dockerfiles/ubuntu-jammy
@@ -35,8 +35,8 @@ ARG js=js
ARG erlang=erlang
# Select version of Node, Erlang and Elixir to install
ARG erlangversion=25.3.2.15
-ARG elixirversion=v1.17.2
-ARG nodeversion=18
+ARG elixirversion=v1.17.3
+ARG nodeversion=20
# Create Jenkins user and group
RUN groupadd --gid 910 jenkins; \
diff --git a/dockerfiles/ubuntu-noble b/dockerfiles/ubuntu-noble
index 969ea51..b6a2514 100644
--- a/dockerfiles/ubuntu-noble
+++ b/dockerfiles/ubuntu-noble
@@ -35,8 +35,8 @@ ARG js=js
ARG erlang=erlang
# Select version of Node, Erlang and Elixir to install
ARG erlangversion=25.3.2.15
-ARG elixirversion=v1.17.2
-ARG nodeversion=18
+ARG elixirversion=v1.17.3
+ARG nodeversion=20
# Create Jenkins user and group
RUN groupadd --gid 910 jenkins; \
