This is an automated email from the ASF dual-hosted git repository.
vatamane pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/couchdb-ci.git
The following commit(s) were added to refs/heads/main by this push:
new 7dbc1e4 Update default Erlang and cleanup unused commands
7dbc1e4 is described below
commit 7dbc1e4bde7cca9210fe5fb32271265a67482a42
Author: Nick Vatamaniuc <[email protected]>
AuthorDate: Fri Dec 30 00:17:24 2022 -0500
Update default Erlang and cleanup unused commands
Some of those haven't worked in more than 2 years and we've only used
buildx to build both the x86 and the multi-arch images.
Ideally we wouldn't want to use proprietary Docker bits but that's the
only one that works well with multi-arch builds. Perhaps Podman or
other newer things can replace it soon enough.
---
README.md | 54 ++++++-----------
bin/install-dependencies.sh | 14 +----
build.sh | 144 ++------------------------------------------
3 files changed, 25 insertions(+), 187 deletions(-)
diff --git a/README.md b/README.md
index d30e43f..8e0a01d 100644
--- a/README.md
+++ b/README.md
@@ -49,20 +49,14 @@ On the other hand, some OSes won't run older Erlangs
because of library changes,
Just specify on the command line any of the `ERLANGVERSION`, `NODEVERSION`, or
`ELIXIRVERSION` environment variables:
```
-NODEVERSION=14 ELIXIRVERSION=v1.13.4 ERLANGVERSION=24.3.4.2 ./build.sh
platform debian-jessie
+NODEVERSION=14 ELIXIRVERSION=v1.13.4 ERLANGVERSION=24.3.4.7 ./build.sh
platform debian-jessie
```
-The tool also recognizes a special `ERLANGVERSION=all` value for the
`debian-buster`
-platform. This builds the lowest, default, and highest versions of Erlang using
-the [kerl](https://github.com/kerl/kerl) build system, and installs them to
-`/usr/local/kerl` for activation before builds. This version is intended for
use
-in standard CI runs, such as for pull requests.
-
## Building images for other architectures
### Multi-arch images with Docker Buildx
-We can use Docker's
+Use Docker's
[Buildx](https://docs.docker.com/buildx/working-with-buildx/) plugin to
generate
multi-architecture container images with a single command invocation. Docker
Desktop ships with buildx support, but you'll need to create a new builder to
@@ -83,36 +77,38 @@ example:
The `$BUILDX_PLATFORMS` environment variable can be used to override the
default
set of target platforms that will be supplied to the buildx builder.
-### Cross-building with $CONTAINERARCH
-
-Alternatively, we can build individual images for each architecture. This only
works from an `x86_64` build host.
+# Useful things you can do
-First, configure your machine with the correct dependencies to build
multi-arch binaries:
+## Update images used for package releases with new Erlang versions
```
-docker run --privileged --rm tonistiigi/binfmt --install all
+ERLANGVERSION=24.3.4.7 ./build.sh buildx-platform-release
```
-This is a one-time setup step. This docker container run will install the
correct qemu static binaries necessary for running foreign architecture
binaries on your host machine. It includes special magic to ensure `sudo` works
correctly inside a container, too.
+This will build all the Debian and RHEL-clone OS images on x86-64 with that
version of Erlang
-Then, override the `CONTAINERARCH` environment variable when starting
`build.sh`:
+## Update images used for CI with new Erlang versions
```
-CONTAINERARCH=arm64v8 ./build.sh platform debian-bullseye
+ERLANGVERSION=24.3.4.7 ./build.sh buildx-platform debian-bullseye
```
-## Publishing a container
+This will update Debian Bullseye OS image for all architectures (x86,
+ppc64le, arm64) with that version of Erlang. Do this for the images
+which are used with non-x86 architectures.
-If you built a single-architecture container image and did not supply `--push`
-as a build arg to upload it automatically you can upload the image using
+If the same image is used for release package building and CI, run
+this command after `buildx-platform-release` to ensure that the
+debian-bullseye will be a rebuilt as a multi-arch image. Otherwise,
+buildx-platform-release creates as x86 only.
+
+## Update Debian Bullseye image with 25.2
```
-./build.sh platform-upload <distro>-<version>
+BUILDX_PLATFORMS=linux/amd64 ERLANGVERSION=25.2 ./build.sh buildx-platform
debian-bullseye
```
----
-
-# Useful things you can do
+In this case, since we're not using 25.2 for multi-arch testing, opt to build
it only for x86.
## Full `build.sh` options
@@ -123,18 +119,8 @@ Recognized commands:
clean <plat> Removes all images for <plat>.
clean-all Removes all images for all platforms.
- *buildx-base <plat> Builds a multi-architecture base image.
*buildx-platform <plat> Builds a multi-architecture image with Erlang & JS
support.
-
- base <plat> Builds the image for <plat> without Erlang or JS
support.
- base-all Builds all images without Erlang or JS support.
- *base-upload <plat> Uploads the apache/couchdbci-{os} base images to
Docker Hub.
- *base-upload-all Uploads all the apache/couchdbci base images to
Docker Hub.
-
- platform <plat> Builds the image for <plat> with Erlang & JS
support.
- platform-all Builds all images with Erlang and JS support.
- *platform-upload <plat> Uploads the apache/couchdbci-{os} images to Docker
Hub.
- *platform-upload-all Uploads all the apache/couchdbci images to Docker
Hub.
+ *buildx-platform-release <plat> Builds x86-64 images with default Erlang for
all supported release OSes
couch <plat> Builds and tests CouchDB for <plat>.
couch-all Builds and tests CouchDB on all platforms.
diff --git a/bin/install-dependencies.sh b/bin/install-dependencies.sh
index 1bbf010..ecc69fc 100755
--- a/bin/install-dependencies.sh
+++ b/bin/install-dependencies.sh
@@ -30,7 +30,7 @@ set -e
# Defaults updated 2019-10-14
NODEVERSION=${NODEVERSION:-14}
-ERLANGVERSION=${ERLANGVERSION:-23.3.4.15}
+ERLANGVERSION=${ERLANGVERSION:-24.3.4.7}
ELIXIRVERSION=${ELIXIRVERSION:-v1.13.4}
@@ -97,18 +97,6 @@ case "${OSTYPE}" in
run_scripts ${EXTRA_SCRIPTS_DIR} 'yum-'
elif [[ ${ID} =~ ${debians} ]]; then
- # Catching this early, so as to avoid user frustration
- if [[ ${ERLANGVERSION%%.*} -le 19 ]] && [[ ${VERSION_CODENAME} =~
${latest} ]]; then
- if [[ $ARCH =~ $arms ]] && [[ ! ${SKIPERLANG} ]]; then
- echo ""
- echo "Recent versions of Linux (Stretch, Bionic, etc) provide a
version of libssl"
- echo "which is too new to complile earlier (<=19) versions of
Erlang. Please"
- echo "either choose an earlier distro release or a more rencent
version of Erlang."
- echo ""
- exit 1
- fi
- fi
-
NODEVERSION=${NODEVERSION} ERLANGVERSION=${ERLANGVERSION} \
${SCRIPTPATH}/apt-dependencies.sh ${JSINSTALL}
if [[ ! ${SKIPERLANG} ]]; then
diff --git a/build.sh b/build.sh
index 4c9aaa5..87aafcc 100755
--- a/build.sh
+++ b/build.sh
@@ -84,49 +84,6 @@ split-os-ver() {
version=${tokens[1]}
}
-build-base-platform() {
- check-envs
- split-os-ver $1
- # invoke as build-base <plat>
- # base images never get JavaScript, nor Erlang
- docker build -f dockerfiles/${os}-${version} \
- --build-arg js=nojs \
- --build-arg erlang=noerlang \
- $buildargs \
- --tag apache/couchdbci-${os}:${CONTAINERARCH}${version}-base \
- ${SCRIPTPATH}
-}
-
-buildx-base-platform() {
- check-envs
- split-os-ver $1
- # invoke as build-base <plat>
- # base images never get JavaScript, nor Erlang
- if [ "$os" == "rockylinux" ]; then
- repo="centos"
- else
- repo="$os"
- fi
- docker buildx build -f dockerfiles/${os}-${version} \
- --build-arg js=nojs \
- --build-arg erlang=noerlang \
- $buildargs \
- --platform ${BUILDX_PLATFORMS} \
- --tag apache/couchdbci-${repo}:${version}-base \
- --push \
- ${SCRIPTPATH}
-}
-
-upload-base() {
- if [[ ! ${DOCKER_ID_USER} ]]; then
- echo "Please set your Docker credentials before using this command:"
- echo " export DOCKER_ID_USER=<username>"
- echo " docker login"
- exit 1
- fi
- docker push couchdbdev/$1-base
-}
-
find-erlang-version() {
if [ -z "${ERLANGVERSION}" ]
then
@@ -139,19 +96,6 @@ pull-os-image() {
docker pull $image_name
}
-build-platform() {
- check-envs
- find-erlang-version $1
- pull-os-image $1
- split-os-ver $1
- docker build -f dockerfiles/${os}-${version} \
- $buildargs \
- --no-cache \
- --tag
apache/couchdbci-${os}:${CONTAINERARCH}${version}-erlang-${ERLANGVERSION} \
- ${SCRIPTPATH}
- unset ERLANGVERSION
-}
-
buildx-platform() {
check-envs
find-erlang-version $1
@@ -169,7 +113,6 @@ buildx-platform() {
--tag apache/couchdbci-${repo}:${version}-erlang-${ERLANGVERSION} \
--push \
${SCRIPTPATH}
- unset ERLANGVERSION
}
clean() {
@@ -178,19 +121,6 @@ clean() {
docker rmi apache/couchdbci-${os} -f || true
}
-upload-platform() {
- if [[ ! ${DOCKER_ID_USER} ]]; then
- echo "Please set your Docker credentials before using this command:"
- echo " export DOCKER_ID_USER=<username>"
- echo " docker login"
- exit 1
- fi
- find-erlang-version $1
- check-envs
- split-os-ver $1
- docker push
apache/couchdbci-${os}:${CONTAINERARCH}${version}-erlang-${ERLANGVERSION}
-}
-
build-test-couch() {
find-erlang-version $1
docker run \
@@ -215,74 +145,18 @@ case "$1" in
clean $plat
done
;;
- buildx-base)
- # Build and upload multi-arch base image using Docker Buildx
- shift
- buildx-base-platform $1
- ;;
- base)
- # Build base image for requested target platform
- shift
- build-base-platform $1
- ;;
- base-all)
- # build all base images
- shift
- for plat in $DEBIANS $UBUNTUS $CENTOSES; do
- build-base-platform $plat $*
- done
- ;;
- base-upload)
- shift
- upload-base $plat $*
- ;;
- base-upload-all)
- shift
- for plat in $DEBIANS $UBUNTUS $CENTOSES; do
- upload-base $plat $*
- done
- ;;
buildx-platform)
# Build and upload multi-arch platform with JS and Erlang support
shift
buildx-platform $1
;;
- platform)
- # build platform with JS and Erlang support
- shift
- build-platform $1
- ;;
- platform-foreign)
- # makes only foreign arch platforms
- shift
- for arch in $XPLAT_ARCHES; do
- CONTAINERARCH=$arch build-platform $XPLAT_BASE
- done
- ;;
- platform-all)
- # build all platforms with JS and Erlang support
- shift
- for plat in $DEBIANS $UBUNTUS $CENTOSES; do
- build-platform $plat $*
- done
- for arch in $XPLAT_ARCHES; do
- CONTAINERARCH=$arch build-platform $XPLAT_BASE
- done
- ERLANGVERSION=all build-platform $ERLANGALL_BASE
- ;;
- platform-upload)
- shift
- upload-platform $plat $*
- ;;
- platform-upload-all)
+ buildx-platform-release)
+ # Build and upload multi-arch platform with JS and Erlang support
+ # For all platforms
shift
for plat in $DEBIANS $UBUNTUS $CENTOSES; do
- upload-platform $plat $*
- done
- for arch in $XPLAT_ARCHES; do
- CONTAINERARCH=$arch upload-platform $XPLAT_BASE $*
+ BUILDX_PLATFORMS=linux/amd64 buildx-platform $plat
done
- ERLANGVERSION=all upload-platform $ERLANGALL_BASE
;;
couch)
# build and test CouchDB on <plat>
@@ -312,16 +186,6 @@ Recognized commands:
*buildx-base <plat> Builds a multi-architecture base image.
*buildx-platform <plat> Builds a multi-architecture image with Erlang & JS
support.
- base <plat> Builds the image for <plat> without Erlang or JS
support.
- base-all Builds all images without Erlang or JS support.
- *base-upload <plat> Uploads the apache/couchdbci-{os} base images to
Docker Hub.
- *base-upload-all Uploads all the apache/couchdbci base images to
Docker Hub.
-
- platform <plat> Builds the image for <plat> with Erlang & JS
support.
- platform-all Builds all images with Erlang and JS support.
- *platform-upload <plat> Uploads the apache/couchdbci-{os} images to Docker
Hub.
- *platform-upload-all Uploads all the apache/couchdbci images to Docker
Hub.
-
couch <plat> Builds and tests CouchDB for <plat>.
couch-all Builds and tests CouchDB on all platforms.
