Hello Laszlo Gaal, Joe McDonnell,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20469
to look at the new patch set (#6).
Change subject: IMPALA-12436: Create multi-platform builder images
......................................................................
IMPALA-12436: Create multi-platform builder images
Updates the toolchain build image creation process to support publishing
multi-platform (aarch64 and amd64) images.
Uses docker buildx to produce multi-platform images. This requires QEMU
on the builder; these builds are quick, so the overhead of emulating arm
is minimal. Currently assumes images are built on amd64 as not all
builds specify `--platform`.
Moves the publishing step to happen within buildx, because buildx does
not support loading multi-platform images in the local Docker registry.
This does mean that a build failure could result in publishing only some
of the images. These are not updated frequently and are usually preceded
by a test run, so this risk seems minimal. The jenkins/build.sh script
avoids this issue by first running a build without publishing, then when
it succeeds running again with publishing (which will largely pull image
layers from the build cache so should be very fast).
Python 2.7 ran into build issues installing awscli on ARM, so OS
targeting ARM were updated to ensure they include Python 3.8+ and
install a newer version of awscli in Python 3.
New options were added to buildall.py:
- registry: publish images to the specified registry
- multi: enables multi-platform builds on RedHat 8 and Ubuntu 20/22
Multi-platform builds require QEMU and a non-default buildx builder.
Jenkins job is updated to install QEMU, create a builder, and build
multi-platform images.
Multi-platform images can't be loaded into a local Docker context (where
they would be listed via docker images); we continue to load them there
by default with --load, but disable it when doing multi-platform builds.
The only way to get images out of the builder for multi-platform builds
is to specify a registry.
Change-Id: I058deb877f26b48a2d87a65710143bba9a42a5fb
---
M docker/README.md
M docker/all/postinstall.sh
M docker/buildall.py
M docker/redhat8.df
M docker/ubuntu2004.df
M docker/ubuntu2204.df
M jenkins/build.sh
7 files changed, 78 insertions(+), 32 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/native-toolchain
refs/changes/69/20469/6
--
To view, visit http://gerrit.cloudera.org:8080/20469
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I058deb877f26b48a2d87a65710143bba9a42a5fb
Gerrit-Change-Number: 20469
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Laszlo Gaal <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
