Michael Smith has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20469
Change subject: IMPALA-12436: Build multi-platform builder images ...................................................................... IMPALA-12436: Build multi-platform builder images Updates the toolchain build image creation process to support publishing multi-platform (aarch64 and amd64) images. Uses docker buildx to produce multi-platform images. This requires QEMU on the builder; these builds are quick, so the overhead of emulating arm is minimal. Currently assumes images are built on amd64 as not all builds specify `--platform`. Moves the publishing step to happen within buildx, because buildx does not support loading multi-platform images in the local Docker registry. This does mean that a build failure could result in publishing only some of the images. These are not updated frequently and are usually preceded by a test run, so this risk seems minimal. Python 2.7 ran into build issues installing awscli on ARM, so OS targeting ARM were updated to ensure they include Python 3.8+ and install a newer version of awscli in Python 3. New options were added to buildall.py: - registry: publish images to the specified registry - multi: enables multi-platform builds on RedHat 8 and Ubuntu 20/22 Multi-platform builds require QEMU and a non-default buildx builder. Jenkins job is updated to install QEMU, create a builder, and build multi-platform images. Multi-platform images can't be loaded into a local Docker context (where they would be listed via docker images); we continue to load them there by default with --load, but disable it when doing multi-platform builds. The only way to get images out of the builder for multi-platform builds is to specify a registry. Change-Id: I058deb877f26b48a2d87a65710143bba9a42a5fb --- M docker/all/postinstall.sh M docker/buildall.py M docker/redhat8.df M docker/ubuntu2004.df M docker/ubuntu2204.df M jenkins/build.sh 6 files changed, 56 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/69/20469/1 -- To view, visit http://gerrit.cloudera.org:8080/20469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I058deb877f26b48a2d87a65710143bba9a42a5fb Gerrit-Change-Number: 20469 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith <michael.sm...@cloudera.com>
