[
https://issues.apache.org/jira/browse/HDDS-8569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-8569:
-----------------------------
Description:
Push ozone-docker-runner arm64 Docker images to Docker Hub. So that devs with
M1/M2 Macs won't have to build that locally first to avoid slow Docker Desktop
qemu emulation.
For example, airflow does have both arm64 and amd64 images, so that should be
doable: https://hub.docker.com/r/apache/airflow/tags
while ozone-docker-runner currently only has amd64 in Docker Hub:
https://hub.docker.com/r/apache/ozone-runner/tags
--
Found this wiki page [Github Actions to
DockerHub|https://cwiki.apache.org/confluence/display/INFRA/Github+Actions+to+DockerHub]
but not sure if this applies to ozone-docker-runner.
--
I played with docker manifest a bit. I'm able to stitch existing x64 (amd64)
image with a locally built arm64 image and push it to my Docker Hub public
repo. Works with Docker Desktop macOS on an M1 Mac:
{code}
# Docker Hub username
DH_USERNAME=siyaom
# git tag to target
TAG=20230503-1
docker pull apache/ozone-runner:${TAG}
# grab x64 image from apache/ozone-runner, since it currently only has the x64
one
docker tag apache/ozone-runner:${TAG} ${DH_USERNAME}/ozone-runner:${TAG}-amd64
# build arm64 image locally, if it isn't built already
DOCKER_BUILDKIT=1 docker build -t ${DH_USERNAME}/ozone-runner:${TAG}-arm64 .
# have to actually push both images to Docker Hub before running `manifest
create`
docker push ${DH_USERNAME}/ozone-runner:${TAG}-amd64
docker push ${DH_USERNAME}/ozone-runner:${TAG}-arm64
# stitch multiple archs into one manifest
docker manifest create ${DH_USERNAME}/ozone-runner:${TAG} \
--amend ${DH_USERNAME}/ozone-runner:${TAG}-amd64 \
--amend ${DH_USERNAME}/ozone-runner:${TAG}-arm64
# `manifest push`! not image `push`
docker manifest push ${DH_USERNAME}/ozone-runner:${TAG}
# Ref: https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/
{code}
Looks like it works: https://hub.docker.com/r/siyaom/ozone-runner/tags
Though not sure why it says `arm64/v8`, not just `arm64` like
[airflow's|https://hub.docker.com/r/apache/airflow/tags].
was:
Push ozone-docker-runner arm64 Docker images to Docker Hub. So that devs with
M1/M2 Macs won't have to build that locally first to avoid slow Docker Desktop
qemu emulation.
For example, airflow does have both arm64 and amd64 images, so that should be
doable: https://hub.docker.com/r/apache/airflow/tags
while ozone-docker-runner currently only has amd64 in Docker Hub:
https://hub.docker.com/r/apache/ozone-runner/tags
--
Found this wiki page [Github Actions to
DockerHub|https://cwiki.apache.org/confluence/display/INFRA/Github+Actions+to+DockerHub]
but not sure if this applies to ozone-docker-runner. cc [~adoroszlai]
> ozone-docker-runner: Push out arm64 Docker image so devs won't have to build
> that locally first
> -----------------------------------------------------------------------------------------------
>
> Key: HDDS-8569
> URL: https://issues.apache.org/jira/browse/HDDS-8569
> Project: Apache Ozone
> Issue Type: Task
> Reporter: Siyao Meng
> Priority: Critical
>
> Push ozone-docker-runner arm64 Docker images to Docker Hub. So that devs with
> M1/M2 Macs won't have to build that locally first to avoid slow Docker
> Desktop qemu emulation.
> For example, airflow does have both arm64 and amd64 images, so that should be
> doable: https://hub.docker.com/r/apache/airflow/tags
> while ozone-docker-runner currently only has amd64 in Docker Hub:
> https://hub.docker.com/r/apache/ozone-runner/tags
> --
> Found this wiki page [Github Actions to
> DockerHub|https://cwiki.apache.org/confluence/display/INFRA/Github+Actions+to+DockerHub]
> but not sure if this applies to ozone-docker-runner.
> --
> I played with docker manifest a bit. I'm able to stitch existing x64 (amd64)
> image with a locally built arm64 image and push it to my Docker Hub public
> repo. Works with Docker Desktop macOS on an M1 Mac:
> {code}
> # Docker Hub username
> DH_USERNAME=siyaom
> # git tag to target
> TAG=20230503-1
> docker pull apache/ozone-runner:${TAG}
> # grab x64 image from apache/ozone-runner, since it currently only has the
> x64 one
> docker tag apache/ozone-runner:${TAG} ${DH_USERNAME}/ozone-runner:${TAG}-amd64
> # build arm64 image locally, if it isn't built already
> DOCKER_BUILDKIT=1 docker build -t ${DH_USERNAME}/ozone-runner:${TAG}-arm64 .
> # have to actually push both images to Docker Hub before running `manifest
> create`
> docker push ${DH_USERNAME}/ozone-runner:${TAG}-amd64
> docker push ${DH_USERNAME}/ozone-runner:${TAG}-arm64
> # stitch multiple archs into one manifest
> docker manifest create ${DH_USERNAME}/ozone-runner:${TAG} \
> --amend ${DH_USERNAME}/ozone-runner:${TAG}-amd64 \
> --amend ${DH_USERNAME}/ozone-runner:${TAG}-arm64
> # `manifest push`! not image `push`
> docker manifest push ${DH_USERNAME}/ozone-runner:${TAG}
> # Ref: https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/
> {code}
> Looks like it works: https://hub.docker.com/r/siyaom/ozone-runner/tags
> Though not sure why it says `arm64/v8`, not just `arm64` like
> [airflow's|https://hub.docker.com/r/apache/airflow/tags].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
