gopidesupavan commented on issue #43268:
URL: https://github.com/apache/airflow/issues/43268#issuecomment-2543101546
@potiuk
Before actually dive into the code, i just tried the above flow with simple
docker image. sorry if i am confusing too much :)
The first flow option 1 is same as above steps
#### Option1:
Build the docker image and pushed to artifactory
Download the image from artifactory and unpack and push to local registry
Build from the local registry.
Looks like this is not using the existing image from the local registry.
instead its building again.
RUN:
https://github.com/gopidesupavan/prt-removal/actions/runs/12329425958/job/34413743618#step:9:208
Workflow:
https://github.com/gopidesupavan/prt-removal/blob/main/.github/workflows/build-push-reuse.yml
After searching further, it seems docker buildx not loading image that
exported from sources(tar file). https://github.com/docker/buildx/issues/847
And as per docker buildx it seems the way is to use named context
https://docs.docker.com/build/ci/github-actions/named-contexts/
#### Option2:
Build the docker image with inline cache and pushed to artifactory
Download the image from artifactory and unpack and push to local
registry
Build from the local registry with inline cache.
It works well with the inline cache. It uses the existing image from the
local registry and not building any layers again. as
there are no changes in the docker file.
RUN:
https://github.com/gopidesupavan/prt-removal/actions/runs/12329496451/job/34413888753#step:9:193
Workflow:
https://github.com/gopidesupavan/prt-removal/blob/main/.github/workflows/with-inline-cache.yml
#### Option3:
Build the docker image with local cache and pushed cache dir to
artifactory
Download the cache from artifactory to tmp dir
Build the image with local cache referring to the local cache dir. (in
pr flow)
It also works well with the local cache type. all the layers from the
local cache dir and not building any layers again.
RUN:
https://github.com/gopidesupavan/prt-removal/actions/runs/12329584735/job/34414063998#step:7:193
Workflow:
https://github.com/gopidesupavan/prt-removal/blob/main/.github/workflows/with-local-cache.yml
Consider that in the above every option first two steps executed in the
schedule run. push the schedule run image and use it pr flows.
So basically i tried to mimic the scenario. in each workflow that i used
above first job is in the context of schedule run and next job is pr flow.
Unable to fully evaluate the side effects of the inline cache. IMHO, the
last option seems similar to what we aim to achieve, but instead of pushing the
image, we could push all the local caches for the different images (3.9, 3.10,
3.11, 3.12) built during the scheduled run. These cache artifacts (using the
run-id as you suggested) could then be downloaded and utilized in the PR
workflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
