potiuk commented on a change in pull request #6373: [AIRFLOW-5704] Optimise
KIND kubernetes setup
URL: https://github.com/apache/airflow/pull/6373#discussion_r336760858
##########
File path: Dockerfile
##########
@@ -174,39 +170,8 @@ RUN if [[ "${APT_DEPS_IMAGE}" == "airflow-apt-deps-ci"
]]; then \
;\
fi
-# TODO: We should think about removing those and moving them into
docker-compose dependencies.
COPY scripts/ci/docker_build/ci_build_install_deps.sh
/tmp/ci_build_install_deps.sh
-# Kubernetes dependencies
-RUN \
-if [[ "${APT_DEPS_IMAGE}" == "airflow-apt-deps-ci" ]]; then \
Review comment:
It's not at runtime - it's just grouping all the CI-related installs in
single RUN command.
This is mostly to decrease the build time for slim (in the future
production) image. The problem is that currently in multi-staging images, even
if you have a stage that is not used, it will not be skipped (this will be
improved when BUILDKIT goes out of beta and we will be able to use it including
caching - in BUILDKIT, unused stages are skipped while building). The "if"
there is a hack to skip building CI dependencies if we are only building slim
image.
Separating it to single script makes it slightly faster to rebuild the slim
image- every RUN command takes couple of seconds when executed so if we have a
separate RUN and if in every RUN, we got 20-30 seconds longer builds always
(because this stage is not part of the image, so we do not have cache for it,
it is also not cached so it is always built - even if we locally built the slim
image. Unfortunately, we cannot use both --cache-from and local cache at the
same time (https://github.com/moby/moby/issues/32612) - this could also solve
the problem.
This will be reverted back to RUN commands in the PROD image change I am
working on #6266
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
