ashb commented on a change in pull request #6266: [AIRFLOW-2439] Production
Docker image support including refactoring of build scripts - depends on
[AIRFLOW-5704]
URL: https://github.com/apache/airflow/pull/6266#discussion_r336904694
##########
File path: Dockerfile
##########
@@ -334,56 +380,52 @@ COPY --chown=airflow:airflow airflow/version.py
${AIRFLOW_SOURCES}/airflow/versi
COPY --chown=airflow:airflow airflow/__init__.py
${AIRFLOW_SOURCES}/airflow/__init__.py
COPY --chown=airflow:airflow airflow/bin/airflow
${AIRFLOW_SOURCES}/airflow/bin/airflow
-# The goal of this line is to install the dependencies from the most current
setup.py from sources
-# This will be usually incremental small set of packages in CI optimized
build, so it will be very fast
-# In non-CI optimized build this will install all dependencies before
installing sources.
-RUN pip install -e ".[${AIRFLOW_EXTRAS}]"
-
+# Setting to 1 speeds up building the image. Cassandra driver without CYTHON
saves around 10 minutes
+# But might not be suitable for production image
+ENV CASS_DRIVER_NO_CYTHON=""
+ENV CASS_DRIVER_BUILD_CONCURRENCY="8"
-WORKDIR ${AIRFLOW_SOURCES}/airflow/www
-
-# Copy all www files here so that we can run npm building for production
-COPY --chown=airflow:airflow airflow/www/ ${AIRFLOW_SOURCES}/airflow/www/
+ENV PATH="/home/airflow/.local/bin:/home/airflow:${PATH}"
-# Package NPM for production
-RUN gosu ${AIRFLOW_USER} npm run prod
+# The goal of this line is to install the dependencies from the most current
setup.py from sources
+# This will be usually incremental small set of packages in CI optimized
build, so it will be very fast
+# For production optimised build it is the first time dependencies are
installed so it will be slower
+RUN pip install --user ".[${AIRFLOW_PROD_EXTRAS}]" \
+ && pip uninstall --yes apache-airflow snakebite
# Cache for this line will be automatically invalidated if any
# of airflow sources change
COPY --chown=airflow:airflow . ${AIRFLOW_SOURCES}/
-WORKDIR ${AIRFLOW_SOURCES}
-
-# Finally install the requirements from the latest sources
-RUN pip install -e ".[${AIRFLOW_EXTRAS}]"
+# Reinstall airflow again - this time with sources and remove the sources
after installation
+RUN pip install --user ".[${AIRFLOW_PROD_EXTRAS}]"
# Additional python deps to install
ARG ADDITIONAL_PYTHON_DEPS=""
RUN if [[ -n "${ADDITIONAL_PYTHON_DEPS}" ]]; then \
- pip install ${ADDITIONAL_PYTHON_DEPS}; \
+ pip install --user ${ADDITIONAL_PYTHON_DEPS}; \
fi
COPY --chown=airflow:airflow ./scripts/docker/entrypoint.sh /entrypoint.sh
-ARG APT_DEPS_IMAGE="airflow-apt-deps-ci-slim"
-ENV APT_DEPS_IMAGE=${APT_DEPS_IMAGE}
+# Copy Airflow www packages
+COPY --chown=airflow:airflow --from=airflow-www /opt/airflow/airflow/www
${HOME}/.local/airflow/www
-COPY --chown=airflow:airflow .bash_completion run-tests-complete run-tests
${HOME}/
-COPY --chown=airflow:airflow .bash_completion.d/run-tests-complete \
- ${HOME}/.bash_completion.d/run-tests-complete
+COPY --chown=airflow:airflow --from=airflow-docs /opt/airflow/docs/_build/html
\
+ ${HOME}/.local/airflow/www/static/docs
-RUN if [[ "${APT_DEPS_IMAGE}" == "airflow-apt-deps-ci" ]]; then \
- ${AIRFLOW_SOURCES}/scripts/ci/docker_build/ci_build_extract_tests.sh; fi
+RUN mkdir -pv "${AIRFLOW_HOME}" \
Review comment:
This first command isn't needed as we have `chown airflow.airflow
${AIRFLOW_HOME}` earlier in the file - this directory must already exist
otherwise that previous command would fail.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
