This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new d722fba54e Add description in a few more places about adding airflow
to pip install (#31448)
d722fba54e is described below
commit d722fba54e844b83025a7189b28371fba7e1af88
Author: Jarek Potiuk <[email protected]>
AuthorDate: Mon May 22 00:23:59 2023 +0200
Add description in a few more places about adding airflow to pip install
(#31448)
It is important to keep airflow as a dependency with pinned version if
you are adding new dependencies. This PR adds it in case of
docker-compose documentation and also improves examples in our
images, by using AIRFLOW_VERSION variable defined in the images. This
way users can write future-proof ways of installind dependencies basing
on our examples.
Follow-up after #30813
---
docker_tests/test_examples_of_prod_image_building.py | 1 -
docs/apache-airflow/howto/docker-compose/index.rst | 11 ++++++++---
docs/apache-airflow/installation/installing-from-pypi.rst | 9 +++++++++
.../extending/add-build-essential-extend/Dockerfile | 2 +-
.../docker-examples/extending/add-providers/Dockerfile | 2 +-
.../docker-examples/extending/add-pypi-packages/Dockerfile | 2 +-
.../extending/add-requirement-packages/Dockerfile | 2 +-
.../docker-examples/extending/custom-providers/Dockerfile | 2 +-
8 files changed, 22 insertions(+), 9 deletions(-)
diff --git a/docker_tests/test_examples_of_prod_image_building.py
b/docker_tests/test_examples_of_prod_image_building.py
index 178d62680a..f67efcd067 100644
--- a/docker_tests/test_examples_of_prod_image_building.py
+++ b/docker_tests/test_examples_of_prod_image_building.py
@@ -59,7 +59,6 @@ def test_dockerfile_example(dockerfile):
new_content = re.sub(
r"FROM apache/airflow:.*", rf"FROM
apache/airflow:{latest_released_version}", content
)
- new_content = re.sub(r"apache-airflow==\S*",
rf"apache-airflow=={latest_released_version}", new_content)
try:
run_command(
["docker", "build", ".", "--tag", image_name, "-f", "-"],
diff --git a/docs/apache-airflow/howto/docker-compose/index.rst
b/docs/apache-airflow/howto/docker-compose/index.rst
index b18f54097d..1ab56b2347 100644
--- a/docs/apache-airflow/howto/docker-compose/index.rst
+++ b/docs/apache-airflow/howto/docker-compose/index.rst
@@ -307,18 +307,23 @@ you should do those steps:
to (use correct image tag):
```
-#image: ${AIRFLOW_IMAGE_NAME:-apache/airflow:2.5.1}
+#image: ${AIRFLOW_IMAGE_NAME:-apache/airflow:2.6.1}
build: .
```
2) Create ``Dockerfile`` in the same folder your ``docker-compose.yaml`` file
is with content similar to:
```
-FROM apache/airflow:2.5.1
+FROM apache/airflow:2.6.1
ADD requirements.txt .
-RUN pip install -r requirements.txt
+RUN pip install apache-airflow==${AIRFLOW_VERSION} -r requirements.txt
```
+It is the best practice to install apache-airflow in the same version as the
one that comes from the
+original image. This way you can be sure that ``pip`` will not try to
downgrade or upgrade apache
+airflow while installing other requirements, which might happen in case you
try to add a dependency
+that conflicts with the version of apache-airflow that you are using.
+
3) Place ``requirements.txt`` file in the same directory.
Run ``docker compose build`` to build the image, or add ``--build`` flag to
``docker compose up`` or
diff --git a/docs/apache-airflow/installation/installing-from-pypi.rst
b/docs/apache-airflow/installation/installing-from-pypi.rst
index e6cf162bc3..97cf3ba41c 100644
--- a/docs/apache-airflow/installation/installing-from-pypi.rst
+++ b/docs/apache-airflow/installation/installing-from-pypi.rst
@@ -181,6 +181,15 @@ Airflow versions or other providers. Some providers have
minimum-required versio
versions of providers might have conflicting requirements with Airflow or
other dependencies you
might have installed.
+It is the best practice to install apache-airflow in the same version as the
one that comes from the
+original image. This way you can be sure that ``pip`` will not try to
downgrade or upgrade apache
+airflow while installing other requirements, which might happen in case you
try to add a dependency
+that conflicts with the version of apache-airflow that you are using:
+
+.. code-block:: bash
+
+ pip install "apache-airflow==|version|"
"apache-airflow-providers-google==8.0.0"
+
Installation and upgrade of Airflow core
========================================
diff --git
a/docs/docker-stack/docker-examples/extending/add-build-essential-extend/Dockerfile
b/docs/docker-stack/docker-examples/extending/add-build-essential-extend/Dockerfile
index afdbee23ce..584827a2ca 100644
---
a/docs/docker-stack/docker-examples/extending/add-build-essential-extend/Dockerfile
+++
b/docs/docker-stack/docker-examples/extending/add-build-essential-extend/Dockerfile
@@ -24,5 +24,5 @@ RUN apt-get update \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
USER airflow
-RUN pip install --no-cache-dir mpi4py
+RUN pip install --no-cache-dir "apache-airflow==${AIRFLOW_VERSION}" mpi4py
# [END Dockerfile]
diff --git
a/docs/docker-stack/docker-examples/extending/add-providers/Dockerfile
b/docs/docker-stack/docker-examples/extending/add-providers/Dockerfile
index ab14642a8c..beab3866ac 100644
--- a/docs/docker-stack/docker-examples/extending/add-providers/Dockerfile
+++ b/docs/docker-stack/docker-examples/extending/add-providers/Dockerfile
@@ -25,5 +25,5 @@ RUN apt-get update \
&& rm -rf /var/lib/apt/lists/*
USER airflow
ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
-RUN pip install --no-cache-dir apache-airflow-providers-apache-spark==2.1.3
apache-airflow==2.6.0dev0
+RUN pip install --no-cache-dir "apache-airflow==${AIRFLOW_VERSION}"
apache-airflow-providers-apache-spark==2.1.3
# [END Dockerfile]
diff --git
a/docs/docker-stack/docker-examples/extending/add-pypi-packages/Dockerfile
b/docs/docker-stack/docker-examples/extending/add-pypi-packages/Dockerfile
index 0c3864118f..988b3a9819 100644
--- a/docs/docker-stack/docker-examples/extending/add-pypi-packages/Dockerfile
+++ b/docs/docker-stack/docker-examples/extending/add-pypi-packages/Dockerfile
@@ -16,5 +16,5 @@
# This is an example Dockerfile. It is not intended for PRODUCTION use
# [START Dockerfile]
FROM apache/airflow:2.7.0.dev0
-RUN pip install --no-cache-dir lxml apache-airflow==2.7.0.dev0
+RUN pip install --no-cache-dir "apache-airflow==${AIRFLOW_VERSION}" lxml
# [END Dockerfile]
diff --git
a/docs/docker-stack/docker-examples/extending/add-requirement-packages/Dockerfile
b/docs/docker-stack/docker-examples/extending/add-requirement-packages/Dockerfile
index df3e6c34d3..4d6e084f3c 100644
---
a/docs/docker-stack/docker-examples/extending/add-requirement-packages/Dockerfile
+++
b/docs/docker-stack/docker-examples/extending/add-requirement-packages/Dockerfile
@@ -17,5 +17,5 @@
# [START Dockerfile]
FROM apache/airflow:2.7.0.dev0
COPY requirements.txt /
-RUN pip install --no-cache-dir -r /requirements.txt apache-airflow==2.7.0.dev0
+RUN pip install --no-cache-dir "apache-airflow==${AIRFLOW_VERSION}" -r
/requirements.txt
# [END Dockerfile]
diff --git
a/docs/docker-stack/docker-examples/extending/custom-providers/Dockerfile
b/docs/docker-stack/docker-examples/extending/custom-providers/Dockerfile
index 0a77b192e2..42a25a0154 100644
--- a/docs/docker-stack/docker-examples/extending/custom-providers/Dockerfile
+++ b/docs/docker-stack/docker-examples/extending/custom-providers/Dockerfile
@@ -16,5 +16,5 @@
# This is an example Dockerfile. It is not intended for PRODUCTION use
# [START Dockerfile]
FROM apache/airflow:2.7.0.dev0
-RUN pip install --no-cache-dir apache-airflow-providers-docker==2.5.1
apache-airflow==2.7.0.dev0
+RUN pip install "apache-airflow==${AIRFLOW_VERSION}" --no-cache-dir
apache-airflow-providers-docker==2.5.1
# [END Dockerfile]
