This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch v2-4-test in repository https://gitbox.apache.org/repos/asf/airflow.git
commit b68fb0960fc288ada3c3de2643c44f0113f0a241 Author: Jarek Potiuk <[email protected]> AuthorDate: Mon Sep 26 00:19:08 2022 +0200 Add requirements.txt example to "extending docker" (#26663) When extending the image, there was an example of adding individual packages via PyPI, but not requirements.txt. This PR fixes it. (cherry picked from commit 25fdb2cc9a48a335079ff4a8d1be986e62a39f29) --- docs/docker-stack/build.rst | 34 ++++++++++++++++++++-- .../extending/add-requirement-packages/Dockerfile | 21 +++++++++++++ .../add-requirement-packages/requirements.txt | 2 ++ 3 files changed, 54 insertions(+), 3 deletions(-) diff --git a/docs/docker-stack/build.rst b/docs/docker-stack/build.rst index 1721615953..5849e17709 100644 --- a/docs/docker-stack/build.rst +++ b/docs/docker-stack/build.rst @@ -27,7 +27,9 @@ Quick start scenarios of image extending ---------------------------------------- The most common scenarios where you want to build your own image are adding a new ``apt`` package, -adding a new ``PyPI`` dependency and embedding DAGs into the image. +adding a new ``PyPI`` dependency (either individually or via requirements.txt) and embedding DAGs +into the image. + Example Dockerfiles for those scenarios are below, and you can read further for more complex cases which might involve either extending or customizing the image. You will find more information about more complex scenarios below, but if your goal is to quickly extend the Airflow @@ -46,8 +48,8 @@ switch to the ``root`` user when running the ``apt`` commands, but do not forget :end-before: [END Dockerfile] -Adding a new ``PyPI`` package -............................. +Adding new ``PyPI`` packages individually +......................................... The following example adds ``lxml`` python package from PyPI to the image. When adding packages via ``pip`` you need to use the ``airflow`` user rather than ``root``. Attempts to install ``pip`` packages @@ -58,6 +60,19 @@ as ``root`` will fail with an appropriate error message. :start-after: [START Dockerfile] :end-before: [END Dockerfile] +Adding packages from requirements.txt +..................................... + +The following example adds few python packages from ``requirements.txt`` from PyPI to the image. +Note that similarly when adding individual packages, you need to use the ``airflow`` user rather than +``root``. Attempts to install ``pip`` packages as ``root`` will fail with an appropriate error message. + +.. exampleinclude:: docker-examples/extending/add-requirement-packages/Dockerfile + :language: Dockerfile + :start-after: [START Dockerfile] + :end-before: [END Dockerfile] + + Embedding DAGs .............. @@ -363,6 +378,19 @@ The following example adds ``lxml`` python package from PyPI to the image. :start-after: [START Dockerfile] :end-before: [END Dockerfile] +Example of adding packages from requirements.txt +................................................ + +The following example adds few python packages from ``requirements.txt`` from PyPI to the image. +Note that similarly when adding individual packages, you need to use the ``airflow`` user rather than +``root``. Attempts to install ``pip`` packages as ``root`` will fail with an appropriate error message. + +.. exampleinclude:: docker-examples/extending/add-requirement-packages/Dockerfile + :language: Dockerfile + :start-after: [START Dockerfile] + :end-before: [END Dockerfile] + + Example when writable directory is needed ......................................... diff --git a/docs/docker-stack/docker-examples/extending/add-requirement-packages/Dockerfile b/docs/docker-stack/docker-examples/extending/add-requirement-packages/Dockerfile new file mode 100644 index 0000000000..503ba3ae7d --- /dev/null +++ b/docs/docker-stack/docker-examples/extending/add-requirement-packages/Dockerfile @@ -0,0 +1,21 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# This is an example Dockerfile. It is not intended for PRODUCTION use +# [START Dockerfile] +FROM apache/airflow:2.5.0.dev0 +COPY requirements.txt / +RUN pip install --no-cache-dir -r /requirements.txt +# [END Dockerfile] diff --git a/docs/docker-stack/docker-examples/extending/add-requirement-packages/requirements.txt b/docs/docker-stack/docker-examples/extending/add-requirement-packages/requirements.txt new file mode 100644 index 0000000000..595508023e --- /dev/null +++ b/docs/docker-stack/docker-examples/extending/add-requirement-packages/requirements.txt @@ -0,0 +1,2 @@ +lxml +beautifulsoup4
