This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/master by this push:
new 4226f64 Better description of UID/GID behaviour in image and
quickstart (#15592)
4226f64 is described below
commit 4226f640864b0996dc156da244047a34c58be63c
Author: Jarek Potiuk <[email protected]>
AuthorDate: Sun May 2 20:06:57 2021 +0200
Better description of UID/GID behaviour in image and quickstart (#15592)
* Better description of UID/GID behaviour in image and quickstart
Following the discussion in
https://github.com/apache/airflow/discussions/15579
seems that the AIRFLOW_UID/GID parameters were not clearly
explained in the Docker Quick-start guide and some users could
find it confusing.
This PR attempts to clarify it.
* fixup! Better description of UID/GID behaviour in image and quickstart
---
docs/apache-airflow/start/docker.rst | 52 +++++++++++++++++++++++++++++++++++-
docs/docker-stack/entrypoint.rst | 22 +++++++++++++++
2 files changed, 73 insertions(+), 1 deletion(-)
diff --git a/docs/apache-airflow/start/docker.rst
b/docs/apache-airflow/start/docker.rst
index abd63dc..26a9132 100644
--- a/docs/apache-airflow/start/docker.rst
+++ b/docs/apache-airflow/start/docker.rst
@@ -62,6 +62,9 @@ Some directories in the container are mounted, which means
that their contents a
This file uses the latest Airflow image (`apache/airflow
<https://hub.docker.com/r/apache/airflow>`__). If you need install a new Python
library or system library, you can :doc:`customize and extend it
<docker-stack:index>`.
+.. _initializing_docker_compose_environment:
+
+
Initializing Environment
========================
@@ -74,6 +77,8 @@ On **Linux**, the mounted volumes in container use the native
Linux filesystem u
mkdir ./dags ./logs ./plugins
echo -e "AIRFLOW_UID=$(id -u)\nAIRFLOW_GID=0" > .env
+See:ref:`Docker Compose environment variables <docker-compose-env-variables>`
+
On **all operating systems**, you need to run database migrations and create
the first user account. To do it, run.
.. code-block:: bash
@@ -185,7 +190,6 @@ Here is a sample ``curl`` command, which sends a request to
retrieve a pool list
--user "airflow:airflow" \
"${ENDPOINT_URL}/api/v1/pools"
-
Cleaning up
===========
@@ -207,3 +211,49 @@ What's Next?
============
From this point, you can head to the :doc:`/tutorial` section for further
examples or the :doc:`/howto/index` section if you're ready to get your hands
dirty.
+
+.. _docker-compose-env-variables:
+
+Environment variables supported by Docker Compose
+=================================================
+
+Do not confuse the variable names here with the build arguments set when image
is built. The
+``AIRFLOW_UID`` and ``AIRFLOW_GID`` build args default to ``50000`` when the
image is built, so they are
+"baked" into the image. On the other hand, the environment variables below can
be set when the container
+is running, using - for example - result of ``id -u`` command, which allows to
use the dynamic host
+runtime user id which is unknown at the time of building the image.
+
++--------------------------------+-----------------------------------------------------+--------------------------+
+| Variable | Description
| Default |
++================================+=====================================================+==========================+
+| ``AIRFLOW_IMAGE_NAME`` | Airflow Image to use.
| apache/airflow:|version| |
++--------------------------------+-----------------------------------------------------+--------------------------+
+| ``AIRFLOW_UID`` | UID of the user to run Airflow containers
as. | ``50000`` |
+| | Override if you want to use use non-default
Airflow | |
+| | UID (for example when you map folders from
host, | |
+| | it should be set to result of ``id -u``
call. If | |
+| | you change it from default 50000, you must
set | |
+| | ``AIRFLOW_GID`` to ``0``. When it is
changed, | |
+| | a 2nd user with the UID specified is
dynamically | |
+| | created with ``default`` name inside the
container | |
+| | and home of the use is set to
``/airflow/home/`` | |
+| | in order to share Python libraries
installed there. | |
+| | This is in order to achieve the OpenShift
| |
+| | compatibility. See more in the
| |
+| | :ref:`Arbitrary Docker User
<arbitrary-docker-user>`| |
++--------------------------------+-----------------------------------------------------+--------------------------+
+| ``AIRFLOW_GID`` | Group ID in Airflow containers. It
overrides the | ``50000`` |
+| | GID of the user. It is ``50000`` by default
but if | |
+| | you want to use different UID than default
it must | |
+| | be set to ``0``.
| |
++--------------------------------+-----------------------------------------------------+--------------------------+
+| ``_AIRFLOW_WWW_USER_USERNAME`` | Username for the administrator UI account.
| |
+| | If this value is specified, admin UI user
gets | |
+| | created automatically. This is only useful
when | |
+| | you want to run Airflow for a test-drive
and | |
+| | want to start a container with embedded
development | |
+| | database.
| |
++--------------------------------+-----------------------------------------------------+--------------------------+
+| ``_AIRFLOW_WWW_USER_PASSWORD`` | Password for the administrator UI account.
| |
+| | Only used when
``_AIRFLOW_WWW_USER_USERNAME`` set. | |
++--------------------------------+-----------------------------------------------------+--------------------------+
diff --git a/docs/docker-stack/entrypoint.rst b/docs/docker-stack/entrypoint.rst
index cc89872..4daa7d2 100644
--- a/docs/docker-stack/entrypoint.rst
+++ b/docs/docker-stack/entrypoint.rst
@@ -96,6 +96,28 @@ Airflow image is Open-Shift compatible, which means that you
can start it with r
group id ``0`` (``root``). If you want to run the image with user different
than Airflow, you MUST set
GID of the user to ``0``. In case you try to use different group, the
entrypoint exits with error.
+OpenShift randomly assigns UID when it starts the container, but you can
utilise this flexible UID
+also in case of running the image manually. This might be useful for example
in case you want to
+mount ``dag`` and ``logs`` folders from host system on Linux, in which case
the UID should be set
+the same ID as your host user.
+
+This can be achieved in various ways - you can change USER when you extend or
customize the image or
+you can dynamically pass the user to ``docker run`` command, by adding
``--user`` flag in one of
+those formats (See `Docker Run reference
<https://docs.docker.com/engine/reference/run/#user>`_ for details):
+
+```
+[ user | user:group | uid | uid:gid | user:gid | uid:group ]
+```
+
+In case of Docker Compose environment it can be changed via ``user:`` entry in
the ``docker-compose.yaml``.
+See `Docker compose reference
<https://docs.docker.com/compose/compose-file/compose-file-v3/#domainname-hostname-ipc-mac_address-privileged-read_only-shm_size-stdin_open-tty-user-working_dir>`_
+for details. In our Quickstart Guide using Docker-Compose, the UID and GID can
be passed via
+``AIRFLOW_UID`` and ``AIRFLOW_GID`` variables as described in
+:ref:`Initializing docker compose environment
<initializing_docker_compose_environment>`.
+
+In case ``GID`` is set to ``0``, the user can be any UID, but in case UID is
different than the default
+``airflow`` (UID=50000), the user will be automatically created when entering
the container.
+
In order to accommodate a number of external libraries and projects, Airflow
will automatically create
such an arbitrary user in (`/etc/passwd`) and make it's home directory point
to ``/home/airflow``.
Many of 3rd-party libraries and packages require home directory of the user to
be present, because they
