This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch v2-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v2-3-test by this push:
new a791f3a7f8 Disable Flower by default from docker-compose (#23685)
a791f3a7f8 is described below
commit a791f3a7f8f1b10e90897c4bf6d59ac51b42baa9
Author: Jarek Potiuk <[email protected]>
AuthorDate: Fri May 13 12:21:36 2022 +0200
Disable Flower by default from docker-compose (#23685)
(cherry picked from commit 6eda265a41fc7c48692305b27c64f811ff02fb94)
---
docs/apache-airflow/executor/kubernetes.rst | 5 +++-
docs/apache-airflow/howto/run-behind-proxy.rst | 2 +-
.../logging-monitoring/check-health.rst | 2 +-
docs/apache-airflow/security/flower.rst | 3 ++
docs/apache-airflow/start/docker-compose.yaml | 32 +++++++++++-----------
docs/apache-airflow/start/docker.rst | 7 +++--
6 files changed, 30 insertions(+), 21 deletions(-)
diff --git a/docs/apache-airflow/executor/kubernetes.rst
b/docs/apache-airflow/executor/kubernetes.rst
index 72f441026e..a59fe5c0e4 100644
--- a/docs/apache-airflow/executor/kubernetes.rst
+++ b/docs/apache-airflow/executor/kubernetes.rst
@@ -214,7 +214,10 @@ To get task logs out of the workers, you can:
Comparison with CeleryExecutor
------------------------------
-In contrast to CeleryExecutor, KubernetesExecutor does not require additional
components such as Redis and Flower, but does require access to Kubernetes
cluster.
+In contrast to CeleryExecutor, KubernetesExecutor does not require additional
components such as Redis,
+but does require access to Kubernetes cluster.
+
+Also monitoring the Pods can be done with the built-in Kubernetes monitoring.
With KubernetesExecutor, each task runs in its own pod. The pod is created
when the task is queued, and terminates when the task completes.
Historically, in scenarios such as burstable workloads, this presented a
resource utilization advantage over CeleryExecutor, where you needed
diff --git a/docs/apache-airflow/howto/run-behind-proxy.rst
b/docs/apache-airflow/howto/run-behind-proxy.rst
index 2901ed01fe..ee7169fe6b 100644
--- a/docs/apache-airflow/howto/run-behind-proxy.rst
+++ b/docs/apache-airflow/howto/run-behind-proxy.rst
@@ -33,7 +33,7 @@ To do so, you need to set the following setting in your
``airflow.cfg``::
base_url = http://my_host/myorg/airflow
-Additionally if you use Celery Executor, you can get Flower in
``/myorg/flower`` with::
+Additionally if you use Celery Executor, and you enable flower, you can get
Flower in ``/myorg/flower`` with::
flower_url_prefix = /myorg/flower
diff --git a/docs/apache-airflow/logging-monitoring/check-health.rst
b/docs/apache-airflow/logging-monitoring/check-health.rst
index c6db2cb0bf..f485483276 100644
--- a/docs/apache-airflow/logging-monitoring/check-health.rst
+++ b/docs/apache-airflow/logging-monitoring/check-health.rst
@@ -103,7 +103,7 @@ with a non-zero error code.
HTTP monitoring for Celery Cluster
----------------------------------
-You can use Flower to monitor the health of the Celery cluster. It also
provides an HTTP API that you can use to build a health check for your
environment.
+You can optionally use Flower to monitor the health of the Celery cluster. It
also provides an HTTP API that you can use to build a health check for your
environment.
For details about installation, see: :ref:`executor:CeleryExecutor`. For
details about usage, see: `The Flower project documentation
<https://flower.readthedocs.io/>`__.
diff --git a/docs/apache-airflow/security/flower.rst
b/docs/apache-airflow/security/flower.rst
index 5aafd80fb0..b1de4a790c 100644
--- a/docs/apache-airflow/security/flower.rst
+++ b/docs/apache-airflow/security/flower.rst
@@ -21,6 +21,9 @@ Flower
Flower is a web based tool for monitoring and administrating Celery clusters.
This topic describes how
to configure Airflow to secure your flower instance.
+This is an optional component that is disabled by default in Community
deployments and you need to
+configure it on your own if you want to use it.
+
Flower Authentication
---------------------
diff --git a/docs/apache-airflow/start/docker-compose.yaml
b/docs/apache-airflow/start/docker-compose.yaml
index cb58d2d89c..9b5f2e1167 100644
--- a/docs/apache-airflow/start/docker-compose.yaml
+++ b/docs/apache-airflow/start/docker-compose.yaml
@@ -252,22 +252,22 @@ services:
- bash
- -c
- airflow
-
- flower:
- <<: *airflow-common
- command: celery flower
- ports:
- - 5555:5555
- healthcheck:
- test: ["CMD", "curl", "--fail", "http://localhost:5555/";]
- interval: 10s
- timeout: 10s
- retries: 5
- restart: always
- depends_on:
- <<: *airflow-common-depends-on
- airflow-init:
- condition: service_completed_successfully
+# you can enable flower by uncommenting those lines
+# flower:
+# <<: *airflow-common
+# command: celery flower
+# ports:
+# - 5555:5555
+# healthcheck:
+# test: ["CMD", "curl", "--fail", "http://localhost:5555/";]
+# interval: 10s
+# timeout: 10s
+# retries: 5
+# restart: always
+# depends_on:
+# <<: *airflow-common-depends-on
+# airflow-init:
+# condition: service_completed_successfully
volumes:
postgres-db-volume:
diff --git a/docs/apache-airflow/start/docker.rst
b/docs/apache-airflow/start/docker.rst
index 305415d72d..a0e72f5ae5 100644
--- a/docs/apache-airflow/start/docker.rst
+++ b/docs/apache-airflow/start/docker.rst
@@ -98,10 +98,14 @@ This file contains several service definitions:
- ``airflow-webserver`` - The webserver is available at
``http://localhost:8080``.
- ``airflow-worker`` - The worker that executes the tasks given by the
scheduler.
- ``airflow-init`` - The initialization service.
-- ``flower`` - `The flower app <https://flower.readthedocs.io/en/latest/>`__
for monitoring the environment. It is available at ``http://localhost:5555``.
- ``postgres`` - The database.
- ``redis`` - `The redis <https://redis.io/>`__ - broker that forwards
messages from scheduler to worker.
+Optionally you can enable flower by uncommenting the commented lines if you
want to be able to monitor celery workers separately.
+
+- ``flower`` - `The flower app <https://flower.readthedocs.io/en/latest/>`__
for monitoring the environment.
+ It is available at ``http://localhost:5555``.
+
In general, if you want to use airflow locally, your DAGs may try to connect
to servers which are running on the host. In order to achieve that, an extra
configuration must be added in ``docker-compose.yaml``. For example, on Linux
the configuration must be in the section ``services: airflow-worker`` adding
``extra_hosts: - "host.docker.internal:host-gateway"``; and use
``host.docker.internal`` instead of ``localhost``. This configuration vary in
different platforms. Please, see document [...]
All these services allow you to run Airflow with :doc:`CeleryExecutor
</executor/celery>`. For more information, see :doc:`/concepts/overview`.
@@ -211,7 +215,6 @@ In the second terminal you can check the condition of the
containers and make su
CONTAINER ID IMAGE |version-spacepad| COMMAND
CREATED STATUS PORTS
NAMES
247ebe6cf87a apache/airflow:|version| "/usr/bin/dumb-init …" 3
minutes ago Up 3 minutes (healthy) 8080/tcp
compose_airflow-worker_1
ed9b09fc84b1 apache/airflow:|version| "/usr/bin/dumb-init …" 3
minutes ago Up 3 minutes (healthy) 8080/tcp
compose_airflow-scheduler_1
- 65ac1da2c219 apache/airflow:|version| "/usr/bin/dumb-init …" 3
minutes ago Up 3 minutes (healthy) 0.0.0.0:5555->5555/tcp, 8080/tcp
compose_flower_1
7cb1fb603a98 apache/airflow:|version| "/usr/bin/dumb-init …" 3
minutes ago Up 3 minutes (healthy) 0.0.0.0:8080->8080/tcp
compose_airflow-webserver_1
74f3bbe506eb postgres:13 |version-spacepad| "docker-entrypoint.s…"
18 minutes ago Up 17 minutes (healthy) 5432/tcp
compose_postgres_1
0bd6576d23cb redis:latest |version-spacepad| "docker-entrypoint.s…"
10 hours ago Up 17 minutes (healthy) 0.0.0.0:6379->6379/tcp
compose_redis_1
