jason810496 commented on code in PR #72797: URL: https://github.com/apache/airflow/pull/72797#discussion_r3976969416
########## scripts/systemd/README: ########## @@ -9,3 +9,25 @@ By default the environment configuration points to /etc/sysconfig/airflow . You directory and adjust it to your liking. With some minor changes they probably work on other systemd systems. + +Required services +----------------- + +Since Airflow 3.0 the airflow processes that handle the different components are split into separate services. At a +minimum you need to run the following services: + +* airflow-scheduler.service +* airflow-dag-processor.service +* airflow-webserver.service or airflow-api.service Review Comment: The same Airflow 3 correction is needed here: `airflow-webserver.service` invokes the removed `airflow webserver` command. ```suggestion * airflow-api.service ``` ########## airflow-core/docs/howto/run-with-systemd.rst: ########## @@ -67,5 +67,24 @@ New Airflow 3.0 Services Since Apache Airflow 3.0, additional components have been split out into separate services. The following new unit files are available: +- ``airflow-dag-processor.service`` for Dag file parsing - ``airflow-triggerer.service`` for deferrable task triggering - ``airflow-api.service`` for the standalone REST API server + +Required services +''''''''''''''''' + +At a minimum, you must run the ``scheduler``, the ``dag-processor`` and either the ``webserver`` or the ``api-server``: + +- ``airflow-scheduler.service`` +- ``airflow-dag-processor.service`` +- ``airflow-webserver.service`` or ``airflow-api.service`` + +Without a running Dag processor, Dag files are never parsed and the ``dag_processor`` entry of the +``/api/v2/monitor/health`` endpoint reports its status as ``unhealthy``. + +The ``airflow-triggerer.service`` is optional: start it only if you use deferrable tasks or +event-driven :doc:`triggers <../authoring-and-scheduling/event-scheduling>`. If it is not running, the +``triggerer`` entry of the health endpoint reports ``unhealthy``, which is expected and safe to ignore. Review Comment: `get_airflow_health()` returns `null` for both status and heartbeat when no corresponding job record exists. An existing job that is no longer alive reports `unhealthy`. Let's distinguish those cases and limit the instruction to ignore triggerer health to deployments that intentionally do not need it. ```suggestion Without a running Dag processor, Dag files are not parsed. The ``airflow-triggerer.service`` is optional if you do not use deferrable tasks or event-driven :doc:`triggers <../authoring-and-scheduling/event-scheduling>`. For both ``dag_processor`` and ``triggerer``, the ``/api/v2/monitor/health`` endpoint returns ``null`` for the status and latest heartbeat when no corresponding job record exists. If a job record exists but the job is no longer alive, its status is ``unhealthy``. You can ignore the ``triggerer`` health status only when your deployment intentionally does not use a triggerer. ``` ########## airflow-core/docs/howto/run-with-systemd.rst: ########## @@ -67,5 +67,24 @@ New Airflow 3.0 Services Since Apache Airflow 3.0, additional components have been split out into separate services. The following new unit files are available: +- ``airflow-dag-processor.service`` for Dag file parsing - ``airflow-triggerer.service`` for deferrable task triggering - ``airflow-api.service`` for the standalone REST API server + +Required services +''''''''''''''''' + +At a minimum, you must run the ``scheduler``, the ``dag-processor`` and either the ``webserver`` or the ``api-server``: + +- ``airflow-scheduler.service`` +- ``airflow-dag-processor.service`` +- ``airflow-webserver.service`` or ``airflow-api.service`` Review Comment: Airflow 3 removed `airflow webserver`; the legacy command raises an error directing users to `airflow api-server`. The supplied webserver unit still invokes the removed command, so it cannot be an alternative here. ```suggestion At a minimum, you must run the ``scheduler``, the ``dag-processor`` and the ``api-server``: - ``airflow-scheduler.service`` - ``airflow-dag-processor.service`` - ``airflow-api.service`` ``` ########## scripts/systemd/README: ########## @@ -9,3 +9,25 @@ By default the environment configuration points to /etc/sysconfig/airflow . You directory and adjust it to your liking. With some minor changes they probably work on other systemd systems. + +Required services +----------------- + +Since Airflow 3.0 the airflow processes that handle the different components are split into separate services. At a +minimum you need to run the following services: + +* airflow-scheduler.service +* airflow-dag-processor.service +* airflow-webserver.service or airflow-api.service + +Additionally the following services are available and can be enabled as needed: + +* airflow-triggerer.service for deferrable task triggering (required only if you use deferrable tasks or event + scheduling) +* airflow-worker.service for a Celery worker +* airflow-flower.service for a Flower monitoring interface +* airflow-kerberos.service for the Kerberos renewal daemon + +The ``/api/v2/monitor/health`` endpoint reports ``dag_processor`` and ``triggerer`` as ``unhealthy`` when no +corresponding service is running, so make sure the services you depend on are started. See the "Checking Airflow +Health Status" documentation for details. Review Comment: Keep this consistent with the actual health endpoint: no job record gives `null`; an existing job that is no longer alive gives `unhealthy`. ```suggestion For both ``dag_processor`` and ``triggerer``, the ``/api/v2/monitor/health`` endpoint returns ``null`` for the status and latest heartbeat when no corresponding job record exists. If a job record exists but the job is no longer alive, its status is ``unhealthy``. Make sure the services you depend on are started; ignore the triggerer health status only when your deployment intentionally does not use a triggerer. See the "Checking Airflow Health Status" documentation for details. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
