jedcunningham commented on a change in pull request #17591: URL: https://github.com/apache/airflow/pull/17591#discussion_r688652693
########## File path: docs/apache-airflow/logging-monitoring/logging-tasks.rst ########## @@ -107,3 +107,16 @@ External Links When using remote logging, users can configure Airflow to show a link to an external UI within the Airflow Web UI. Clicking the link redirects a user to the external UI. Some external systems require specific configuration in Airflow for redirection to work but others do not. + +Serving logs from workers +------------------------- + +Most task handlers send logs upon completion of a task. In order to view the log in real time, airflow starts the server serving the log in the following cases: + +- If ``SchedulerExecutor`` or ``LocalExecutor`` is used, then after running the ``airflow scheduler`` command. +- If ``CeleryExecutor`` is used, then after running the ``airflow worker`` command. + +The server is running on the port specified by ``worker_log_server_port`` option in ``celery`` section. By default, it is ``8793``. +Communication between the webserver and the worker is signed with the key specified by ``secret_key`` option in ``webserver`` section. You must ensure that the key matches so that communication can take place without problems. + +We are using `Gunicorm <https://gunicorn.org/>`__ as WSGI server. Its configuration options can be overridden by ``GUNiCORN_CMD_ARGS`` env variable. For details, see `Gunicorn settings <https://docs.gunicorn.org/en/latest/settings.html#settings>`__ Review comment: ```suggestion We are using `Gunicorm <https://gunicorn.org/>`__ as a WSGI server. Its configuration options can be overridden with the ``GUNICORN_CMD_ARGS`` env variable. For details, see `Gunicorn settings <https://docs.gunicorn.org/en/latest/settings.html#settings>`__. ``` ########## File path: docs/apache-airflow/logging-monitoring/logging-tasks.rst ########## @@ -107,3 +107,16 @@ External Links When using remote logging, users can configure Airflow to show a link to an external UI within the Airflow Web UI. Clicking the link redirects a user to the external UI. Some external systems require specific configuration in Airflow for redirection to work but others do not. + +Serving logs from workers +------------------------- + +Most task handlers send logs upon completion of a task. In order to view the log in real time, airflow starts the server serving the log in the following cases: + +- If ``SchedulerExecutor`` or ``LocalExecutor`` is used, then after running the ``airflow scheduler`` command. +- If ``CeleryExecutor`` is used, then after running the ``airflow worker`` command. Review comment: ```suggestion - If ``SchedulerExecutor`` or ``LocalExecutor`` is used, then when ``airflow scheduler`` is running. - If ``CeleryExecutor`` is used, then when ``airflow worker`` is running. ``` Still not 100% satisfied with the wording here 🤷♂️ ########## File path: docs/apache-airflow/logging-monitoring/logging-tasks.rst ########## @@ -107,3 +107,16 @@ External Links When using remote logging, users can configure Airflow to show a link to an external UI within the Airflow Web UI. Clicking the link redirects a user to the external UI. Some external systems require specific configuration in Airflow for redirection to work but others do not. + +Serving logs from workers +------------------------- + +Most task handlers send logs upon completion of a task. In order to view the log in real time, airflow starts the server serving the log in the following cases: Review comment: ```suggestion Most task handlers send logs upon completion of a task. In order to view logs in real time, airflow automatically starts an http server to serve the logs in the following cases: ``` ########## File path: docs/apache-airflow/logging-monitoring/logging-tasks.rst ########## @@ -107,3 +107,16 @@ External Links When using remote logging, users can configure Airflow to show a link to an external UI within the Airflow Web UI. Clicking the link redirects a user to the external UI. Some external systems require specific configuration in Airflow for redirection to work but others do not. + +Serving logs from workers +------------------------- + +Most task handlers send logs upon completion of a task. In order to view the log in real time, airflow starts the server serving the log in the following cases: + +- If ``SchedulerExecutor`` or ``LocalExecutor`` is used, then after running the ``airflow scheduler`` command. +- If ``CeleryExecutor`` is used, then after running the ``airflow worker`` command. + +The server is running on the port specified by ``worker_log_server_port`` option in ``celery`` section. By default, it is ``8793``. +Communication between the webserver and the worker is signed with the key specified by ``secret_key`` option in ``webserver`` section. You must ensure that the key matches so that communication can take place without problems. + +We are using `Gunicorm <https://gunicorn.org/>`__ as WSGI server and that the configuration options can be overridden by ``GUNiCORN_CMD_ARGS`` env variable. For details, see `Gunicorn settings <https://docs.gunicorn.org/en/latest/settings.html#settings>`__ Review comment: ```suggestion We are using `Gunicorm <https://gunicorn.org/>`__ as a WSGI server and configuration options can be overridden with the ``GUNICORN_CMD_ARGS`` env variable. For details, see `Gunicorn settings <https://docs.gunicorn.org/en/latest/settings.html#settings>`__. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
