potiuk commented on a change in pull request #17591:
URL: https://github.com/apache/airflow/pull/17591#discussion_r688468320
##########
File path: airflow/utils/serve_logs.py
##########
@@ -73,10 +74,47 @@ def serve_logs_view(filename):
return flask_app
+class StandaloneGunicornApplication(gunicorn.app.base.BaseApplication):
+ """
+ Standalone Gunicorn application/serve for usage with any WSGI-application.
+
+ Code inspired by an example from the Gunicorn documentation.
+
https://github.com/benoitc/gunicorn/blob/cf55d2cec277f220ebd605989ce78ad1bb553c46/examples/standalone_app.py
+
+ For details, about standalone gunicorn application, see:
+ https://docs.gunicorn.org/en/stable/custom.html
+ """
+
+ def __init__(self, app, options=None):
+ self.options = options or {}
+ self.application = app
+ super().__init__()
+
+ def load_config(self):
+ config = {
+ key: value
+ for key, value in self.options.items()
+ if key in self.cfg.settings and value is not None
+ }
+ for key, value in config.items():
+ self.cfg.set(key.lower(), value)
+
+ def load(self):
+ return self.application
+
+
def serve_logs():
"""Serves logs generated by Worker"""
setproctitle("airflow serve-logs")
- app = flask_app()
+ wsgi_app = create_app()
worker_log_server_port = conf.getint('celery', 'WORKER_LOG_SERVER_PORT')
- app.run(host='0.0.0.0', port=worker_log_server_port)
+ options = {
+ 'bind': f"0.0.0.0:{worker_log_server_port}",
+ 'workers': 2,
Review comment:
Yeah 2 is better just in case of memory errors/crashes. Just one small
nit (in case somene has problems with memory usage etc.) i think it would be
great to mention in the docs of worker that we are using Gunicorm and that the
configuration options can be overridden by GUNiCORN_CMD_ARGS env variable
https://docs.gunicorn.org/en/latest/settings.html#settings
It's not at all obvious from docs without looking at the code now that wlwe
have separate Gunicorm processes forked and that you can configure their
behaviour via ENV vars.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
