Thanks, I would prefer the logs not to be in a shared location but it's not a blocker.
Is there an option no to run the log server when using the "airflow worker" command? I'm looking at the cli docs and nothing jumps out to me: https://airflow.apache.org/docs/stable/cli.html#worker Damian From: Aaron Grubb <[email protected]> Sent: Monday, December 2, 2019 3:40 PM To: [email protected] Subject: RE: How do I get logs to display for multiple workers on the same machine? My understanding of the log server is the server points at a single directory that you specify in your config and serves the logs based on the DAG (test_data), execution date and attempt index. Therefore, have each worker instance write its logs to a shared folder that's writeable by each user and readable by the log server and only run one instance of the log server. From: Shaw, Damian P. <[email protected]<mailto:[email protected]>> Sent: Monday, December 2, 2019 3:34 PM To: [email protected]<mailto:[email protected]> Subject: How do I get logs to display for multiple workers on the same machine? I have a requirement to run different celery workers on the same machine under different user accounts. If I start the second worker with "worker_log_server_port = 8793" I get an error that that port is already bound to. If I start the second worker with "worker_log_server_port = 8794" I get an error when the webserver tries to find the logs: *** Failed to fetch log file from worker. 404 Client Error: NOT FOUND for url: http://myhostname:8793/log/test_executor/test_data/2019-12-01T12:00:00+00:00/1.log Is there any way I can serve logs from different users from the same machine? Thanks, Damian ============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html ============================================================================== =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html ===============================================================================
