opendatakraken opened a new issue #18937: URL: https://github.com/apache/airflow/issues/18937
### Apache Airflow version 2.2.0 (latest released) ### Operating System Alpine on Kubernetes ### Versions of Apache Airflow Providers apache-airflow-providers-apache-hdfs==2.1.1 apache-airflow-providers-cncf-kubernetes==2.0.3 apache-airflow-providers-elasticsearch==2.0.3 apache-airflow-providers-exasol==2.0.1 apache-airflow-providers-ftp==2.0.1 apache-airflow-providers-http==2.0.1 apache-airflow-providers-imap==2.0.1 apache-airflow-providers-jdbc==2.0.1 apache-airflow-providers-microsoft-mssql==2.0.1 apache-airflow-providers-odbc==2.0.1 apache-airflow-providers-oracle==2.0.1 apache-airflow-providers-postgres==2.3.0 apache-airflow-providers-slack==4.1.0 apache-airflow-providers-sqlite==2.0.1 apache-airflow-providers-ssh==2.2.0 ### Deployment Other 3rd-party Helm chart ### Deployment details We have an AirFlow Pod running the scheduler and the webserver containers plus some other sidecar containers. To use the KubernetesExecutor, we have NFS-PVCs for the dags and the logs. ### What happened The configuration with LocalExecutor works well. The configuration with KubernetesExecutor can successfully execute dags on the worker pods and logs are consequently present on the main pod via the PVC. However, the UI is almost not working during the task execution. In detail, it freezes trying to get the logs from the worker pods, and navigation through the whole UI gives lots of "Bad Gateway" white screens and uncompleted pages without formatting. Inside the worker pods the there is no running log-server (like the one running on port 8793 on the main-pod). If no DAG is running either by completion or after killing the worker, the UI reverts to the normal behaviour. ### What you expected to happen My understanding of the KubernetesExecutor is that a log-server should be running on the worker-pods on port 8793. Logs shoud be available during the execution of the tasks and the UI should remain responsive. ### How to reproduce Setup an AirFlow instance with the KubernetesExecutor an start a dag with at least a long running task to see the behaviour of the UI during the task execution. ### Anything else I don't notice anything special in the pod logs neither the scheduler nor in the webserver logs on the PVC ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
