ecodina opened a new issue, #28647: URL: https://github.com/apache/airflow/issues/28647
### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened On Airflow 2.4.3 When using a deferrable operator the log disappears from the webpage while the operator is deferred and appears when it is running/finished. This doesn't happen consistently over all DAG runs. The error has been reproduced in the new log panel on the grid page & the "standalone" log page. As an example, I've tried running a simple DAG (just executes `sleep 10`). The logs are for the 3rd retry and I am using the "Auto-refresh" button on the grid page. There are no visible errors on the web console when this happens, but the web server logs look like Airflow doesn't know it is the 3rd try while the operator is deferred and gets the 2nd try. ### What you think should happen instead _No response_ ### How to reproduce Use a deferrable operator. We use this custom one: https://gist.github.com/ecodina/157b5dc44b79b13fe296b1275b4f0967 Trigger it from the webpage and see how the log appears intermittently. ### Operating System CentOS Linux 8 ### Versions of Apache Airflow Providers apache-airflow-providers-cncf-kubernetes | 4.4.0 | Kubernetes apache-airflow-providers-common-sql | 1.2.0 | Common SQL Provider apache-airflow-providers-ftp | 3.1.0 | File Transfer Protocol (FTP) apache-airflow-providers-http | 4.0.0 | Hypertext Transfer Protocol (HTTP) apache-airflow-providers-imap | 3.0.0 | Internet Message Access Protocol (IMAP) apache-airflow-providers-postgres | 5.2.2 | PostgreSQL apache-airflow-providers-sqlite | 3.2.1 | SQLite apache-airflow-providers-ssh | 3.2.0 | Secure Shell (SSH) ### Deployment Other ### Deployment details Installed using PIP in a conda environment (as if it was a virtualenv). Using postgresql . ### Anything else Relevant lines of the webserver's log: ``` --> the log doesn't appear Dec 30 08:53:52 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:52 +0000] "GET /object/grid_data?dag_id=dag.testing&num_runs=25 HTTP/1.1" 200 1800 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0" Dec 30 08:53:52 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:52 +0000] "GET /api/v1/dags/dag.testing/dagRuns/scheduled__2022-12-30T07:00:00+00:00/taskInstances/prova/logs/2?full_content=false HTTP/1.1" 200 14349 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0" Dec 30 08:53:55 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:55 +0000] "GET /object/grid_data?dag_id=dag.testing&num_runs=25 HTTP/1.1" 200 1800 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0" Dec 30 08:53:55 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:55 +0000] "GET /api/v1/dags/dag.testing/dagRuns/scheduled__2022-12-30T07:00:00+00:00/taskInstances/prova/logs/2?full_content=false HTTP/1.1" 200 14349 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0" Dec 30 08:53:58 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:58 +0000] "GET /object/grid_data?dag_id=dag.testing&num_runs=25 HTTP/1.1" 200 1801 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0" Dec 30 08:53:58 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:58 +0000] "GET /api/v1/dags/dag.testing/dagRuns/scheduled__2022-12-30T07:00:00+00:00/taskInstances/prova/logs/2?full_content=false HTTP/1.1" 200 14349 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0" Dec 30 08:53:58 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:58 +0000] "GET /api/v1/dags/dag.testing/dagRuns/scheduled__2022-12-30T07:00:00+00:00/taskInstances/prova/logs/3?full_content=false HTTP/1.1" 200 7804 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0" -> the log starts appearing Dec 30 08:54:01 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:54:01 +0000] "GET /object/grid_data?dag_id=dag.testing&num_runs=25 HTTP/1.1" 200 1802 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0" Dec 30 08:54:02 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:54:02 +0000] "GET /api/v1/dags/dag.testing/dagRuns/scheduled__2022-12-30T07:00:00+00:00/taskInstances/prova/logs/3?full_content=false HTTP/1.1" 200 8198 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0" ``` ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
