lanxinxs opened a new issue, #44683:
URL: https://github.com/apache/airflow/issues/44683
### Apache Airflow version
Other Airflow 2 version (please specify below)
### If "Other Airflow 2 version" selected, which one?
2.6.3
### What happened?
when task is deferred, the webserver UI is unable to read trigger log:
`*** Could not read served logs: Client error '404 NOT FOUND' for url
'http://<triggerer
hostname>:8794/log/dag_id=xxx/run_id=xxx/task_id=xxx/attempt=18.log.trigger.3509.log'
`
after task is completed, the above error message will disappear, it says the
trigger log is found on s3 (we enable remote logging):
`*** Found logs in s3:
*** * s3://<bucket>/path_to_log/attempt=18.log
*** * s3://<bucket>/path_to_log/attempt=18.log.trigger.3509.log`
and the content of the trigger log is displayed with the task log in the UI
`[2024-12-05, 07:34:00 UTC] {triggerer_job_runner.py:616} INFO - Trigger
xxx/-1/18 (ID 160) fired: TriggerEvent<{'state': 'SUCCESS', 'message': None,
'extra_data': {...}}>
`
While task is deferred I exec into the triggerer pod, I can't find the
expected xxx/attempt=18.log.trigger.3509.log in log folder; while after the
task completes, I can find this log in the folder
### What you think should happen instead?
while task is deferred, I expect to view the corresponding trigger log in
real-time, like the video [here](https://github.com/apache/airflow/pull/27758)
### How to reproduce
run a DAG with deferrable task, and view the task log when it is deferred
### Operating System
Ubuntu
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==8.1.0
apache-airflow-providers-celery==3.2.0
apache-airflow-providers-cncf-kubernetes==7.0.0
apache-airflow-providers-common-sql==1.5.1
apache-airflow-providers-docker==3.7.0
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
triggerer pod has open 8794 port, and has a service listening on that port
`triggerer ClusterIP None <none> 8794/TCP
22d
`
and webserver secret key has configured properly.
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
