Haapalaj opened a new issue, #24244:
URL: https://github.com/apache/airflow/issues/24244
### Apache Airflow version
2.3.2 (latest released)
### What happened
Having the external log's configured to be stored in AWS S3. After updating
from 2.2.x version to the 2.3.x, the backward compatibility of fetching task
log entries made with the earlier version (2.2.5), fails because the default
log template has changed and are not found with that.
The log template with airflow version before 2.3.x was "'{{ ti.dag_id }}/{{
ti.task_id }}/{{ ts }}/{{ try_number }}.log'"
With version 2.3.x it is "{{ ti.dag_id }}/run_id={{ ti.run_id }}/task_id={{
ti.task_id }}/{% if ti.map_index >= 0 %}map_index={{ ti.map_index }}/{% endif
%}attempt={{ try_number }}.log"
There was discussion as a workaround
(https://github.com/apache/airflow/discussions/22992) to add the old template
to airflow template table:
`INSERT INTO log_template (id, filename, elasticsearch_id, created_at)
VALUES (0, '{{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ try_number }}.log',
'{dag_id}_{task_id}_{run_id}_{try_number}', NOW());`
That did not change the behaviour. Old task log entries still searched by
the new template.
There is a reference log_template_id from the airflow table dag_run to
templates. That reference is null for the old entries.
Tested to update those records to use the old template:
`UPDATE dag_run set log_template_id = 0 WHERE log_template_id IS NULL;`
But that did not change the behaviour either.
### What you think should happen instead
Task log entries made with the airflow version 2.2.x (Or pre 2.3.x) should
be still opened with the log template they were made with.
### How to reproduce
Try to open task log entries made with airflow version 2.2.x with the
airflow 2.3.x.
### Operating System
Linux, Centos
### Versions of Apache Airflow Providers
apache-airflow-providers-snowflake==2.6.0
apache-airflow-providers-amazon==3.3.0
apache-airflow-providers-google==6.1.0
apache-airflow-providers-microsoft-azure==3.1.1
apache-airflow-providers-odbc==2.0.1
apache-airflow-providers-postgres==2.2.0
### Deployment
Other Docker-based deployment
### Deployment details
The official Airflow docker image hosted in the AWS ECS Cluster.
apache/airflow:2.3.2-python3.7
### Anything else
This happen's every time older task log's is fetched.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
