devgonvarun opened a new issue, #53926:
URL: https://github.com/apache/airflow/issues/53926
### Apache Airflow Provider(s)
opensearch
### Versions of Apache Airflow Providers
apache-airflow-providers-opensearch | 1.7.1
### Apache Airflow version
3.0.3
### Operating System
Debian GNU/Linux 12 (bookworm)
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### What happened
When using Airflow with the OpenSearch provider under the default
log_id_template, the template value is:
{dag_id}-{task_id}-{run_id}-{map_index}-{try_number}. However, the actual
log_id generated by OpensearchTaskHandler is built using
{dag_id}-{task_id}-{logical_date}-{try_number}.
Default log_id_template value confirmed by running "airflow config get-value
opensearch log_id_template" command in airflow cli and it is indeed
{dag_id}-{task_id}-{run_id}-{map_index}-{try_number}.
But when Airflow needs to pull task logs from Opensearch it uses a log_id
with {dag_id}-{task_id}-{logical_date}-{try_number} as can be seen here:
<img width="1663" height="41" alt="Image"
src="https://github.com/user-attachments/assets/a443cb0f-7254-474e-9ec5-e47f4f27930b";
/>
### What you think should happen instead
The log_id should be of the format
{dag_id}-{task_id}-{run_id}-{map_index}-{try_number} instead of
{dag_id}-{task_id}-{logical_date}-{try_number}.
### How to reproduce
With Airflow 3.0.3 and Opensearch provider package installed, in the
values.yaml make the following appropriate changes to enable opensearch task
handler:
<img width="594" height="312" alt="Image"
src="https://github.com/user-attachments/assets/2e403e66-588a-4e80-a223-624f9846ddee";
/>
Additionally set AIRFLOW__LOGGING__LOGGING_LEVEL environment variable to
value: DEBUG.
Then simply navigate to any Task logs in Airflow UI. This will trigger the
Opensearch Task Handler to fetch these task logs from OpenSearch as can be seen
in the airflow-api-server pod logs. There check the log_id in the match_phrase
of the opensearch command.
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
