Subham-KRLX opened a new pull request, #53784:
URL: https://github.com/apache/airflow/pull/53784
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Fix fetching logs for previous task retries with S3 remote logging (#53400)
#### Summary
This PR fixes the bug where Apache Airflow 3.0.3 fails to fetch logs for
previous task retries when using remote logging with S3 (amazon provider
v9.10.0).
Prior to this fix, only the latest retry’s logs could be fetched from the
Airflow web UI, causing errors for earlier retries.
#### Root Cause
The root cause was that the S3 log path generation did not correctly include
the `try_number` (retry number) in the log file key. This caused the log
handler to always fetch the latest retry's logs regardless of the requested
retry number.
#### What was changed
- Modified the `_render_filename` method in `S3TaskHandler`
(`airflow/providers/amazon/aws/log/s3_task_handler.py`) to incorporate the
retry number (`try_number`) into the S3 log file path.
- Ensured that upload and read operations use this path, uniquely
identifying each retry’s logs.
#### How to verify
- Deploy Airflow 3.0.3 with this patch and amazon provider 9.10.0.
- Configure remote S3 logging.
- Trigger a DAG with a task that retries multiple times.
- From the Airflow UI, access the logs of each retry attempt.
- Confirm that logs for **all** retries are accessible without error.
#### Related Issue
Fixes: #53400
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
