justinpakzad commented on PR #60398:
URL: https://github.com/apache/airflow/pull/60398#issuecomment-3736000773
Hey, I think I know why the test was flaky. Whats happening is in the
S3Hook, the download msg isn't appended it's reassigned, so if it fits both
criteria, meaning the size changed AND the last_modified date has changed, then
it only gets the latter message and not both which is why the test wasn't
working correctly. Might be worth fixing that directly, maybe we have a more
general download message or we log both reasons.
Code I'm referring to:
```
if s3_object.size != local_stats.st_size:
should_download = True
download_msg = f"S3 object size ({s3_object.size}) and local file size
({local_stats.st_size}) differ."
s3_last_modified = s3_object.last_modified
if local_stats.st_mtime < s3_last_modified.timestamp():
should_download = True
download_msg = f"S3 object last modified
({s3_last_modified.microsecond}) and local file last modified
({local_stats.st_mtime}) differ."```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
