potiuk commented on pull request #21981:
URL: https://github.com/apache/airflow/pull/21981#issuecomment-1060105013
> generally the marginal benefit will be for a single retry. s3 oneshot
uploads fail randomly very occasionally but failures do happen. Consecutive
failures generally indicate a non-ephemeral condition. A single retry would
resolve the 90-99% use case with minimal complexity. Multiple retries wouldn't
provide much additional benefit.
>
> what do you think of parametrizing with a conf setting similar to
`encrypt=conf.getboolean('logging',` 'ENCRYPT_S3_LOGS')` ?
The less options the better. I think It's fine to be hard-coded as a value.
The explanation above is good enough but it should find its way to the comment
in the code explaining why we try only twice. And even it it is 2, I agree with
@uranusjr this is not the best way - it will be less duplication and even less
complex if it is in a while or for loop rather than try/except with twice the
same code. It's far too easy to miss the second one if we decide to add a new
parameter to the call (label for example).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
