[
https://issues.apache.org/jira/browse/BEAM-12567?focusedWorklogId=636627&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-636627
]
ASF GitHub Bot logged work on BEAM-12567:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Aug/21 20:49
Start Date: 10/Aug/21 20:49
Worklog Time Spent: 10m
Work Description: ryanthompson591 commented on pull request #15303:
URL: https://github.com/apache/beam/pull/15303#issuecomment-896307262
@KevinGG
Great I've squashed the commits. Let me know what I need to do to merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 636627)
Time Spent: 1h 40m (was: 1.5h)
> Beam should retry failing GCS uploads
> --------------------------------------
>
> Key: BEAM-12567
> URL: https://issues.apache.org/jira/browse/BEAM-12567
> Project: Beam
> Issue Type: New Feature
> Components: io-py-gcp
> Reporter: Valentyn Tymofieiev
> Assignee: Ryan Thompson
> Priority: P2
> Labels: starter
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Sometimes GCS upload initiated by Beam stager fail due to a retriable error
> (such as 503), however Beam code and the underlying apitools-based storage
> client do not retry.
> We should retry some reasonable amount of times with a backoff logic, either
> on all or (better) on retriable errors.
> Code pointers:
> https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py#L553-L554
> and similar:
> https://github.com/apache/beam/blob/c026c9fa00239a6f3650a6fa69f3aa864f692afe/sdks/python/apache_beam/io/gcp/gcsio.py#L635-L639
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
