[
https://issues.apache.org/jira/browse/BEAM-6202?focusedWorklogId=285119&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-285119
]
ASF GitHub Bot logged work on BEAM-6202:
----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Jul/19 17:21
Start Date: 30/Jul/19 17:21
Worklog Time Spent: 10m
Work Description: udim commented on pull request #9182: [BEAM-6202]
Change retry policy for dataflow's get_job
URL: https://github.com/apache/beam/pull/9182#discussion_r308843154
##########
File path: sdks/python/apache_beam/utils/retry.py
##########
@@ -104,6 +104,15 @@ def retry_on_server_errors_filter(exception):
return not isinstance(exception, PermanentException)
+# TODO(BEAM-6202): Dataflow returns 404 for job ids that actuall exist.
+# Retry on those errors.
+def retry_on_server_errors_and_notfound_filter(exception):
Review comment:
Not for this PR: these retry filters should probably follow a builder
pattern.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 285119)
Time Spent: 1h 10m (was: 1h)
> Gracefully handle exceptions when waiting for Dataflow job completion.
> ----------------------------------------------------------------------
>
> Key: BEAM-6202
> URL: https://issues.apache.org/jira/browse/BEAM-6202
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core, test-failures
> Reporter: Robert Bradshaw
> Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> If there is an error when trying to contact the dataflow service in Python's
> Dataflow.poll_for_job_completion, we may exit the thread prematurely.
> A typical manifestation is: Dataflow Runner fails with:
> {noformat}
> AssertionError: Job did not reach to a terminal state after waiting
> indefinitely.
> {noformat}
> however job execution continues, and succeeds.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
