[
https://issues.apache.org/jira/browse/BEAM-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Beam JIRA Bot updated BEAM-5040:
--------------------------------
Labels: (was: stale-assigned)
> BigQueryIO retries infinitely in WriteTable and WriteRename
> -----------------------------------------------------------
>
> Key: BEAM-5040
> URL: https://issues.apache.org/jira/browse/BEAM-5040
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.5.0
> Reporter: Reuven Lax
> Priority: P2
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> BigQueryIO retries infinitely in WriteTable and WriteRename
> Several failure scenarios with the current code:
> # It's possible for a load job to return failure even though it actually
> succeeded (e.g. the reply might have timed out). In this case, BigQueryIO
> will retry the job which will fail again (because the job id has already been
> used), leading to indefinite retries. Correct behavior is to stop retrying as
> the load job has succeeded.
> # It's possible for a load job to be accepted by BigQuery, but then to fail
> on the BigQuery side. In this case a retry with the same job id will fail as
> that job id has already been used. BigQueryIO will sometimes detect this, but
> if the worker has restarted it will instead issue a load with the old job id
> and go into a retry loop. Correct behavior is to generate a new deterministic
> job id and retry using that new job id.
> # In many cases of worker restart, BigQueryIO ends up in infinite retry
> loops.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
