[ https://issues.apache.org/jira/browse/BEAM-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Beam JIRA Bot reassigned BEAM-5040: ----------------------------------- Assignee: (was: Reuven Lax) > BigQueryIO retries infinitely in WriteTable and WriteRename > ----------------------------------------------------------- > > Key: BEAM-5040 > URL: https://issues.apache.org/jira/browse/BEAM-5040 > Project: Beam > Issue Type: Bug > Components: io-java-gcp > Affects Versions: 2.5.0 > Reporter: Reuven Lax > Priority: P2 > Labels: stale-assigned > Time Spent: 2.5h > Remaining Estimate: 0h > > BigQueryIO retries infinitely in WriteTable and WriteRename > Several failure scenarios with the current code: > # It's possible for a load job to return failure even though it actually > succeeded (e.g. the reply might have timed out). In this case, BigQueryIO > will retry the job which will fail again (because the job id has already been > used), leading to indefinite retries. Correct behavior is to stop retrying as > the load job has succeeded. > # It's possible for a load job to be accepted by BigQuery, but then to fail > on the BigQuery side. In this case a retry with the same job id will fail as > that job id has already been used. BigQueryIO will sometimes detect this, but > if the worker has restarted it will instead issue a load with the old job id > and go into a retry loop. Correct behavior is to generate a new deterministic > job id and retry using that new job id. > # In many cases of worker restart, BigQueryIO ends up in infinite retry > loops. -- This message was sent by Atlassian Jira (v8.3.4#803005)