chamikaramj commented on a change in pull request #13001:
URL: https://github.com/apache/beam/pull/13001#discussion_r500416620
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_tools.py
##########
@@ -421,8 +418,36 @@ def _insert_load_job(
),
jobReference=reference,
))
- response = self.client.jobs.Insert(request)
- return response.jobReference
+ return self._start_job(request).jobReference
+
+ def _start_job(
+ self,
+ request # type: bigquery.BigqueryJobsInsertRequest
+ ):
+ """Inserts a BigQuery job.
+
+ If the job exists already, it returns it.
+ """
+ try:
+ response = self.client.jobs.Insert(request)
+ _LOGGER.info(
+ "Stated BigQuery job: %s\n "
+ "bq show -j --format=prettyjson --project_id=%s %s",
+ response.jobReference,
+ response.jobReference.projectId,
+ response.jobReference.jobId)
+ return response
+ except HttpError as exn:
+ if exn.status_code == 409:
Review comment:
Probably this conflict handling deserves a mention in the PR description
and a JIRA.
Also, when do we run into this ? Work item retries ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
