chamikaramj commented on code in PR #24041:
URL: https://github.com/apache/beam/pull/24041#discussion_r1018290203
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQueryHelper.java:
##########
@@ -119,16 +119,21 @@ public static TableReference executeQuery(
}
// Step 2: Create a temporary dataset in the query location only if the
user has not specified
- // a temp dataset.
- String queryJobId =
+ // a temp dataset. The temp table name may be deterministic but the
query job ID has to be
+ // non-deterministic to protect against retries. If BigQuery sees a
repeated query job ID, it
+ // will be skipped.
+ String tempTableID =
BigQueryResourceNaming.createJobIdPrefix(options.getJobName(),
stepUuid, JobType.QUERY);
+ String queryJobId =
+ BigQueryResourceNaming.createJobIdPrefix(
+ options.getJobName(), stepUuid, JobType.QUERY,
BigQueryHelpers.randomUUIDString());
Optional<String> queryTempDatasetOpt =
Optional.ofNullable(queryTempDatasetId);
TableReference queryResultTable =
createTempTableReference(
options.getBigQueryProject() == null
? options.getProject()
: options.getBigQueryProject(),
- queryJobId,
+ tempTableID,
Review Comment:
Should we be passing "queryJobId" here instead ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
