[
https://issues.apache.org/jira/browse/BEAM-13689?focusedWorklogId=711549&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-711549
]
ASF GitHub Bot logged work on BEAM-13689:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Jan/22 18:32
Start Date: 19/Jan/22 18:32
Worklog Time Spent: 10m
Work Description: steveniemitz commented on a change in pull request
#16561:
URL: https://github.com/apache/beam/pull/16561#discussion_r788032113
##########
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteResult.java
##########
@@ -77,13 +96,41 @@ private WriteResult(
PCollection<TableRow> failedInserts,
TupleTag<BigQueryInsertError> failedInsertsWithErrTag,
PCollection<BigQueryInsertError> failedInsertsWithErr,
- PCollection<TableRow> successfulInserts) {
+ PCollection<TableRow> successfulInserts,
+ TupleTag<TableDestination> successfulInsertsTag,
+ PCollection<TableDestination> successfulBatchInserts) {
this.pipeline = pipeline;
this.failedInsertsTag = failedInsertsTag;
this.failedInserts = failedInserts;
this.failedInsertsWithErrTag = failedInsertsWithErrTag;
this.failedInsertsWithErr = failedInsertsWithErr;
this.successfulInserts = successfulInserts;
+ this.successfulBatchInsertsTag = successfulInsertsTag;
+ this.successfulBatchInserts = successfulBatchInserts;
+ }
+
+ /**
+ * Returns a {@link PCollection} containing the {@link TableDestinations}s
that were successfully
+ * inserted.
+ *
+ * <p>Successful Inserts are only produced when using batch inserts.
+ */
+ public PCollection<TableDestination> getSuccessfulBatchInserts() {
+ checkArgument(
+ successfulBatchInsertsTag != null,
+ "Cannot use getSuccessfulInserts because this WriteResult was not
configured "
+ + "to produce them. Note: streaming inserts do not produce
successful "
+ + "insert results.");
+
+ return successfulBatchInserts;
Review comment:
I'm cool with that, I wonder if we want to remove `batch` from it
though, since you could in theory use this same method for storage API writes
if it were implemented, and those aren't technically "batch". I'm open to
anything for the name though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 711549)
Time Spent: 1h 10m (was: 1h)
> BigQueryIO batch writes should output "void" elements when successful
> ---------------------------------------------------------------------
>
> Key: BEAM-13689
> URL: https://issues.apache.org/jira/browse/BEAM-13689
> Project: Beam
> Issue Type: New Feature
> Components: io-java-gcp
> Reporter: Steve Niemitz
> Assignee: Steve Niemitz
> Priority: P2
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Similar to how other IOs (Bigtable, FileIO, etc) output token elements when
> their writes complete, BigQueryIO should do the same. It already does for
> streaming writes, we should do something similar for batch ones as well.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
