slilichenko commented on code in PR #25723:
URL: https://github.com/apache/beam/pull/25723#discussion_r1128761070
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWriteUnshardedRecords.java:
##########
@@ -558,7 +574,23 @@ long flush(
appendFailures.inc();
return RetryType.RETRY_ALL_OPERATIONS;
},
- c -> recordsAppended.inc(c.protoRows.getSerializedRowsCount()),
+ c -> {
+ recordsAppended.inc(c.protoRows.getSerializedRowsCount());
+ if (successfulRowsReceiver != null) {
+ for (ByteString rowBytes :
c.protoRows.getSerializedRowsList()) {
+ try {
+ TableRow row =
Review Comment:
As long as docs will be clear about the windowing differences and the fact
that the TableRow is not the same as the original (possibly different data
types and missing fields that were ignored) this should work.
Would be good to come up with the metadata approach that covers most use
cases. Do you want to do it as part of this PR or as a separate case (with a
separate BigQueryIO.Write method to indicate that this metadata collection
should be produced?).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
