robertwb commented on code in PR #31106:
URL: https://github.com/apache/beam/pull/31106#discussion_r1633552968
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java:
##########
@@ -52,16 +52,18 @@
/** This {@link PTransform} manages loads into BigQuery using the Storage API.
*/
public class StorageApiLoads<DestinationT, ElementT>
extends PTransform<PCollection<KV<DestinationT, ElementT>>, WriteResult> {
- final TupleTag<KV<DestinationT, StorageApiWritePayload>>
successfulConvertedRowsTag =
- new TupleTag<>("successfulRows");
+ final TupleTag<KV<DestinationT, KV<ElementT, StorageApiWritePayload>>>
Review Comment:
As a side comment, this is another motivation to use schema coders more
ubiquitously--adding another field is update compatible.
On another note, anything that involves shuffling more data in the main data
path should be looked at carefully from a perf standpoint. We've gone to a lot
of effort (e.g. with dynamic destinations) to ensure shuffling metadata doesn't
become a perf impediment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
