chamikaramj commented on code in PR #31106:
URL: https://github.com/apache/beam/pull/31106#discussion_r1594280411
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java:
##########
@@ -52,16 +52,18 @@
/** This {@link PTransform} manages loads into BigQuery using the Storage API.
*/
public class StorageApiLoads<DestinationT, ElementT>
extends PTransform<PCollection<KV<DestinationT, ElementT>>, WriteResult> {
- final TupleTag<KV<DestinationT, StorageApiWritePayload>>
successfulConvertedRowsTag =
- new TupleTag<>("successfulRows");
+ final TupleTag<KV<DestinationT, KV<ElementT, StorageApiWritePayload>>>
Review Comment:
I think you can manually test by running a streaming job using the old
version and running a replacement job that includes your changes with the
`--update` option.
https://cloud.google.com/dataflow/docs/guides/updating-a-pipeline#Launching
Dataflow streaming team might also have internal tests for update
compatibility (but I haven't looked into this).
With `updateCompatibilityVersion` option you could fork based on the Beam
version to preserve update compatibility. See following for an example.
https://github.com/apache/beam/blob/8328d073ab8492927927ad0a49d3d2f2169023bf/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Reshuffle.java#L77
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
