TheNeuralBit commented on a change in pull request #11532:
URL: https://github.com/apache/beam/pull/11532#discussion_r427421548
##########
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
##########
@@ -1066,7 +1079,12 @@ public SpannerWriteResult
expand(PCollection<MutationGroup> input) {
spec.getBatchSizeBytes(),
spec.getMaxNumMutations(),
spec.getMaxNumRows(),
- spec.getGroupingFactor(),
+ // Do not group on streaming unless explicitly
set.
+ spec.getGroupingFactor()
+ .orElse(
+ input.isBounded() == IsBounded.BOUNDED
Review comment:
Is there any chance that someone using SpannerIO in a streaming pipeline
is relying on the default grouping factor being 1000? I'm concerned this
backwards-incompatible change could break them. Would it be sufficient to just
give users the _option_ to disable batching by setting the grouping factor to 1?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
