jrmccluskey commented on code in PR #32082: URL: https://github.com/apache/beam/pull/32082#discussion_r1725292257
########## sdks/python/apache_beam/transforms/util.py: ########## @@ -802,6 +802,16 @@ class BatchElements(PTransform): corresponding to its contents. Each batch is emitted with a timestamp at the end of their window. + When the max_batch_duration_secs arg is provided, a stateful implementation + of BatchElements is used to batch elements across bundles. This is most + impactful in streaming applications where many bundles only contain one + element. Larger max_batch_duration_secs values can reduce the throughput of + the transform, while smaller values will improve the throughput but make it + more likely that batches are smaller than the target batch size. + + For more information on tuning parameters to this transform, see + https://beam.apache.org/documentation/patterns/batch-elements Review Comment: That's a fair callout! And also aligns with your experiences well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
