Abacn commented on PR #30259: URL: https://github.com/apache/beam/pull/30259#issuecomment-1935052197
The idea sounds good to me. There are two options controlling the batch, batch_size and buffering_duration. For example, if batch_size set to 100 and buffering_duration=200ms. - if batch size reaches 1000 within 100ms - if batch size does not reach 1000 within 200ms If I understood correctly, in both cases, the pipeline will commit the buffer? Good to add some documentation about the spec. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
