gpintus commented on issue #20676: URL: https://github.com/apache/beam/issues/20676#issuecomment-1597266764
Hello. I got to understand that a temp file per bundle is created, and that the number and size of bundles is not a knob that can be regulated. This is a problem with high volume streaming, as the number of generated files per window can easily get out of hand. It is a problem i am facing in my current pipeline too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
