ahmedabu98 opened a new pull request, #23012: URL: https://github.com/apache/beam/pull/23012
Streaming FILE_LOADS is currently not working with large loads. After the first load, it lags because `WaitForBQJobs` is only triggered once with `beam.Create([None])`. These changes move the wait into the finish_bundle of its respective step. Note: Streaming with small loads is done in one step and thus was not getting stuck. Also adding tests that more accurately determine if streaming with batch loads is working. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
