dpcollins-google commented on PR #22762: URL: https://github.com/apache/beam/pull/22762#issuecomment-1219697942
There is not a linked issue. Whenever dataflow scales up or down, it closes the reader while there are still outstanding checkpointmarks that will be committed asynchronously and fail, logging a bunch of warnings. CheckpointMark has no lifecycle methods- hence the only way to ensure the committer lives long enough for all the CheckpointMarks to be used (while also not requiring that they are used) is with finalize. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
