shrutig commented on issue #25514: [SPARK-28784][SS]StreamExecution and StreamingQueryManager should utilize CheckpointFileManager to interact with checkpoint directories URL: https://github.com/apache/spark/pull/25514#issuecomment-525081211 @zsxwing CheckpointFileManager is already being used to handle checkpoint files in multiple places, such as HDFSMetadataLog, HDFSBackedStateStoreProvider and others as part of PR #21048 . This PR aims to make that change uniformly in other classes. As far as the issue in https://issues.apache.org/jira/browse/SPARK-28025 is concerned, in my opinion that should not hinder this PR. That issue has been approved and merged as part of a PR - https://github.com/apache/spark/pull/25565 . Also, it was caused due to a bug in a specific implementation of CheckpointFileManager (FileContextBasedCheckpointFileManager).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
