wirybeaver commented on PR #10815: URL: https://github.com/apache/pinot/pull/10815#issuecomment-1612220804
> Do we need to clean up tmp files periodically, or just when the controller starts? Will controller create extra tmp files during the regular life cycle without shut down? Hi Jackie, there are two kind of segment uploader during SegmentUpload phase for split commit. 1. Server2ControllerSegmentUploader, controller upload the tmp file <seg_name>.tmp.<uuid> to deep store. 2. PinotFSSegmentUploader, server upload the tmp file <seg_name><uuid> to deep store. In the CommitEnd phase, the deep storage can have outage due to flaky network when controller attempt to rename the tmp file. If peer downloading is enabled, controller will continue to commit the segment in ZK. The tmp file is left there in the regular lifecycle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
