zchovan commented on pull request #2017: URL: https://github.com/apache/hive/pull/2017#issuecomment-785731823
@pvargacl The Cleaner changes are planned for a separate commit. I agree that the scenarios you've mentioned have to be tested, but without the final cleaner changes they don't really make sense here. The way I see it, the Cleaner first checks if the partition still exists in the HMS, if it doesn't, then the partition has not been yet recreated and the whole location dir can be deleted, no compaction needed. If the partition exists that means that between the dropPartition and the compaction's start the partition was recreated and should be compacted, e.g the files created before the truncated/deleted base file was written can be compacted/deleted. This still leaves the last scenario where the Cleaner is already running and the partition is recreated, so yeah that should be checked and tested. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
