pvargacl commented on pull request #2017: URL: https://github.com/apache/hive/pull/2017#issuecomment-785744497
> @pvargacl > The Cleaner changes are planned for a separate commit. I agree that the scenarios you've mentioned have to be tested, but without the final cleaner changes they don't really make sense here. > The way I see it, the Cleaner first checks if the partition still exists in the HMS, if it doesn't, then the partition has not been yet recreated and the whole location dir can be deleted, no compaction needed. > If the partition exists that means that between the dropPartition and the compaction's start the partition was recreated and should be compacted, e.g the files created before the truncated/deleted base file was written can be compacted/deleted. > This still leaves the last scenario where the Cleaner is already running and the partition is recreated, so yeah that should be checked and tested. I don't think this can go in with some basic Cleaner change, even if it does not delete the partition directory, you have to handle if the partition record is missing, otherwise the Cleaner will just fail. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
