nsivabalan commented on issue #6373: URL: https://github.com/apache/hudi/issues/6373#issuecomment-1218565541
may I know what cleaning policy you are using? I see that for KEEP_LATEST_FILE_VERSIONS, we call getPartitionPathsForFullCleaning() within which we use file system based listing and not metadata table based listing. and if you are using KEEP_LATEST_COMMITS, within incremental clean mode enabled, if there is no prior clean ever, we trigger getPartitionPathsForFullCleaning() (within which we use file system based listing and not metadata table based listing). If not for these, we should be hitting only metadata based listing. Can you confirm which one among the above is your case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
