nsivabalan commented on code in PR #6548:
URL: https://github.com/apache/hudi/pull/6548#discussion_r964230632
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java:
##########
@@ -206,15 +206,7 @@ private List<String>
getPartitionPathsForIncrementalCleaning(HoodieCleanMetadata
*/
private List<String> getPartitionPathsForFullCleaning() {
// Go to brute force mode of scanning all partitions
- try {
- // Because the partition of BaseTableMetadata has been deleted,
- // all partition information can only be obtained from
FileSystemBackedTableMetadata.
Review Comment:
guess we can't go w/ this change. essentially what we are doing in cleaning
is
1. fetch list of partitions to clean
2. Fetch list of files to clean
3. for those partitions where there is no valid file groups, we mark the
partition to be deleted and physically delete the directory (lazy execution of
DELETE_PARTITION)
So, in step1, we have to go w/ FS based listing. if we revert to metadata
table based listing, we may not get the deleted partition only. and so the
physical directory will never get deleted.
Need to think about an optimized way to go about this. for now, will punt on
the patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
