JingsongLi commented on code in PR #10964:
URL: https://github.com/apache/paimon/pull/10964#discussion_r4456264922


##########
paimon-hive/paimon-hive-catalog/src/main/java/org/apache/paimon/hive/HiveCatalog.java:
##########
@@ -782,8 +817,20 @@ private List<Map<String, String>> 
removePartitionsExistsInOtherBranches(
                 continue;
             }
 
-            branchTableForPartitionExistence(mainTable, branchName).newScan()
-                    .withPartitionsFilter(new 
ArrayList<>(inputsToRemove)).listPartitions().stream()
+            FileStoreTable branchTable = 
branchTableForPartitionExistence(mainTable, branchName);
+            InternalRowPartitionComputer partitionComputer =
+                    new InternalRowPartitionComputer(
+                            branchTable.coreOptions().partitionDefaultName(),
+                            
branchTable.rowType().project(branchTable.partitionKeys()),
+                            branchTable.partitionKeys().toArray(new String[0]),
+                            branchTable.coreOptions().legacyPartitionName());
+            DataTableScan scan = branchTable.newScan();

Review Comment:
   [P2] Check marker ownership against current physical metadata
   
   `newScan()` honors persisted read selectors, so it can miss a live literal 
`.done` partition on the current branch. I extended 
`testMarkerCleanupKeepsCurrentBranchRealDoneSuffixPartition` by first writing 
`keep` as snapshot 1, then writing `20230101` and the real `20230101.done`, and 
setting `scan.snapshot-id=1` with `catalog.alterTable` before the existing 
explicit DROP of `20230101`. After removing the selector again, the latest 
physical-data assertion still sees `keep` and `20230101.done`, but HMS contains 
only `keep`: this new cleanup deleted the live partition's registration because 
its ownership scan saw snapshot 1. The original explicit DROP did not touch 
that unrequested HMS entry. Please inspect the physical branch's latest 
snapshot partition metadata independently of query startup options, rather than 
using a configured batch scan, and retain this real-partition collision 
regression.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to