kerwin-zk commented on PR #10964:
URL: https://github.com/apache/paimon/pull/10964#issuecomment-6605969862

   > Could we keep `.done` cleanup inside `HiveCatalog.dropPartitions` instead 
of adding new public methods to `Catalog` and `PartitionModification`? After 
deleting the requested data partitions, HiveCatalog could derive the 
corresponding marker specs from those original specs and remove only their HMS 
entries, without truncating data or creating another snapshot. Normal and 
chain-table expiration would then use the ordinary partition-drop path without 
a second call for markers. This would keep the Hive marker convention within 
HiveCatalog and avoid expanding the public API for this fix.
   > 
   > The cleanup needs to account for the shared HMS namespace: retain a marker 
if any branch still owns its underlying ordinary partition, or if any branch, 
including the current branch, contains a real partition whose literal spec 
equals the marker spec. These checks should use physical branch partitions 
without fallback reads and compare rendered partition specs, rather than 
converting synthetic values such as `20230101.done` to the partition type. 
Marker identity should come from deriving it from the original deletion 
request, not from treating every `.done` suffix as synthetic; an explicit DROP 
of a real STRING partition ending in `.done` must still delete its data with 
the usual branch protection for HMS metadata.
   
   @JingsongLi done


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to