danielhumanmod commented on code in PR #1508:
URL: https://github.com/apache/polaris/pull/1508#discussion_r2072320859
##########
service/common/src/main/java/org/apache/polaris/service/task/TableCleanupTaskHandler.java:
##########
@@ -112,7 +113,6 @@ public boolean handleTask(TaskEntity cleanupTask,
CallContext callContext) {
metaStoreManager,
polarisCallContext);
- // TODO: handle partition statistics files
Review Comment:
> It looks like these weren't left out due to an oversight but rather they
were intentionally excluded. I'm curious if there is any background on why that
is -- is there some specific pitfall related to cleaning up the partition stats?
Good catch — to the best of my knowledge, Polaris `drop table prune`
currently has a gap compared to Iceberg's implementation due to some reasons
(which I don't know either, curious too)
Iceberg will delete all file types under the metadata/ directory, including
manifests, manifest lists, metadata files, previous metadata, and .stats files
(both table and partition-level). Iceberg code pointer:
[CatalogUtil.java#L124](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/CatalogUtil.java#L124)
for reference.
This gap also discussed earlier in [this issue
comment](https://github.com/apache/polaris/issues/289#issuecomment-2347423467).
Happy to learn more if there’s additional context I missed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
