lasdf1234 opened a new issue, #14674: URL: https://github.com/apache/gravitino/issues/14674
### Version main branch ### Describe what's wrong For Iceberg maintenance strategies that require partition statistics (for example compaction), `BaseExpressionStrategyHandler` can decide `shouldTrigger` using **table-level** statistics: 1. It first tries to evaluate `trigger-expr` with an empty partition-stat list (table context only). 2. Even when evaluating per partition, the expression context is seeded with table-level statistics. `builtin-iceberg-update-stats` refreshes **partition** statistics for partitioned tables and does not rewrite table-level keys. Stale table-level values such as `custom-data-file-mse` can therefore still make `submit-strategy-jobs` recommend or submit work. Expected: for partition-aware strategies on partitioned tables, trigger and score use partition statistics (plus table metadata and policy rules as today), not table-level statistic values. Non-partitioned tables keep using table-level statistics. update-stats can keep writing only partition statistics for partitioned tables. ### Error message and/or stacktrace No exception. Dry-run or submit can proceed from stale table-level metrics while partition statistics differ. ### How to reproduce 1. Create a partitioned Iceberg table and attach a compaction policy whose trigger uses `custom-data-file-mse` (or equivalent). 2. PUT a large table-level `custom-data-file-mse`. 3. Run `builtin-iceberg-update-stats` so partition statistics are refreshed to real values. 4. Run `submit-strategy-jobs --dry-run` for that table and policy. Actual: the recommendation can still follow the stale table-level value. Expected: trigger and score follow partition statistics only. ### Additional context Partition-aware handlers declare `PARTITION_STATISTICS`. Table-scoped handlers that do not require partition statistics are out of scope for this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
