lasdf1234 opened a new issue, #14674:
URL: https://github.com/apache/gravitino/issues/14674

   ### Version
   
   main branch
   
   ### Describe what's wrong
   
   For Iceberg maintenance strategies that require partition statistics (for 
example compaction), `BaseExpressionStrategyHandler` can decide `shouldTrigger` 
using **table-level** statistics:
   
   1. It first tries to evaluate `trigger-expr` with an empty partition-stat 
list (table context only).
   2. Even when evaluating per partition, the expression context is seeded with 
table-level statistics.
   
   `builtin-iceberg-update-stats` refreshes **partition** statistics for 
partitioned tables and does not rewrite table-level keys. Stale table-level 
values such as `custom-data-file-mse` can therefore still make 
`submit-strategy-jobs` recommend or submit work.
   
   Expected: for partition-aware strategies on partitioned tables, trigger and 
score use partition statistics (plus table metadata and policy rules as today), 
not table-level statistic values. Non-partitioned tables keep using table-level 
statistics. update-stats can keep writing only partition statistics for 
partitioned tables.
   
   ### Error message and/or stacktrace
   
   No exception. Dry-run or submit can proceed from stale table-level metrics 
while partition statistics differ.
   
   ### How to reproduce
   
   1. Create a partitioned Iceberg table and attach a compaction policy whose 
trigger uses `custom-data-file-mse` (or equivalent).
   2. PUT a large table-level `custom-data-file-mse`.
   3. Run `builtin-iceberg-update-stats` so partition statistics are refreshed 
to real values.
   4. Run `submit-strategy-jobs --dry-run` for that table and policy.
   
   Actual: the recommendation can still follow the stale table-level value.  
   Expected: trigger and score follow partition statistics only.
   
   ### Additional context
   
   Partition-aware handlers declare `PARTITION_STATISTICS`. Table-scoped 
handlers that do not require partition statistics are out of scope for this 
change.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to