roryqi opened a new pull request, #11664:
URL: https://github.com/apache/gravitino/pull/11664
### What changes were proposed in this pull request?
Two related optimizations to the `maintenance/optimizer` recommender's
trigger-expr / score-expr evaluation:
1. **Extend evaluation context with table metadata** — the trigger-expr
context now exposes `column_count`, `partition_count`, `sort_order_count`, and
table properties (numeric values parsed to `long`, others kept as `string`), in
addition to partition and table statistics. Both partitioned and
non-partitioned tables now evaluate against partition statistics (when
present), table statistics, and table metadata. The trigger-expr string
representation is unchanged.
2. **Speed up partitioned-table evaluation** (port of Pinterest
gravitino-pinterest#249):
- Short-circuit: evaluate the expression with table-level context only;
if it resolves without referencing partition variables, skip the per-partition
loop (relies on the QL engine's left-to-right `&&` / `||` short-circuiting).
- Precompute the table-level context once per `initialize()` instead of
rebuilding it for every partition.
- Cache compiled hyphen-to-underscore regex patterns in
`QLExpressionEvaluator` to avoid `Pattern.compile` on every evaluation.
- Adds `ExpressionEvaluator#tryToEvaluateBool` returning
`Optional<Boolean>`.
### Why are the changes needed?
Trigger expressions previously could only reference partition/table
statistics, limiting the rules users can write. They also re-evaluated every
partition even when a table-level expression already decided the outcome, which
is costly for large partitioned tables.
### Does this PR introduce _any_ user-facing change?
No API changes. Trigger-expr authors gain new referenceable variables
(`column_count`, `partition_count`, `sort_order_count`, and table properties).
### How was this patch tested?
New/extended unit tests: `TestTableMetadataTriggerExpressionUtils`,
`TestQLExpressionEvaluator`, and `TestCompactionStrategyHandler`. `./gradlew
:maintenance:optimizer:test` passes locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
