ArnavBalyan opened a new pull request, #8339:
URL: https://github.com/apache/paimon/pull/8339
### Purpose
- StartsWith on a btree global index returns every non null row today, so a
prefix query like col like 'abc%' reads the whole segment and the lower level
filters afterwards.
- The index keys are sorted, so a prefix is a contiguous range and can be
checked with the min and max values of the keys, similar to bitmap global index.
- Add support for btree level pushdown which checks the min/max keys and
prunes startwith values based on the btree values.
### Benchmark (Before vs After):
High cardinality keys were generated, and query selects single prefix using
the startswith. Performance was measured against a highly selective query
(defined by the % match illustrated in the workload column).
| Workload | Before | After | Speedup |
| --- | --- | --- | --- |
| 50K rows, 1% match | 2.73 ms | 0.085 ms | 32x |
| 1M rows, 1% match | 42.8 ms | 0.857 ms | 50x |
| 1M rows, 0.1% match | 42.2 ms | 0.061 ms | 697x |
### Tests
- UTs Added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
