Gabriel39 opened a new pull request, #66263:
URL: https://github.com/apache/doris/pull/66263
### What problem does this PR solve?
Issue Number: N/A
Related PR: #65674
Problem Summary:
Parquet V2 repeated leaves currently rebuild a parent-row filter into a
level-entry filter, scan definition levels into null runs plus an ancestor-null
hash set, build the leaf selection, and scan the levels again to compact
surviving parent shapes. This fixed O(level entries) work is especially visible
when only 1% or 10% of parent rows survive.
This change adds a single-pass nested selection builder. It maps parent
filters to repeated entries, classifies ancestor and leaf nulls, builds the
decoder selection, appends selected nulls, and compacts repetition/definition
levels in place. Ancestor-null placeholders remain in the surviving parent
shape but do not consume leaf selection entries. Unfiltered and fully filtered
paths keep their existing behavior.
The `ParquetKernel/nested_selection` microbenchmark uses 65,536 parent rows,
eight repeated values per non-null parent, 10% ancestor/leaf nulls, and
clustered or alternating parent selections. Release build, one pinned CPU,
three warmups, 10 repetitions with a one-second minimum, A-B-B-A order; the
table averages the two run medians for each implementation:
| Parent survivors | Pattern | Before CPU | After CPU | Speedup |
|---:|---|---:|---:|---:|
| 1% | clustered | 3.501 ms | 1.839 ms | 1.90x |
| 1% | alternating | 3.132 ms | 1.491 ms | 2.10x |
| 10% | clustered | 3.107 ms | 1.509 ms | 2.06x |
| 10% | alternating | 3.340 ms | 1.474 ms | 2.27x |
| 50% | clustered | 3.664 ms | 2.047 ms | 1.79x |
| 50% | alternating | 3.628 ms | 1.919 ms | 1.89x |
### Release note
Improve Parquet V2 sparse reads of nested and repeated columns by reducing
selection reconstruction work.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [x] Manual test (Release microbenchmark detailed above)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [x] No. The change preserves nested row shape and leaf cursor
semantics.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
