Gabriel39 opened a new pull request, #66036: URL: https://github.com/apache/doris/pull/66036
### What problem does this PR solve? File Scanner V2 could only use row-level Parquet dictionary filtering for equality predicates. Range predicates on dictionary-encoded columns fell back to materializing the predicate value column, and predicate-only scans such as TPC-DS Q9 paid repeated typed dictionary decode/materialization costs. ### What is changed? - Enable exact typed dictionary evaluation for `=`, `!=`, `<`, `<=`, `>`, and `>=`, including symmetric literal-on-left comparisons. - Accept both `RLE_DICTIONARY` and legacy `PLAIN_DICTIONARY` data-page encodings for supported primitive types. - Decode selected dictionary IDs at the native page-reader layer and apply the per-entry predicate bitmap without materializing a complete typed predicate column. - Skip typed dictionary conversion entirely for predicate-only columns. - Cache typed dictionaries for projected columns, write surviving INT values directly in the filter loop, and use compact survivor-ID gathers for other fixed-width and variable-width types. - Add direct-path profile counters and extend the Parquet reader microbenchmark matrix. ### Verification - ASAN BE unit tests: 179/179 passed across `ExprZonemapFilterTest`, `ParquetScanTest`, `ParquetV2NativeDecoderTest`, and `ParquetBenchmarkScenariosTest`. - Release microbenchmark: same host/core/fixture, 3 warmups per binary, A/B/B/A order, 10 repetitions. The reported stable runs have CV below 3%. | Dictionary INT32 scan | Baseline median | Optimized median | Change | |---|---:|---:|---:| | 10% selectivity, predicate only | 798,618 ns | 466,085 ns | -41.6% | | 50% selectivity, predicate only | 797,469 ns | 476,725 ns | -40.2% | | 10% selectivity, predicate projected | 1,055,127 ns | 1,131,412 ns | +7.2% | | 50% selectivity, predicate projected | 1,138,954 ns | 1,181,781 ns | +3.8% | The small projected fixture exposes the fixed cost of dictionary probing and bitmap construction; it no longer materializes the full predicate value column. The Q9-style predicate-only path is the main target and improves by about 40–42%. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
