airborne12 opened a new pull request, #69891:
URL: https://github.com/apache/doris/pull/69891
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: A phrase query decodes the positions of its candidate
documents a window at a
time. In a PFOR window the position counts of all the documents come first,
and they are decoded to
find where the positions of a selected document start, however few documents
are selected.
`decode_selected_pfor_count_ranges` went through every count of the window,
with the running checks
and a test for the next selected document at each one. On a 99M-row table
that loop was 16% of the
SNII stack samples of a phrase prefix query, nearly all of it in the loop
rather than in the PFOR
decode, because such a query selects no document in most runs.
A run of counts is now added up whole, with no branch per document, so the
compiler can vectorize
it, and then walked from one selected document to the next, adding up only
the counts in between.
The sum of a run is kept in 64 bits and checked against the cap before any
of its documents is used,
so a window above the cap is rejected with the same error. The on-disk
format and the write path do
not change, and neither do the positions or the window shape returned.
| Query (99M rows) | Hot, new/previous | Cold, new/previous |
|---|---|---|
| phrase prefix `'retry budget dep'` | 0.96 / 0.94 / 0.95 / 0.95 (four runs)
| 0.99 |
| phrase with a latency filter | 0.97 / 0.94 / 0.97 / 0.98 | 0.99 |
| phrase `'unable to open'` in input or output | 0.98 / 0.96 / 0.98 / 0.99 |
0.92 |
| the other 22 queries | around 1.0, no trend (0.90-1.06) | |
In stack samples of the phrase prefix, with the same binary and both
versions switched at run time,
`decode_selected_pfor_count_ranges` fell from 16.1% to 3.4% of the SNII
samples.
Hot: both versions interleaved in one BE process, on a host with a load of
about 100 to 150 from
other builds, so differences of a few percent in a single run are noise.
Cold: BE restarted and the
index files evicted before every run, configuration order alternated, 5 runs
each. The change only
saves CPU, so cold runs, which mostly wait for reads, stay level as
expected. The build also had
#69879, #69882 and #69887, with `snii_inline_read_max_segments` at 16.
### Release note
Speed up SNII phrase queries on selective candidates.
### Check List (For Author)
- Test: Unit Test (`SniiPrxPod`: random PFOR windows with sparse and dense
selections, run edges,
documents with no positions and PFOR exceptions match the input and the
window shape; counts that
add up to 2^38, 0 in 32 bits, are rejected at the cap; six mutants of the
new loop all fail tests;
all `*Snii*` and `*Prx*` tests pass under ASAN).
Manual test: the hot and cold A/B above.
- Behavior changed: No
- Does this need documentation: No
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
