Mryange opened a new pull request, #64121:
URL: https://github.com/apache/doris/pull/64121
### What problem does this PR solve?
`count_substrings(str, pattern)` previously scanned each row byte by byte
and compared the pattern at every candidate offset with
`memcmp_small_allow_overflow15`. This is expensive when `pattern` is constant
across a block, especially for long strings or rare matches.
Root cause: the existing implementation did not reuse a prebuilt string
searcher for constant patterns, so every row still used the naive per-offset
comparison path.
This change builds one `ASCIICaseSensitiveStringSearcher` per block when
`pattern` is constant and uses it to count non-overlapping matches. The
non-constant pattern path is unchanged. A BE benchmark was added for
`count_substrings(str, const_pattern)` to compare the old naive path,
`StringSearch`, direct searcher, and the actual function path.
| Case | Naive | FunctionConstPattern | Speedup |
|---|---:|---:|---:|
| SmallFrequent | 1298 us | 446 us | 2.9x |
| MediumRare | 2753 us | 278 us | 9.9x |
| LongNoFirstByte | 21689 us | 1231 us | 17.6x |
| LongFalseFirstByte | 21026 us | 1215 us | 17.3x |
| LongRare | 21810 us | 1265 us | 17.2x |
| LongFrequent | 20672 us | 1435 us | 14.4x |
| LongNeedle | 20962 us | 1331 us | 15.7x |
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
