morrySnow opened a new pull request, #66902:
URL: https://github.com/apache/doris/pull/66902
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
`DorisLexer.STRING_LITERAL` evaluated SQL-mode semantic predicates for every
character, so predicate calls and allocation scaled with the literal payload.
This PR selects the SQL-mode-specific loop once after the opening quote while
preserving the accepted language and token stream.
It also adds:
- an isolated, opt-in JMH module that does not affect the default parser jar
or its runtime dependencies;
- lexer-only, parser end-to-end, allocation, and no-string control
benchmarks;
- deterministic token/error differential tooling;
- string literal boundary, long-input, predicate-complexity, and 8-thread
concurrency tests;
- a gated TODO for subsequent parser optimization work.
Representative plain single-quoted measurements:
- 4 KiB lexer: about 1.40–1.74 ms/op to 0.029–0.059 ms/op;
- 64 KiB lexer: about 21.6–24.4 ms/op to 0.55–0.70 ms/op;
- 4–64 KiB allocation reduction: about 99.6%–99.7%.
The benchmark host did not meet the strict 3% A/A stability gate for
microsecond control cases. The PR therefore reports raw ranges and explicitly
leaves fine-grained G2/G5 validation open instead of claiming small
control-path changes.
### Release note
None
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [x] Manual test
- `./run-fe-ut.sh --run
org.apache.doris.sqlparser.DorisLexerStringLiteralTest,org.apache.doris.sqlparser.DorisSqlParserTest`
- `DISABLE_BUILD_UI=ON ./build.sh --fe`
- 46,908-case baseline/candidate token and error snapshot
differential: zero differences
- JMH lexer-only, parser end-to-end, allocation, and no-string
control benchmarks
- [ ] No need to test or manual test.
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
