Baymine opened a new pull request, #67376:
URL: https://github.com/apache/doris/pull/67376
### What problem does this PR solve?
Issue Number: N/A
Related PR: #59371
Problem Summary:
`json_extract_string` and its aliases on VARCHAR input currently parse and
materialize the whole document as JSONB before extracting a value. This PR adds
an on-demand VARCHAR extraction path:
- Parse paths with the authoritative `JsonbPath` parser, preserving existing
invalid-path errors.
- Traverse simple paths leg by leg with simdjson. MEMBER legs require
objects; non-negative ARRAY legs require arrays.
- Fall back to `JsonbParser::parse` + `JsonbPath::findValue` for root paths,
wildcards, `[last]`, negative indices, misses, and type mismatches.
- Preserve existing formatting, JSON null handling,
numeric-key/type-mismatch semantics, and embedded trailing-NUL bytes.
A previous attempt (#59371) was automatically closed as stale. This
implementation targets current master and adds stronger BE, FE, and regression
coverage.
#### Benchmark
A local throwaway RELEASE A/B harness generated 50,000 rows containing ~2.6
KB JSON documents. It extracted an early field before a large trailing array
using `$.profile.prefs.theme`, with 3 warmups and 7 measured iterations.
| Metric | Cast to JSONB | On-demand traversal |
| --- | ---: | ---: |
| Median latency | 678.8 ms | 42.3 ms |
| Speedup | - | 16.04x |
| Peak RSS | ~509708 KB | ~509784 KB |
| Output | byte-identical | byte-identical |
The harness is not committed and is not Google Benchmark. This is a
synthetic favorable shape: improvement varies with document layout and path
depth. Root, wildcard, `[last]`, and negative-index paths use the compatibility
fallback and receive no fast-path speedup.
### Release note
None
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [x] Manual test
Testing performed:
- BE ASAN unit tests: 2 passed, including path parity,
numeric-key/type-mismatch cases, trailing-NUL handling, and invalid-path errors.
- FE unit tests: 5 passed; FE build and checkstyle succeeded.
- Existing p0 `test_json_function`: 1 suite passed with 0 failures and
unchanged expected output.
- Candidate-cluster SQL covered simple and fallback paths, aliases, JSON
null/scalars, wildcards, `[last]`, invalid paths, and leg-type mismatch cases.
- BE ASAN/RELEASE builds passed; clang-format and changed-line clang-tidy
were clean.
- Behavior changed:
- [x] No.
- [ ] Yes.
Results and errors remain compatible; only execution strategy and
simple-path performance change.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
