chaokunyang opened a new pull request, #19672:
URL: https://github.com/apache/pinot/pull/19672
Follow up on #19223 by upgrading the optional Fory JSON dependency to 1.7.5
and removing the temporary Strings created for unrelated JSON values. The
existing extractor still calls `readString()` for every skipped string even
though recent Fory releases provide a fast validating `skipValue()`.
When the input String is no longer than Jackson's configured string limit,
every decoded string is already within that limit, so the extractor uses
`skipValue()`. Larger documents use Fory's reusable quoted-text view to check
decoded UTF-16 length without constructing a String. This preserves
escaped-string length semantics, full-document validation, duplicate-key
behavior, and the existing Jackson/Jayway fallback. Runtime dependency examples
are updated to 1.7.5.
## Validation
- Common extraction and missing-runtime suites: 521 tests passed.
- `JsonExtractScalarTransformFunctionTest`: 246 tests passed.
- Full `pinot-perf -am package` reactor: 64 modules built successfully on
JDK 25.0.4.
- Spotless, license formatting, Checkstyle, and license checks passed for
the root POM, `pinot-common`, and `pinot-core`.
- Tests cover large ASCII, Latin1, Unicode and escaped skipped strings;
malformed skipped strings and recovery; and configured decoded-string length
boundaries in an isolated JVM.
## Benchmark
Existing `BenchmarkJsonPathExtraction.jsonPathStringFory` and
`jsonPathStringFast`, late-field hit, JMH 1.37 GC profiler, Apple M4 Pro, JDK
25.0.4, one thread/fork, 2 x 1s warmup, 3 x 1s measurement, 1 GiB heap.
Baseline and current runs were sequential after all builds completed. Both used
Maven Central release artifacts.
| Document | Fory on master (1.7.4), B/op | This PR (1.7.5), B/op | Fast
control, B/op |
| --- | ---: | ---: | ---: |
| 700 B | 1,120 | 48 | 816 |
| 8 KiB | 8,608 | 48 | 816 |
| 64 KiB | 65,952 | 48 | 880 |
Allocation values are rounded. At 64 KiB, allocation falls by 99.93%; Fory
throughput was about 169k ops/s before and 170k ops/s after. These are short
local runs, with wide throughput confidence intervals. Baseline:
`d4365386d8ba3605ec9b2fddd688a23c8ece80e5`.
JMH selection and timing:
```text
org.openjdk.jmh.Main
'^org\.apache\.pinot\.perf\.BenchmarkJsonPathExtraction\.jsonPathString(Fory|Fast)$'
-p _fieldPosition=late -p _documentBytes=700,8192,65536 -p _valueCase=hit
-t 1 -f 1 -wi 2 -i 3 -w 1s -r 1s -prof gc
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
