Vamsi-klu opened a new pull request, #18976: URL: https://github.com/apache/pinot/pull/18976
Fixes #12312 ## What changed This PR preserves exact decimal precision when Pinot ingests JSON records through the built-in JSON ingestion path. The change adds BigDecimal-aware parsing helpers to `JsonUtils` and routes the default JSON record reader, message decoder, and record extractor through those helpers. Numeric JSON values that cannot be represented exactly as `double` now remain `BigDecimal` until Pinot's downstream type conversion logic maps them into the target column type. The implementation also keeps the custom extractor compatibility boundary explicit: - The built-in `JSONRecordExtractor` now receives BigDecimal-preserving maps by default. - A user-supplied custom `recordExtractorClass` keeps the historical `Double` parsing behavior by default, so existing custom extractor code is not surprised by new numeric object types. - Custom extractors can opt into BigDecimal-preserving parsing with `preserveDecimalPrecision=true`. ## Why it matters JSON ingestion previously parsed decimal literals through Jackson's default floating-point path. That silently rounded values such as high-precision monetary amounts, identifiers encoded as decimal strings, or scientific measurements before Pinot had a chance to apply the configured schema conversion. That behavior is especially risky because the corruption happens during ingestion and can be hard to detect later: queries see a stable but already-rounded value. Preserving the JSON decimal as `BigDecimal` keeps the source value intact until Pinot decides how to store it for the configured column. The custom extractor compatibility guard matters because many deployments use custom extractors that may inspect raw map values directly. Changing those extractors from `Double` to `BigDecimal` without an opt-in could break otherwise unrelated ingestion jobs. ## Impact Built-in JSON ingestion now keeps decimal precision for values that need it. Existing ingestion pipelines that use Pinot's built-in JSON extractor get safer precision behavior without requiring table config changes. Existing custom JSON record extractors keep their previous raw numeric value types unless they explicitly opt into the new precision-preserving map parsing mode. No query execution behavior is changed by this PR. The change is limited to JSON input decoding and extraction. ## Testing - `./mvnw -Dmaven.repo.local=/tmp/m2-pinot-12312 -pl pinot-spi,pinot-plugins/pinot-input-format/pinot-json -Dtest=JsonUtilsTest,JSONRecordReaderTest,JSONMessageDecoderTest,JSONRecordExtractorTest test` - `./mvnw -Dmaven.repo.local=/tmp/m2-pinot-12312 -pl pinot-spi,pinot-plugins/pinot-input-format/pinot-json spotless:apply checkstyle:check license:check` - `git diff --check` The tests cover BigDecimal parsing from strings and byte slices, JSON record reader precision preservation, default message-decoder precision preservation, built-in extractor behavior, legacy custom extractor behavior, and explicit custom extractor opt-in. --- Drafted-by: Codex (GPT-5) (no human review before posting) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
