xiangfu0 opened a new pull request, #18503:
URL: https://github.com/apache/pinot/pull/18503
## Summary
- Fix `ExpressionTransformer` overriding an existing non-null `byte[]`
(BYTES column) with null when a transform expression evaluates to null.
`byte[]` is logically a scalar in Pinot, but it accidentally falls into the
`isArray()` branch intended for nested fields, so the null-result path called
`applyTransformedValue(record, column, null)` and cleared the BYTES column.
- Extract a `shouldPreserveExistingValue` helper that skips the apply step
when the transform yields null AND an existing non-null value is present, for
two cases:
1. Implicit map transforms (`mapField__KEYS`/`__VALUES`) — generalizes the
prior guard introduced in #17308 so it also covers the `shouldApplyTransform`
branch (e.g. columns flagged null via `putDefaultNullValue`).
2. BYTES single-value columns — treat `byte[]` as the scalar it represents.
- Guard applies in both the populate-missing/overwrite branch and the
existing array/Collection/Map branch, including the post-upsert overwrite path
(`_overwriteExistingValues=true`).
Related: #17308.
## Test plan
- [x] Added
`testImplicitMapTransformDoesNotOverrideExistingBytesValueWhenSourceMissing` —
implicit map + BYTES with default-null marker
- [x] Added `testTransformReturningNullDoesNotOverrideExistingBytesValue` —
explicit transform yielding null, regular ingestion path
- [x] Added
`testPostUpsertTransformReturningNullDoesNotOverrideExistingBytesValue` —
explicit transform yielding null, post-upsert overwrite path
- [x] All 18 tests in `ExpressionTransformerTest` pass
- [x] `./mvnw spotless:apply license:format checkstyle:check license:check
-pl pinot-segment-local` clean
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
