Akash3121 opened a new pull request, #10031:
URL: https://github.com/apache/paimon/pull/10031
### Purpose
### Purpose
Fixes #9984.
The JSON format provides the following options for handling null map keys:
- `json.map-null-key-mode=FAIL`
- `json.map-null-key-mode=DROP`
- `json.map-null-key-mode=LITERAL`
- `json.map-null-key-literal`
However, `JsonFormatWriter` did not apply these options. It unconditionally
converted every map key and inserted it into the output map. When a map
contained a null key, the key remained null and serialization later failed
inside Jackson, regardless of the configured mode.
This PR makes the JSON writer honor the configured null-map-key behavior:
| Mode | Behavior |
|------|----------|
| `FAIL` | Throws a clear `IllegalArgumentException` identifying
`json.map-null-key-mode` |
| `DROP` | Removes the complete entry containing the null key |
| `LITERAL` | Replaces the null key with `json.map-null-key-literal` |
For `LITERAL` mode, the configured literal is validated against the declared
map key type. This prevents the writer from producing JSON that the reader
cannot convert back to the expected key type.
For example:
- The literal `"0"` is valid for `MAP<INT, ...>`.
- The default literal `"null"` is invalid for `MAP<INT, ...>` and now
produces a clear error.
- String-keyed maps continue to support the default `"null"` literal and
custom literals.
A shared package-private `JsonMapKeyConverter` is introduced so the writer
validation and reader conversion use the same rules. The existing reader
conversion behavior is preserved while avoiding separate implementations that
could become inconsistent.
If the configured literal collides with an existing map key, normal map
insertion semantics apply: the entry processed last wins.
### Tests
Updated `JsonFileFormatTest` to cover:
- `FAIL` mode with an actual null map key.
- `DROP` mode removing the null-key entry.
- `LITERAL` mode with the default literal.
- `LITERAL` mode with multiple custom literals.
- Null keys inside nested maps.
- Raw serialized JSON for `DROP`, default `LITERAL`, and custom `LITERAL`.
- Literal collisions in both insertion orders.
- Invalid literals for non-string map key types.
- Valid literals for integer map key types.
- Round-trip serialization and deserialization.
The focused tests were run with:
```shell
mvn -pl paimon-format -am \
-DfailIfNoTests=false \
-DwildcardSuites=none \
-Dtest=JsonFileFormatTest#testMapNullKeyModeFailWithWriteRead+testMapNullKeyModeDropWithWriteRead+testDifferentMapNullKeyLiteralsWithWriteRead+testMapNullKeyLiteralInNestedMap+testMapNullKeySerializedOutput+testMapNullKeyLiteralCollisionUsesLastEntry+testMapNullKeyLiteralMustMatchKeyType+testMapNullKeyLiteralWithNonStringKeyType
\
test
```
Result:
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0
BUILD SUCCESS
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
