LuciferYang opened a new pull request, #9588:
URL: https://github.com/apache/paimon/pull/9588
### Purpose
close #9587
`JsonFileReader.convertPrimitiveStringToType`'s default branch dereferenced
the result of `CastExecutors.resolve`, whose javadoc says it returns null when
no rule can be resolved. For a column whose type has no cast from STRING,
`MULTISET`, `VARIANT` or `BLOB`, the read failed with a `NullPointerException`
carrying no message. This throws instead, naming the type.
The exception and its wording are the ones the same format already uses at
create time, `UnsupportedOperationException("Unsupported data type for JSON
format: X")` from `JsonFileFormat.validateDataType`, so both stages report the
same thing. The other STRING-to-X resolve sites already do something sensible:
`CsvParser` falls back to the raw string, and `DefaultValueUtils` and the
`StringToArray`/`Map`/`Row` rules throw.
Reaching it needs a format table. `SchemaValidation` is the only entry point
for `validateDataFields`, so a managed table rejects these types at create
time, while a format table is created through
`CatalogUtils.validateCreateTable`, which does not call it.
One limit worth stating plainly: with `json.ignore-parse-errors = true`,
`handleParseError` swallows this like any other conversion failure, so the row
is emitted with that column null. This improves the default configuration.
Making that path fail loudly means resolving the casts once when the reader is
built, which is a change of its own.
### Tests
`JsonFileFormatTest.testUnresolvableCastFailsWithClearMessage` reads a
one-line JSON file with a `MULTISET<STRING>` read type and asserts both the
root cause type and its message. Asserting the type matters here, since the
message alone would also pass if the thrown type changed, and the type is the
point of the change.
Against the unfixed reader the root cause is a `NullPointerException` with
no message, so the assertion fails on the type.
`mvn -pl paimon-format test` on JDK 8: 597 tests, 0 failures.
`spotless:check` and `checkstyle:check` are clean.
Adjacent and not included: the same unguarded `CastExecutors.resolve`
dereference exists in `ArrayToStringCastRule`, `MapToStringCastRule`,
`RowToStringCastRule` and `InternalRowPartitionComputer`. I checked that they
are unguarded, not that they are reachable, so they seem better handled
separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
