LuciferYang opened a new issue, #9584: URL: https://github.com/apache/paimon/issues/9584
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master, `2788fe596` (2.1-SNAPSHOT). ### Compute Engine Flink and Spark, creating a managed table with `'file.format' = 'json'`. ### Minimal reproduce step Create a JSON table with an unsupported type nested inside a container: ```sql CREATE TABLE t (v ARRAY<VARIANT>) WITH ('file.format' = 'json'); ``` `CREATE TABLE` succeeds. The failure comes later: on write, `JsonFormatWriter` reaches `CastExecutors.resolveToString` and throws `Cast VARIANT to StringType is not supported.`, and on read `JsonFileReader.convertPrimitiveStringToType` gets a null cast executor and dereferences it, so the query fails with a `NullPointerException` carrying no message. `JsonFileFormat.validateDataType` only ever looks at the outermost type root, because the four container cases fall through to a single `break`: ```java case ARRAY: case VECTOR: case MAP: case ROW: // All types are supported in JSON break; default: throw new UnsupportedOperationException("Unsupported data type for JSON format: " + dataType); ``` An unsupported type at the top level is rejected at create time with that message; the same type one level down is not. ### What doesn't meet your expectations? The other formats check nesting: ORC through `OrcTypeUtil.convertToOrcType`, Parquet through `ParquetSchemaConverter.convertToParquetType` and Avro through `AvroSchemaConverter.convertToSchema` all walk into element, key, value and field types and reject what they cannot represent. JSON was written like CSV, whose whitelist has no container types at all, but with the container types added to it, so the recursion was never there. The comment saying all types are supported is not accurate either: the whitelist has always rejected `MULTISET`, `VARIANT`, `BLOB`, `GEOMETRY` and `GEOGRAPHY` at the top level. ### Anything else? This is about when the error arrives, not about data being lost: none of those five types has a cast rule to or from string, so they already fail at runtime today. Note also that `validateDataFields` is only reached for managed tables; a format table is created through `CatalogUtils.validateCreateTable`, which does not call it. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
