WZhuo opened a new issue, #1141:
URL: https://github.com/apache/iceberg-cpp/issues/1141

   The iceberg-verification fixture 
[`schema/unknown/reject-unknown-required`](https://github.com/apache/iceberg-verification/blob/fb0be4d6cb2af5529a176b976b4a775dc87b778f/table-spec/schema/unknown/cases.json#L24)
 fails: `SchemaFromJson` accepts a schema with a required field of type 
`unknown`. This came up while adopting the fixtures (#909).
   
   ```json
   {"type": "struct", "schema-id": 0, "fields": [
     {"id": 1, "name": "id", "required": true, "type": "long"},
     {"id": 2, "name": "future", "required": true, "type": "unknown"}]}
   ```
   
   The spec says an `unknown` field [must be 
optional](https://github.com/apache/iceberg/blob/1440e49e39a7c436a687b0289be067a5cf411e2c/format/spec.md#L281).
 iceberg-cpp does have this check, in 
[`ValidateFieldNullability`](https://github.com/apache/iceberg-cpp/blob/77fbd76e50dc8d5fb296bd8214a6fd7fe20627f7/src/iceberg/schema.cc#L141-L144),
 but only 
[`Schema::Validate(format_version)`](https://github.com/apache/iceberg-cpp/blob/77fbd76e50dc8d5fb296bd8214a6fd7fe20627f7/src/iceberg/schema.cc#L448)
 calls it. Table metadata parsing calls `Validate` after `SchemaFromJson`, so a 
schema read from table metadata is rejected. A schema read with 
`SchemaFromJson` alone is accepted.
   
   Java rejects this in the [`NestedField` 
constructor](https://github.com/apache/iceberg/blob/1440e49e39a7c436a687b0289be067a5cf411e2c/api/src/main/java/org/apache/iceberg/types/Types.java#L882-L885),
 so every path that builds a field rejects it.
   
   The nullability rule does not depend on the format version, so 
`SchemaFromJson` could run `ValidateFieldNullability` itself and return the 
error.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to