Shruthirdy opened a new pull request, #2585:
URL: https://github.com/apache/iceberg-go/pull/2585
Closes #2540.
`Schema.UnmarshalJSON` accepted a schema where two fields share ID `0`, but
rejected the same schema when both fields have ID `1`. The check it uses,
`checkDuplicateFieldIDs`, skips IDs <= 0, because those are placeholders
while a
schema is being built (#879). A schema read from metadata JSON is already
complete, so a repeated `0` is not a placeholder: lookups by ID then return
an
arbitrary one of the two fields, which is the problem behind #2393.
#2475 closed the same bug on the creation path (`NewMetadata`). This is the
follow-up for the read path.
### Change
- `checkFieldIDs` is the shared walker, with an `allIDs` parameter that
decides
whether IDs <= 0 take part in the uniqueness check.
- `checkDuplicateFieldIDs` keeps the previous behaviour for the four
construction call sites (`NewSchemaFromJSONFields`,
`NewSchemaWithIdentifiers`,
`AssignFreshSchemaIDsWithBase`, `ApplyNameMapping`), which are allowed to
hold
repeated `-1` or `0` IDs before IDs are assigned.
- `checkAllDuplicateFieldIDs` checks every ID and is called only from
`UnmarshalJSON`.
Reading a schema with duplicate IDs now fails the same way for `0` as it
already did for positive IDs:
```
invalid schema: multiple fields for id 0: created_at and event
```
This matches Java, which rejects such a schema when `SchemaParser` builds a
`Schema`: the constructor builds the ID-to-name index eagerly, and
`IndexByName.byId()` uses an `ImmutableMap.Builder`, which fails on a
repeated
key.
### Notes
- Manifest files embed the schema they were written with, so a table with
such a
schema fails on manifest reads too, not only on metadata load.
`releases.md`
has a note under "Unreleased compatibility notes".
- The construction paths are unchanged on purpose. Switching them over fails
real flows: an Arrow schema with no field IDs starts with every field at
`-1`
(`table/arrow_utils.go`), and `ApplyNameMapping` can leave unmapped fields
at `-1`.
### Approach
Rejecting on read follows Java and closes the only gap left, since
`UnmarshalJSON` already rejects duplicate positive IDs. Happy to change the
approach if maintainers prefer a different one.
### Tests
Two cases added to `TestUnmarshalSchemaRejectsDuplicateFieldIDs`: a duplicate
`0` at the top level, and a duplicate `0` nested in a struct. Both fail
without
the change; the existing positive-ID cases pass either way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]