badalprasadsingh commented on code in PR #2585:
URL: https://github.com/apache/iceberg-go/pull/2585#discussion_r4575294230


##########
schema.go:
##########
@@ -422,7 +435,7 @@ func (s *Schema) UnmarshalJSON(b []byte) error {
                return fmt.Errorf("%w: invalid schema fields: %w", 
ErrInvalidSchema, err)
        }
 
-       if err := checkDuplicateFieldIDs(nil, fields); err != nil {
+       if err := checkAllDuplicateFieldIDs(fields); err != nil {

Review Comment:
   it can now writes metadata it cannot read: 
   
   "table.MetadataBuilder.AddSchema" and "view.MetadataBuilder" keep caller ids 
without a duplicate check -> so sql CreateView, or CommitTable + 
NewAddSchemaUpdate on non-REST catalogs: accept unset ids then fail to load
   
   better: make both builders reject duplicates, like NewMetadata does
   
   a blocking approval comment



##########
schema_test.go:
##########
@@ -917,6 +917,42 @@ func TestUnmarshalSchemaRejectsDuplicateFieldIDs(t 
*testing.T) {
                        }`,
                        contains: "multiple fields for id 1: id and key",
                },
+               {
+                       name: "duplicate zero ids",
+                       schema: `{
+                               "type": "struct",
+                               "fields": [
+                                       {"id": 0, "name": "created_at", "type": 
"timestamptz", "required": false},

Review Comment:
   the new cases only cover struct fields
   
   can you please add a duplicate -1, pleus a duplicate 0 on a list element and 
a map value? each branch of "checkFieldIDs" passes allIDs on its own, and the 
doc promises negatives too



##########
website/src/releases.md:
##########
@@ -49,6 +49,7 @@ Per-release notes (highlights, breaking changes, 
contributors) are published on
 - **SigV4 keeps the token or credential as `Original-Authorization`.** With 
`rest.sigv4-enabled=true` (or `rest.auth.type=sigv4`) and a `token` or 
`credential`, requests used to be signed with the token silently dropped, 
because the SigV4 signature replaced the `Authorization` header. They now carry 
it as `Original-Authorization`, included in the signed headers, as Iceberg Java 
does. A server behind SigV4 that ignores the header is unaffected.
 - **Float and double literal comparators use Iceberg ordering.** 
`TypedLiteral.Comparator()` now orders `-0.0` before `+0.0` and NaN after all 
other values; NaN compares equal to NaN. Callers that use these public 
comparators may see a different order or equality result.
 - **Table creation rejects duplicate field IDs.** `NewMetadata`, and 
`CreateTable` on catalogs that build metadata through it (not REST), now return 
`ErrInvalidSchema` when two fields in the schema share an ID, including unset 
`0` or `-1` IDs, even for unpartitioned, unsorted tables. Before, the partition 
spec or sort order could bind to the wrong column. Callers should assign unique 
IDs (or let the schema be built with fresh IDs). This matches Java, whose 
`Schema` constructor rejects duplicate IDs.
+- **Schema reads now reject duplicate field IDs.** Table, view and manifest 
metadata whose schema has two fields sharing an ID now fails to load with 
`ErrInvalidSchema`, for example `multiple fields for id 0: created_at and 
event`. Duplicate positive IDs were already rejected on read, but `0` and 
negative IDs were skipped because they are placeholders while a schema is being 
built, so such metadata loaded and lookups by ID returned an arbitrary one of 
the fields. Java Iceberg already rejects these schemas when `SchemaParser` 
builds a `Schema`. Repair affected metadata through an atomic catalog metadata 
replacement instead of editing the currently referenced metadata file in place. 
Schemas built in process are unaffected.

Review Comment:
   pllease say who is affected and how to repair
   
   go wrote these: sql CreateView and non-REST "CommitTable" + 
"NewAddSchemaUpdate" with unset ids, an old bad schema in history also blocks 
loading
   
   views can be dropped and recreated -> tables need the catalog pointed at a 
metadata copy without that schema
   
   a blocking approval comment as well



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to