badalprasadsingh commented on code in PR #2591:
URL: https://github.com/apache/iceberg-go/pull/2591#discussion_r4577807660


##########
partitions.go:
##########
@@ -90,7 +90,20 @@ func (p *PartitionField) EscapedName() string {
        return url.QueryEscape(p.Name)
 }
 
+func validatePartitionFieldSourceCount(transform Transform, sourceIDs []int) 
error {
+       if len(sourceIDs) > 1 && isBuiltInPartitionTransform(transform) {
+               return fmt.Errorf("%w: built-in partition transform %s requires 
exactly one source ID, got %d",
+                       ErrInvalidPartitionSpec, transform, len(sourceIDs))
+       }
+
+       return nil
+}
+
 func (p PartitionField) MarshalJSON() ([]byte, error) {
+       if err := validatePartitionFieldSourceCount(p.Transform, p.SourceIDs); 
err != nil {
+               return nil, err
+       }

Review Comment:
   p1: once reads stay lenient (lne228), this makes those tables read-only on 
sql, glue, hive and hadoop: each commit writes all specs, fails, and leaves an 
empty metadata file
   
   "addSpecFieldInternal" already rejects new specs when they bind on create or 
commit, so better drop this check.



##########
partitions_test.go:
##########
@@ -895,6 +899,88 @@ func TestPartitionFieldUnmarshalJSON(t *testing.T) {
        })
 }
 
+func TestPartitionFieldJSONRejectsMultipleSourcesForBuiltInTransforms(t 
*testing.T) {

Review Comment:
   no test decodes a create or add-spec payload with such a field, so the 
payload path is untested: if the change is introduced, better: cover both sides 
here: payloads fail, and the same JSON still loads and round-trips as a 
"PartitionSpec"



##########
website/src/releases.md:
##########
@@ -45,6 +45,7 @@ Per-release notes (highlights, breaking changes, 
contributors) are published on
 - **Data-file replacements now reject concurrent deletes.** Commits staged 
through `ReplaceDataFiles`, `ReplaceDataFilesByPath`, `ReplaceFiles`, or 
rewrite operations can return `table.ErrConflictingDeleteFiles` when a 
concurrent position delete targets a replaced file or any concurrent equality 
delete is added. Compaction on tables with streaming equality deletes may see 
additional conflicts.
 - **Orphan cleanup now rejects `gc.enabled=false`.** `DeleteOrphanFiles`, 
`PlanOrphanFiles`, `ExecuteOrphanCleanup`, and dry runs now return an error 
before filesystem access. The `iceberg clean-orphan-files` command is also 
affected.
 - **Partition specs reject source types a transform cannot handle.** Creating 
or updating a spec with an incompatible source, such as `day(string)`, now 
fails. Writes to existing incompatible specs return `ErrInvalidTransform` 
instead of writing null partition values.
+- **Partition JSON rejects multiple sources for built-in transforms.** 
Decoding or encoding a field with multiple source IDs and a built-in transform, 
such as `bucket[16]`, now returns `ErrInvalidPartitionSpec`. Multi-source 
unknown transforms still round-trip.

Review Comment:
   this must change either way:
   
   if reads stay strict, say tables holding such a spec stop loading and can't 
be repaired, or, with the changes in partitions.go, create and add-spec 
payloads only fail earlier (they already fail when bound), so the bullet can go



##########
partitions.go:
##########
@@ -212,6 +225,9 @@ func (p *PartitionField) unmarshal(b []byte, binding 
specBinding) error {
                // which carry no source column to validate.
                next.SourceIDs = []int{0}
        }
+       if err := validatePartitionFieldSourceCount(next.Transform, 
next.SourceIDs); err != nil {
+               return err
+       }

Review Comment:
   this runs on every metadata load: bind, lin448, and the writers already 
reject these specs, so it adds no safety, but any such table go main wrote 
earlier stops loading for good
   
   line783 says refusing to read a spec costs the table -> can only payloads 
(unboundSpec) check this?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to