rshdhere commented on code in PR #18935:
URL: https://github.com/apache/iceberg/pull/18935#discussion_r4572552411


##########
core/src/test/java/org/apache/iceberg/TestSchemaUpdate.java:
##########
@@ -2590,4 +2590,119 @@ public void 
testCaseInsensitiveMoveAfterNewlyAddedField() {
 
     assertThat(actual.asStruct()).isEqualTo(expected.asStruct());
   }
+
+  private static final Schema FILE_SCHEMA =
+      new Schema(
+          required(1, "id", Types.LongType.get()),
+          optional(2, "photo", Types.FileType.of(2)),
+          optional(9, "data", Types.StringType.get()));
+
+  private static SchemaUpdate fileUpdate() {
+    return new SchemaUpdate(FILE_SCHEMA, FILE_SCHEMA.highestFieldId());
+  }
+
+  @Test
+  void cannotAddColumnToFileColumn() {
+    assertThatThrownBy(() -> fileUpdate().addColumn("photo", "extra", 
Types.StringType.get()))
+        .isInstanceOf(IllegalArgumentException.class)
+        .hasMessage("Cannot add to non-struct column: photo: file");
+  }
+
+  @Test
+  void cannotDeleteFileNestedField() {
+    assertThatThrownBy(() -> fileUpdate().deleteColumn("photo.checksum"))
+        .isInstanceOf(IllegalArgumentException.class)
+        .hasMessage("Cannot change a nested field of a file column: 
photo.checksum");
+  }
+
+  @Test
+  void cannotRenameFileNestedField() {
+    assertThatThrownBy(() -> fileUpdate().renameColumn("photo.uri", 
"location"))
+        .isInstanceOf(IllegalArgumentException.class)
+        .hasMessage("Cannot change a nested field of a file column: 
photo.uri");
+  }
+
+  @Test
+  void cannotPromoteFileNestedField() {
+    assertThatThrownBy(() -> fileUpdate().updateColumn("photo.size", 
Types.LongType.get()))
+        .isInstanceOf(IllegalArgumentException.class)
+        .hasMessage("Cannot change a nested field of a file column: 
photo.size");
+  }
+
+  @Test
+  void cannotUpdateFileNestedFieldDoc() {
+    assertThatThrownBy(() -> fileUpdate().updateColumnDoc("photo.uri", "the 
location"))
+        .isInstanceOf(IllegalArgumentException.class)
+        .hasMessage("Cannot change a nested field of a file column: 
photo.uri");
+  }
+
+  @Test
+  void cannotUpdateFileNestedFieldDefault() {
+    assertThatThrownBy(
+            () -> fileUpdate().updateColumnDefault("photo.uri", 
Literal.of("s3://bucket/key")))
+        .isInstanceOf(IllegalArgumentException.class)
+        .hasMessage("Cannot change a nested field of a file column: 
photo.uri");
+  }
+
+  @Test
+  void cannotUpdateFileNestedFieldRequirement() {
+    assertThatThrownBy(() -> fileUpdate().requireColumn("photo.uri"))
+        .isInstanceOf(IllegalArgumentException.class)
+        .hasMessage("Cannot change a nested field of a file column: 
photo.uri");
+    assertThatThrownBy(() -> fileUpdate().makeColumnOptional("photo.uri"))
+        .isInstanceOf(IllegalArgumentException.class)
+        .hasMessage("Cannot change a nested field of a file column: 
photo.uri");
+  }
+
+  @Test
+  void cannotMoveFileNestedField() {
+    assertThatThrownBy(() -> fileUpdate().moveFirst("photo.checksum"))
+        .isInstanceOf(IllegalArgumentException.class)
+        .hasMessage("Cannot move fields in non-struct type: file");
+    assertThatThrownBy(() -> fileUpdate().moveBefore("photo.checksum", 
"photo.uri"))
+        .isInstanceOf(IllegalArgumentException.class)
+        .hasMessage("Cannot move fields in non-struct type: file");
+    assertThatThrownBy(() -> fileUpdate().moveAfter("photo.uri", 
"photo.inline"))
+        .isInstanceOf(IllegalArgumentException.class)
+        .hasMessage("Cannot move fields in non-struct type: file");
+  }
+
+  @Test
+  void renameFileColumn() {
+    Schema renamed = fileUpdate().renameColumn("photo", "image").apply();
+
+    
assertThat(renamed.findField("image").type()).isEqualTo(Types.FileType.of(2));
+    assertThat(renamed.findField("image.uri").fieldId()).isEqualTo(3);
+  }
+
+  @Test
+  void deleteFileColumn() {
+    Schema deleted = fileUpdate().deleteColumn("photo").apply();
+
+    assertThat(deleted.findField("photo")).isNull();
+    assertThat(deleted.asStruct())
+        .isEqualTo(
+            new Schema(
+                    required(1, "id", Types.LongType.get()),
+                    optional(9, "data", Types.StringType.get()))
+                .asStruct());
+  }
+
+  @Test
+  void addFileColumnReservesNestedIds() {

Review Comment:
   The description says `addColumn` handles a file column "at any depth", but 
this only covers a top-level add. I tried it locally and it works: adding 
`list<file>` then `map<string, file>` then a string column gives IDs 4 / 12 / 
21. It'd be good to pin that with a test, since the nested path goes through 
`assignFreshIds` with the reserving `NextID` rather than `typeForNewColumn`, 
and that's the easier one to regress. Something like adding a struct containing 
a file, then another column, and asserting the IDs.



##########
api/src/main/java/org/apache/iceberg/Schema.java:
##########
@@ -161,6 +163,20 @@ public Schema(
     this.highestFieldId = lazyIdToName().keySet().stream().mapToInt(i -> 
i).max().orElse(0);
   }
 
+  private void validateFileBaseIds() {
+    for (NestedField field : lazyIdToField().values()) {

Review Comment:
   This makes every `Schema` build the id-to-field index eagerly in the 
constructor, including schemas with no file columns. Before, that index was 
only built here when identifier fields were set. Schemas are created often 
(select / projections / parsing), so is it worth gating this, e.g. only 
building the index when the struct actually contains a file type, or doing the 
check in the traversal that already happens for `highestFieldId`?



##########
api/src/test/java/org/apache/iceberg/types/TestFileType.java:
##########
@@ -64,6 +64,52 @@ void findsNestedFieldsByName() {
     assertThat(FILE.fieldType("missing")).isNull();
   }
 
+  @Test
+  void rejectsASchemaWhereAnotherColumnHoldsADerivedId() {

Review Comment:
   The collision is reported through the generic duplicate-key message 
(`Multiple entries with same key: 3=photo.uri and 3=data`). For a user, `data` 
having ID 3 looks perfectly valid, since the reservation is implicit. Since 
`validateFileBaseIds` already walks the file columns, could it also check that 
`baseId + 1 .. baseId + 6` aren't taken by another field and say so explicitly 
(e.g. "ID 3 is reserved by file column photo")? Not blocking, just thinking 
about the person who hits it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to