RussellSpitzer commented on code in PR #18252:
URL: https://github.com/apache/iceberg/pull/18252#discussion_r4541017843
##########
api/src/main/java/org/apache/iceberg/types/Types.java:
##########
@@ -1155,6 +1155,147 @@ private Map<Integer, NestedField> lazyFieldsById() {
}
}
+ public static final class FileType extends NestedType {
+ public static final String NAME = "file";
+ public static final int NUM_NESTED_FIELDS = 6;
+
+ private static final String URI = "uri";
+ private static final String OFFSET = "offset";
+ private static final String SIZE = "size";
+ private static final String CONTENT_TYPE = "content_type";
+ private static final String CHECKSUM = "checksum";
+ private static final String INLINE = "inline";
+
+ /**
+ * Returns a file type for the field with the given ID.
+ *
+ * <p>The nested fields of the file use the {@link #NUM_NESTED_FIELDS} IDs
that follow {@code
+ * baseId}. {@code baseId} must be the ID of the field that holds this
type. UpdateSchema
+ * assigns new IDs to each column that it adds. A file type for a new
column can use any {@code
+ * baseId}.
+ *
+ * @param baseId the ID of the field that holds this type
+ * @return a file type with nested field IDs derived from baseId
+ */
+ public static FileType of(int baseId) {
+ return new FileType(baseId);
+ }
+
+ private final int baseId;
+
+ // Lazy and derived from baseId
+ private transient List<NestedField> fieldList = null;
Review Comment:
The unfortunate problem is "kryo". This is why we do the same thing in
Struct (and Map and List) because otherwise we have these fun
kryo-unserializable containers and Spark breaks.
The other solution I've done before with Kryo things is explicitly implement
readResolve but I find that even a bit more magical than doing the lazy
transient thing.
I think those are the only two realistic options here
1. Lazy Transient
2. implement kryo specific serde methods.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
