fbocse commented on a change in pull request #244: [WIP] Add new Update schema
API to support adding of all new fields detected based on a companion schema
URL: https://github.com/apache/incubator-iceberg/pull/244#discussion_r313681588
##########
File path: core/src/main/java/org/apache/iceberg/SchemaUpdate.java
##########
@@ -186,6 +188,88 @@ public UpdateSchema updateColumnDoc(String name, String
doc) {
return this;
}
+ @Override
+ public UpdateSchema equateNewFields(Schema schema) {
+ Preconditions.checkArgument(schema != null, "Cannot add new fields off of
a null schema");
+ List<TypeUtil.WrappedType> wrappedTypes =
TypeUtil.CompanionSchemaVisitor.schema(this.schema, schema,
+ new CompanionSchemaNewFieldsCollector());
+
+ // With Iceberg we can only add to pre-existing fields so we need to
"hydrate" all nested types.
+ wrappedTypes.forEach(
+ wrappedType -> {
+ Map.Entry<String, Types.NestedField> entry = inflate(wrappedType);
+ // Based on being top-level or nested we use the corresponding API
for adding new columns to the schema
+ Preconditions.checkArgument(entry != null, "Cannot inflate types of
null predecessor");
+ if (entry.getKey().isEmpty()) {
+ addColumn(entry.getValue().name(), entry.getValue().type(),
entry.getValue().doc());
+ } else {
+ addColumn(entry.getKey(), entry.getValue().name(),
entry.getValue().type(), entry.getValue().doc());
+ }
+ }
+ );
+
+ return this;
+ }
+
+ /**
+ * Given a nested type structure this will provide a fully hydrated {@link
Types.NestedField} accompanied by
+ * predecessor name as dot separated string.
+ * TODO - any suggestions for modelling a tuple other than
java.util.Map.Entry?
+ *
+ * @param wrappedType the nested type we want to inflate (will use method
recursively for non-primitive structures)
+ * @return a tuple having a String representation of the type predecessor as
key and the nested field as value
+ */
+ private <T extends TypeUtil.WrappedType> Map.Entry<String,
Types.NestedField> inflate(T wrappedType) {
Review comment:
Good point, when writing the code I did kind of ended up with this synthetic
`WrappedType` construct but after putting together the code for inflating
nested types I should just ditch using the `WrappedType` construct altogether.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
