yihua commented on code in PR #13654:
URL: https://github.com/apache/hudi/pull/13654#discussion_r2272187007
##########
hudi-common/src/main/java/org/apache/hudi/avro/AvroSchemaUtils.java:
##########
@@ -376,6 +376,95 @@ public static Schema
createNewSchemaFromFieldsWithReference(Schema schema, List<
return newSchema;
}
+ /**
+ * If schemas are projection equivalent, then a record with schema1 does not
need to be projected to schema2
+ * because the projection will be the identity.
+ *
+ * Two schemas are considered projection equivalent if the field names and
types are equivalent.
+ * The names of records, namespaces, or docs do not need to match.
Nullability is ignored.
+ */
+ public static boolean areSchemasProjectionEquivalent(Schema schema1, Schema
schema2) {
+ return
AvroSchemaComparatorForRecordProjection.areSchemasProjectionEquivalent(schema1,
schema2);
Review Comment:
nit: this can be directly used without adding
`AvroSchemaUtils#areSchemasProjectionEquivalent`
##########
hudi-common/src/main/java/org/apache/hudi/avro/AvroSchemaComparatorForRecordProjection.java:
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.avro;
+
+import org.apache.avro.Schema;
+
+import java.util.List;
+
+import static org.apache.hudi.avro.AvroSchemaUtils.resolveNullableSchema;
+
+public class AvroSchemaComparatorForRecordProjection extends
AvroSchemaComparatorForSchemaEvolution {
+
+ private static final AvroSchemaComparatorForRecordProjection INSTANCE = new
AvroSchemaComparatorForRecordProjection();
+
+ public static boolean areSchemasProjectionEquivalent(Schema s1, Schema s2) {
+ return INSTANCE.schemaEqualsInternal(s1, s2);
+ }
+
+ @Override
+ protected boolean schemaEqualsInternal(Schema s1, Schema s2) {
+ if (s1 == s2) {
+ return true;
+ }
+ if (s1 == null || s2 == null) {
+ return false;
+ }
+ return super.schemaEqualsInternal(resolveNullableSchema(s1),
resolveNullableSchema(s2));
+ }
+
+ @Override
+ protected boolean validateRecord(Schema s1, Schema s2) {
+ return true;
+ }
+
+ @Override
+ protected boolean validateField(Schema.Field f1, Schema.Field f2) {
+ return f1.name().equalsIgnoreCase(f2.name());
Review Comment:
Does this account for case insensitivity of column names?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
