ZZZxDong commented on code in PR #10983:
URL: https://github.com/apache/paimon/pull/10983#discussion_r4464382183


##########
paimon-core/src/main/java/org/apache/paimon/schema/SchemaValidation.java:
##########
@@ -1361,17 +1363,72 @@ private static void 
validateMergeFunctionFactory(TableSchema schema) {
             default:
         }
 
-        for (int i = 0; i < schema.logicalRowType().getFieldNames().size(); 
i++) {
-            String fieldName = schema.logicalRowType().getFieldNames().get(i);
-            String aggFuncName = options.fieldAggFunc(fieldName);
-            aggFuncName = aggFuncName == null ? options.fieldsDefaultFunc() : 
aggFuncName;
-            if (aggFuncName != null) {
-                FactoryUtil.discoverFactory(
-                        FieldAggregator.class.getClassLoader(),
-                        FieldAggregatorFactory.class,
-                        aggFuncName);
+        RowType rowType = schema.logicalRowType();
+        List<String> primaryKeys = schema.primaryKeys();
+        List<String> sequenceFields = options.sequenceField();
+        Set<String> excludedFields = new HashSet<>();
+        if (options.mergeEngine() == MergeEngine.PARTIAL_UPDATE) {
+            // Sequence-group ordering fields never get an aggregator.
+            excludedFields.addAll(allSequenceGroupOrderingFields(options));
+        }
+        for (int i = 0; i < rowType.getFieldNames().size(); i++) {
+            String fieldName = rowType.getFieldNames().get(i);
+            if (excludedFields.contains(fieldName)) {
+                continue;
+            }
+            String aggFuncName =
+                    resolveAggFuncName(options, fieldName, primaryKeys, 
sequenceFields);
+            if (aggFuncName == null) {
+                continue;
+            }
+            // Instantiate once so that type-incompatible configurations (e.g. 
'collect' on a
+            // non-array column) are rejected at DDL time instead of at 
read/write time.
+            FieldAggregatorFactory.create(
+                    rowType.getFieldTypes().get(i), fieldName, aggFuncName, 
options);
+        }
+    }
+
+    /**
+     * The fields named in the option keys of all sequence groups, e.g. {@code 
f2} for {@code
+     * fields.f2.sequence-group = f3}. They order the group instead of being 
aggregated.
+     */
+    private static List<String> allSequenceGroupOrderingFields(CoreOptions 
options) {
+        List<String> orderingFields = new ArrayList<>();
+        for (String optionKey : options.toMap().keySet()) {
+            if (isSequenceGroupOption(optionKey)) {
+                orderingFields.addAll(sequenceGroupOrderingFields(optionKey));
             }
         }
+        return orderingFields;
+    }
+
+    /**
+     * Mirrors how each merge engine resolves the aggregation function of a 
field, so that the
+     * aggregation factory can be instantiated once at DDL time.
+     */
+    @Nullable
+    private static String resolveAggFuncName(
+            CoreOptions options,
+            String fieldName,
+            List<String> primaryKeys,
+            List<String> sequenceFields) {
+        if (options.mergeEngine() == MergeEngine.AGGREGATE) {
+            return AggregateMergeFunction.getAggFuncName(
+                    fieldName, options, primaryKeys, sequenceFields);
+        }
+
+        if (sequenceFields.contains(fieldName)) {

Review Comment:
   Good catch, thanks. You are right: `sequence.field` columns are not 
sequence-group ordering fields, so they still go through aggregation resolution 
and can inherit `fields.default-aggregate-function`. My previous version 
skipped them entirely, which would have let an unknown function name slip 
through.
   
   I replaced my hand-written resolver with the runtime one, and now also let 
the runtime resolution drive the check:
   
   - `partial-update`: call `PartialUpdateMergeFunction#getAggFuncName` with 
the sequence-group ordering fields and protected fields, then instantiate the 
factory. This also keeps the "must use sequence group" check at DDL time.
   - `aggregation`: unchanged, `AggregateMergeFunction#getAggFuncName`.
   
   Added `testPartialUpdateGlobalSequenceFieldStillValidated`, which sets 
`sequence.field = f2` together with `fields.default-aggregate-function = 
collect` and asserts the DDL now reports the missing sequence group for `f2`. 
`SchemaValidationTest`, `SchemaManagerTest`, `PartialUpdateMergeFunctionTest` 
and `TableSchemaTest` all pass locally (224 tests).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to