JingsongLi commented on code in PR #11163:
URL: https://github.com/apache/paimon/pull/11163#discussion_r4581461583


##########
paimon-python/pypaimon/execution/python/read/reader/aggregate/aggregators.py:
##########
@@ -1021,15 +1024,31 @@ def _check_option_dependencies(
                 "requires 'fields.<field-name>.nested-key' to be configured."
             )
 
-        if (
-                options.field_nested_update_agg_nested_sequence_field(field)
-                and not nested_key
-        ):
+        nested_sequence_field = (
+            options.field_nested_update_agg_nested_sequence_field(field)
+        )
+        if nested_sequence_field and not nested_key:
             raise ValueError(
                 "Option 'fields.<field-name>.nested-sequence-field' "
                 "requires 'fields.<field-name>.nested-key' to be configured."
             )
 
+        overwrite_on_equal_configured = options.options.contains_key(
+            f"{CoreOptions.FIELDS_PREFIX}.{field}."
+            f"{CoreOptions.NESTED_SEQUENCE_OVERWRITE_ON_EQUAL}"
+        )
+        if overwrite_on_equal_configured and not nested_sequence_field:
+            raise ValueError(
+                "Option 
'fields.<field-name>.nested-sequence-overwrite-on-equal' "
+                "requires 'fields.<field-name>.nested-sequence-field' to be 
configured."
+            )
+
+    def _should_update(self, new_row: Record, old_row: Record) -> bool:
+        if not self.has_sequence_field:
+            return True
+        comparison = self._compare_sequence(new_row, old_row)
+        return comparison > 0 or (comparison == 0 and 
self.overwrite_on_equal_sequence)

Review Comment:
   [P1] Gate Native reads and writes until they honor this policy
   
   This new policy is only applied by the Python aggregator, while Native read 
and write remain eligible (and enabled by default). With 
`fields.updates.aggregate-function=nested_update`, nested key `key`, nested 
sequence `seq`, and `nested-sequence-overwrite-on-equal=false`, I committed two 
versions of one PK/nested key at the same `seq=7`: Python reads `first`, but 
Native Rust 0.5.0 reads `second`, with Python fallback forbidden. More 
seriously, a single public `write_arrow` batch containing those two versions 
through `NativeBatchTableWrite` persists only `second`; reopening via the 
Python reader cannot recover `first`. The Python-writer control retains 
`first`, and explicit `true` controls agree on `second`.
   
   Please gate both Native read and write for this setting until the supported 
runtime implements it (or update/require that runtime), and add real-table 
regression coverage for both paths. Otherwise normal backend selection silently 
changes the result and can irreversibly discard the value this option promises 
to retain.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to