JingsongLi commented on code in PR #1406:
URL: https://github.com/apache/paimon-rust/pull/1406#discussion_r4586084530


##########
crates/paimon/src/spec/aggregation.rs:
##########
@@ -272,6 +274,10 @@ fn parse_field_scoped_option_key(key: &str) -> 
Option<(&str, FieldScopedOptionKi
             NESTED_SEQUENCE_FIELD_SUFFIX,
             FieldScopedOptionKind::NestedSequenceField,
         ),
+        (
+            NESTED_SEQUENCE_OVERWRITE_ON_EQUAL_SUFFIX,

Review Comment:
   [P2] Register the new field option in rename and drop handling too
   
   This adds the suffix to field validation, but 
`FIELD_SCOPED_RENAMEABLE_SUFFIXES`, used by both `rename_field_scoped_options` 
and `remove_field_scoped_options`, omits it. Consequently the old 
`fields.items.nested-sequence-overwrite-on-equal=false` key survives 
renaming/dropping items, and final schema validation rejects the ALTER because 
items no longer exists.
   
   I isolated this from the pre-existing omissions for older nested options 
with public `TableSchema::apply_changes`: explicitly migrate 
`nested-sequence-field` alongside the rename (nested-key is renamed by the 
existing field-list handler), or explicitly remove `nested-key` and 
`nested-sequence-field` before the drop. Both controls without the new option 
succeed; adding only this new option makes both operations fail with 
`Aggregation field 'items' ... is not declared`. Please include this suffix in 
the lifecycle registry and add rename/drop regressions.



##########
crates/paimon/src/table/aggregator/nested.rs:
##########
@@ -293,6 +316,18 @@ impl NestedAgg {
         Ok(Ordering::Equal)
     }
 
+    /// Whether `new` may replace `old` for `nested_update` (Java 
`shouldUpdate`).
+    fn should_update(&self, new: &StructArray, old: &StructArray) -> 
crate::Result<bool> {
+        if self.sequence_indices.is_empty() {
+            return Ok(true);
+        }
+        match self.compare_sequence(new, old)? {
+            Ordering::Greater => Ok(true),
+            Ordering::Equal => Ok(self.overwrite_on_equal_sequence),

Review Comment:
   [P1] Also reject or support false with aggregation and an outer sequence 
field
   
   The unsafe provenance case remains reachable outside partial-update. 
Reproduced with PK `pk`, `bucket=1`, `merge-engine=aggregation`, 
`sequence.field=version`, and `items ARRAY<ROW<item_id INT, amount INT>>` 
configured with `aggregate-function=nested_update`, `nested-key=item_id`, 
`nested-sequence-field=item_id`, `nested-sequence-overwrite-on-equal=false`. 
For one PK and item_id=1, input `(version,amount)` in arrival order 
`(3,30),(1,10),(2,20)` reads amount 10 when committed separately or in one 
batch, but 20 when versions 3 and 1 are flushed together before committing 
version 2. Explicit compaction preserves 10 versus 20. The true control returns 
30 for every grouping; without the outer sequence field both option values are 
grouping-consistent.
   
   The first flush retains the version-1 nested winner but records version 3 in 
the aggregate row. The next merge orders version 2 before that summary; this 
equal branch then retains 20. The new guard only covers PartialUpdateConfig. 
Please preserve the nested winner's ordering/provenance or reject this 
aggregation + outer-sequence combination until supported, and add a public 
regression with identical arrival order and different flush groupings.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to