JingsongLi commented on code in PR #1406:
URL: https://github.com/apache/paimon-rust/pull/1406#discussion_r4581215025


##########
crates/paimon/src/table/aggregator/nested.rs:
##########
@@ -293,6 +316,18 @@ impl NestedAgg {
         Ok(Ordering::Equal)
     }
 
+    /// Whether `new` may replace `old` for `nested_update` (Java 
`shouldUpdate`).
+    fn should_update(&self, new: &StructArray, old: &StructArray) -> 
crate::Result<bool> {
+        if self.sequence_indices.is_empty() {
+            return Ok(true);
+        }
+        match self.compare_sequence(new, old)? {
+            Ordering::Greater => Ok(true),
+            Ordering::Equal => Ok(self.overwrite_on_equal_sequence),

Review Comment:
   [P1] Reject or correctly support false with partial-update sequence groups
   
   The new tie rule makes query results depend on flush boundaries. Reproduced 
through public Table write/commit/read/compact APIs with `id INT, version INT, 
items ARRAY<ROW<item_id INT, amount INT>>`, PK `id`, `bucket=1`, 
`merge-engine=partial-update`, `fields.version.sequence-group=items`, and 
`fields.items.{aggregate-function=nested_update, nested-key=item_id, 
nested-sequence-field=item_id, nested-sequence-overwrite-on-equal=false}`. For 
one PK, write `(version,items)` in order `(3,[(1,30)]), (1,[(1,10)]), 
(2,[(1,20)])`. Three separate commits read `[(1,20)]`; committing version 3 
first and flushing versions 1 and 2 together reads `[(1,10)]`. The true control 
returns 30 with both groupings; aggregation returns 10 with both false 
groupings.
   
   After merging outer versions 3 and 1, the nested winner is 10, but 
PartialUpdateMergeFunction still records group max version 3. Version 2 then 
enters `agg_reversed`, incorrectly prepending 20 ahead of the version-1 winner; 
this equal branch prevents 10 from replacing it. Full compaction preserves each 
differing result. Please retain the ordering/provenance needed for each nested 
winner, or reject false with partial-update sequence groups until supported. 
Add a regression with identical arrival order and different flush groupings.



##########
docs/src/sql.md:
##########
@@ -3556,9 +3556,14 @@ Field-scoped options used by these functions:
 - `fields.<field>.ts-field` selects the key-time field for
   `merge_map_with_keytime` (defaults to the last ROW field).
 - `fields.<field>.nested-key`, `fields.<field>.nested-sequence-field`,
+  `fields.<field>.nested-sequence-overwrite-on-equal` (default `true`;
+  requires `nested-sequence-field`),
   `fields.<field>.nested-key-null-strategy`, and `fields.<field>.count-limit`
   for nested array aggregators. `nested_partial_update` requires
-  `nested-key`.
+  `nested-key`. When `nested-sequence-field` is set, a nested ROW overwrites

Review Comment:
   [P2] Limit the sequence-overwrite guarantee to nested_update
   
   The new rule follows both nested aggregators, but 
`NestedMode::PartialUpdate` bypasses `should_update`. `nested_partial_update` 
still applies equal/lower-sequence non-null fields even with false, as 
`nested_partial_update_unaffected_by_overwrite_on_equal` confirms. Explicitly 
scope this guarantee to `nested_update` so users do not rely on it for 
stale-event protection with `nested_partial_update`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to