JeonDaehong commented on code in PR #1434:
URL: https://github.com/apache/paimon-rust/pull/1434#discussion_r4599175146


##########
crates/paimon/src/table/sort_merge.rs:
##########
@@ -945,7 +977,21 @@ impl PartialUpdateMergeFunction {
         if !initialized {
             return Ok(PickedMerge::Result(MergeResult::Omit));
         }
-        let deleted = current_delete_row || !saw_add;
+        // A row removed only by UPDATE_BEFORE stays UPDATE_BEFORE, so merging
+        // a pre-merged result again does not re-initialize the row like a
+        // DELETE would. This also covers keys with only UPDATE_BEFORE rows.
+        let result_kind = if current_delete_row {

Review Comment:
   Same as Java's `getResult`. Since `meet_delete` isn't reset by a later add, 
DELETE, +I, -U still ends as DELETE, as in Java.



##########
crates/paimon/src/table/sort_merge.rs:
##########
@@ -837,8 +869,9 @@ impl PartialUpdateMergeFunction {
                     continue;
                 }
                 if self.remove_record_on_delete {
+                    current_delete_row = true;

Review Comment:
   This matches the Java change line for line: every retract sets 
`currentDeleteRow`, only DELETE sets `meetDelete` and re-initializes the row, 
and -U goes through `updateNonNullFields`.



##########
crates/paimon/src/table/sort_merge.rs:
##########
@@ -505,6 +501,39 @@ impl PartialUpdateMergeFunction {
         Ok(())
     }
 
+    /// Java `updateNonNullFields`: ungrouped fields take the row's non-null
+    /// values, or feed their aggregator.
+    fn update_non_null_fields(
+        &self,
+        selected_by_col: &mut [Option<(usize, usize)>],
+        mut aggregators: Option<&mut [Option<Box<dyn FieldAggregator>>]>,
+        row: &MergeRow,
+        batch_buffer: &[BufferedBatch],
+        source_output_col_indices: &[usize],
+    ) -> crate::Result<()> {
+        for (output_col_idx, selected) in 
selected_by_col.iter_mut().enumerate() {
+            if self.grouped_fields.contains(&output_col_idx) {
+                continue;
+            }
+            let source_array = batch_buffer[row.batch_idx]
+                .column_for_output(output_col_idx, source_output_col_indices);
+            if let Some(aggregator) = aggregators
+                .as_mut()
+                .and_then(|aggregators| aggregators.get_mut(output_col_idx))
+                .and_then(Option::as_mut)
+            {
+                aggregator.agg_array(

Review Comment:
   Java's `updateNonNullFields` doesn't call aggregators, but without sequence 
groups `validated_aggregate_functions` only allows `last_non_null_value`, so 
`agg_array` keeps the same value Java would.



##########
crates/paimon/src/table/compact/primary_key.rs:
##########
@@ -193,19 +193,24 @@ impl MergeFunction for CompactReducer {
             });
         }
         Ok(match self.inner.merge(rows, batches, columns, schema)? {
-            MergeResult::MaterializedDeleteRow(_) if self.reject_delete_state 
&& rows.len() > 1 => {
+            MergeResult::MaterializedRetractRow(_, RowKind::Delete)

Review Comment:
   Rejecting UPDATE_BEFORE states here too makes 
`partial_update_compaction_keeps_update_before_state` fail, so the test covers 
this change.



##########
crates/paimon/src/table/sort_merge.rs:
##########
@@ -1024,8 +1070,10 @@ impl PartialUpdateMergeFunction {
         })?;
 
         if deleted {
-            Ok(PickedMerge::Result(MergeResult::MaterializedDeleteRow(
+            debug_assert!(result_kind.is_retract());
+            Ok(PickedMerge::Result(MergeResult::MaterializedRetractRow(

Review Comment:
   The picked read path from #1420 only takes live rows (`pick && !deleted`), 
so this result still reaches the reader as `MaterializedRetractRow` and is 
omitted, as before.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to