PyRSA commented on code in PR #11031:
URL: https://github.com/apache/paimon/pull/11031#discussion_r4585827588


##########
paimon-python/pypaimon/execution/python/read/reader/aggregate/aggregators.py:
##########
@@ -1339,10 +1364,23 @@ def _merge_input_map(self, result_map: Dict[Any, Any], 
input_field: Any):
                 continue
 
             existing_ts = self._get_ts_field(existing_row)
-            if (existing_ts is None or new_ts > existing_ts
-                    or (self.overwrite_on_equal_ts and new_ts == existing_ts)):
+            if existing_ts is None or 
self._should_replace_by_timestamp(new_ts, existing_ts):
                 result_map[key] = new_row
 
+    def _should_replace_by_timestamp(self, new_ts: Any, existing_ts: Any) -> 
bool:
+        # CHAR/VARCHAR keep Java String.compareTo (UTF-16 code units).
+        # Other supported types compare by their native order, including
+        # TIMESTAMP(9) / TIMESTAMP_LTZ(9) values that still carry a
+        # nanosecond tail after Arrow row materialization. Equal timestamps
+        # replace only when overwrite-on-equal-ts is enabled.
+        if self._string_timestamp:
+            if _utf16_code_unit_greater(new_ts, existing_ts):

Review Comment:
   Thanks for pointing this out. Fixed in the latest commit.
   
   Native Writer is now bypassed for `merge_map_with_keytime` tables, and the 
Python fallback handles Arrow MAP entries materialized as `(key, value)` 
tuples. Added a regression test using U+E000 and U+10000 in the same 
`write_arrow` batch, verifying that the UTF-16 ordering is preserved after 
commit and reopen.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to