PyRSA commented on code in PR #11031: URL: https://github.com/apache/paimon/pull/11031#discussion_r4651851392
########## docs/docs/primary-key-table/merge-engine/aggregation.mdx: ########## @@ -195,19 +195,13 @@ count of distinct event IDs. ### merge_map_with_keytime The merge_map_with_keytime function merges input maps with key-level partial updates based on timestamps. - Each key in the map carries a timestamp. During merging, the entry with the greater - timestamp is retained. The timestamp field supports: - - - `CHAR`, `VARCHAR`, or `STRING`: compared lexicographically by UTF-8 bytes. Use equally padded - numeric strings or a consistent sortable date/time format. This differs from the previous Java - UTF-16 string ordering for some supplementary Unicode characters; ASCII timestamps are unaffected. - - `TINYINT`, `SMALLINT`, `INT`, or `BIGINT`: compared numerically without conversion to floating point. - - `DATE`: compared by date. - - `TIME`: compared by time of day, without a date component. - - `TIMESTAMP` or `TIMESTAMP_LTZ`: compared at the declared precision, including nanoseconds. - Spark's `TIMESTAMP_NTZ` maps to Paimon's `TIMESTAMP` and is also supported. - + Each key in the map carries a timestamp. During merging, the entry with the greater timestamp is retained. It only supports `MAP<key_type, ROW<value_field, ts_field>>` type, where the ROW must have at least 2 fields. + The timestamp field supports `CHAR`, `VARCHAR`, `TINYINT`, `SMALLINT`, `INTEGER`, `BIGINT`, `DATE`, `TIME`, + `DECIMAL`, `TIMESTAMP` and `TIMESTAMP_LTZ`. + Spark's `TIMESTAMP_NTZ` maps to Paimon's `TIMESTAMP` and is also supported. + `CHAR` / `VARCHAR` timestamps use UTF-16 code unit lexicographical ordering: Java Review Comment: Thanks for pointing this issue out. This inconsistency stems from the comparison semantics for `STRING` type keys within the Rust-based SQL execution path; it is independent of the Python-based table read/write implementation addressed in this PR. A comprehensive fix would require modifying the underlying native execution path, which is outside the scope of this PR. I would prefer to address this in a separate PR rather than further expanding the scope of the current one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
