PyRSA commented on code in PR #11031:
URL: https://github.com/apache/paimon/pull/11031#discussion_r4651860132


##########
paimon-python/pypaimon/read/reader/iface/record_batch_reader.py:
##########
@@ -77,6 +77,40 @@ def _polars_arrow_table(batch):
     return Table.from_batches([batch]) if schema == batch.schema else 
Table.from_arrays(arrays, schema=schema)
 
 
+def _iter_arrow_table_rows(table: Table) -> Iterator[tuple]:
+    """Yield row tuples from an Arrow table without Polars ``iter_rows``.
+
+    Polars converts ``Datetime(time_unit='ns')`` values to microsecond
+    ``datetime.datetime``, collapsing distinct TIMESTAMP(9) /
+    TIMESTAMP_LTZ(9) tails. Prefer Arrow ``Table.to_pylist()`` when
+    present so field order and nested conversions stay unchanged.
+    Older PyArrow builds without table-level ``to_pylist`` fall back to
+    per-column ``to_pylist`` over ``table.to_batches()`` — never to
+    Polars row iteration.
+    """
+    to_pylist = getattr(table, "to_pylist", None)
+    if callable(to_pylist):
+        names = table.column_names
+        for row in to_pylist():

Review Comment:
   Fixed the Arrow materialization path to preserve the physical nanosecond 
value instead of converting INT64_MIN to NaT. Added regression coverage for 
both top-level timestamps and nested MAP values to verify the value is 
preserved in the final Arrow output.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to