JingsongLi opened a new pull request, #10639: URL: https://github.com/apache/paimon/pull/10639
### Purpose `TableRead.to_iterator()` currently always opens Python split readers, even when `read.native.enabled=true` and the same read already works through the Native Arrow APIs. Route this existing row API through the Native reader while preserving its `InternalRow` contract. ### Changes - Lazily consume Native batches one split at a time, retaining input split order and passing the remaining LIMIT to each subsequent reader before it opens BLOB payloads. - Reuse the existing MAP/ROW conversion and row view, preserving NULL containers, nested/map-key projection, row kind, schema evolution, and typed BLOB/VECTOR accessors. - Keep the Native row-kind transport separate from legal user columns named `_row_kind`; align duplicate output columns by position when adding Arrow metadata. - Keep setup fallback, captured query authorization and early-close/error cleanup. Errors after reading starts propagate without replaying Python reads. - Default ordinary Python rows to INSERT, matching Java `GenericRow` and Native snapshot reads. - Add 109 row-iterator cases covering append, primary-key and Data Evolution reads, actual changelog kinds, partial updates/deletion vectors, BLOB and vector sidecars, MAP<BLOB> leaves, authorization, limits and lifecycle failures. This uses the existing Rust `main` bindings and adds no Rust public API or new dependency. ### Validation - 24 relevant read/update/schema-evolution/authorization test files: **1184 passed, 4 skipped**, plus 32 subtests passed. - Final `native_iterator_test.py`, including the additional MAP<BLOB> leaf regressions: **109 passed**. - Changed Python files pass Flake8; `git diff --check` passes. - Native tests used the existing installed extension; it was not rebuilt or installed while testing. - Two focused review rounds covered resource ownership, authorization, projection and row/type semantics; identified issues were fixed and verified. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
