JingsongLi commented on code in PR #10003:
URL: https://github.com/apache/paimon/pull/10003#discussion_r4056533969
##########
paimon-python/pypaimon/read/table_read.py:
##########
@@ -219,7 +219,7 @@ def to_arrow_batch_reader(
table reads do not guarantee row order. Python fallback reads remain
serial.
"""
- effective = self._resolve_parallelism(parallelism, len(splits))
+ effective = self._effective_parallelism(parallelism, len(splits))
Review Comment:
Fixed in 66ce56469b. The actual native fan-out still uses the LIMIT-capped
effective parallelism, while the public and managed wrappers now use the
uncapped resolved parallelism solely to retain iterator cleanup ownership.
Added a regression with two splits, parallelism 2, limit 1, one read_next_batch
call, and public close; it verifies that the sole native reader is closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
