stark256-spec commented on PR #3454: URL: https://github.com/apache/iceberg-python/pull/3454#issuecomment-4682988585
Pushed in 769728d1 — implemented all of these (plus the reflected `+`): - `count()`, `index()`, `__reversed__()`, `.copy()`, and `+` / reflected `+` were all inherited straight from `list`'s C implementation, which reads the underlying array directly and skips `_fetch_all()`/`_fetch_through_index()`. On a multi-page result they'd silently only see whatever pages had been fetched so far. - Each is now overridden to fetch all remaining pages first, then delegate to `list`'s real implementation. `.copy()` and `+`/reflected `+` return a plain `list` once everything's fetched — nothing left to paginate at that point. - Added the reflected `+` (`__radd__`) too, since without it `[x] + pagination_list` dispatches straight to `list.__add__` at the C level and hits the same silent-truncation bug from the other side. - `__add__` needed an `@overload` to match `list.__add__`'s own overloaded signature (the `list[T] + list[S] -> list[S | T]` case) — otherwise mypy flagged it as an incompatible override. Added 13 new tests in `tests/utils/test_pagination.py`: for each method, a multi-page case that proves it now picks up items from a page that hadn't been fetched yet (e.g. `pl.count(x)` only finding the second match after page 2 is fetched, `[0] + pl` including page-2 items, `pl.index()` raising for a missing value), plus single-page sanity checks. Ran the full `tests/utils/` + `tests/catalog/test_rest.py` suite (338 passed) and `make lint` clean. Left the mutating side (`append`/`extend`/`sort`/`+=`) out of this — that's a different question (mutating while pages are still pending) and you didn't flag those, but happy to take a pass at it if you think it's worth covering too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
