YannByron opened a new pull request, #11192: URL: https://github.com/apache/paimon/pull/11192
### Purpose Python BLOB index reads use `seek` + `read` for the footer and index. On the OSS backend this can trigger sequential read-ahead for a small metadata range. The SDK already supports positional reads, but Arrow's `PythonFile` / `PyFileSystem` wrappers hide that implementation from Python FileIO callers. Preserve the SDK `pread` path for Python `read_at` calls and use bounded positional reads for the BLOB footer and index. Streams without positional-read support retain the seek/read fallback; SDKs without `pread` retain Arrow's existing implementation. File-descriptor reads also check that `os.pread` is available, keeping the fallback usable on Windows. The wrapper remains an Arrow NativeFile, keeps its filesystem owner alive until close, and preserves the stream cursor when the SDK supports positional reads. Invalid footer lengths are rejected before issuing an out-of-bounds read. This does not change the default small-read policy or the BLOB format. ### Tests - Python execution lane on Python 3.12.14 / PyArrow 19.0.1: **403 passed, 20 skipped** across the BLOB, deferred resolver, index cache, FileIO, caching/resolving FileIO, OSS wrapper and format-writer test modules. Skips require the real SDK/service environment or native Windows paths. - Two new regressions fail on the upstream implementation: exact footer/index positional reads and preservation of the stream cursor through FileIO. - Added coverage for the unavailable `os.pread` fallback; verified it fails without the capability check and passes with it. - Regressions cover an SDK without `pread`, Arrow Parquet consumers, stream/owner cleanup, malformed indexes and read failures. - Flake8 with the repository configuration and `git diff --check` pass. The request-level tests verify the I/O path; this PR makes no standalone end-to-end speedup claim. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
