YannByron opened a new pull request, #11192:
URL: https://github.com/apache/paimon/pull/11192

   ### Purpose
   
   Python BLOB index reads use `seek` + `read` for the footer and index. On the 
OSS backend this can trigger sequential read-ahead for a small metadata range. 
The SDK already supports positional reads, but Arrow's `PythonFile` / 
`PyFileSystem` wrappers hide that implementation from Python FileIO callers.
   
   Preserve the SDK `pread` path for Python `read_at` calls and use bounded 
positional reads for the BLOB footer and index. Streams without positional-read 
support retain the seek/read fallback; SDKs without `pread` retain Arrow's 
existing implementation. File-descriptor reads also check that `os.pread` is 
available, keeping the fallback usable on Windows.
   
   The wrapper remains an Arrow NativeFile, keeps its filesystem owner alive 
until close, and preserves the stream cursor when the SDK supports positional 
reads. Invalid footer lengths are rejected before issuing an out-of-bounds 
read. This does not change the default small-read policy or the BLOB format.
   
   ### Tests
   
   - Python execution lane on Python 3.12.14 / PyArrow 19.0.1: **403 passed, 20 
skipped** across the BLOB, deferred resolver, index cache, FileIO, 
caching/resolving FileIO, OSS wrapper and format-writer test modules. Skips 
require the real SDK/service environment or native Windows paths.
   - Two new regressions fail on the upstream implementation: exact 
footer/index positional reads and preservation of the stream cursor through 
FileIO.
   - Added coverage for the unavailable `os.pread` fallback; verified it fails 
without the capability check and passes with it.
   - Regressions cover an SDK without `pread`, Arrow Parquet consumers, 
stream/owner cleanup, malformed indexes and read failures.
   - Flake8 with the repository configuration and `git diff --check` pass.
   
   The request-level tests verify the I/O path; this PR makes no standalone 
end-to-end speedup claim.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to