YannByron opened a new pull request, #9547: URL: https://github.com/apache/paimon/pull/9547
### Purpose Creating a `FormatBlobReader` currently decompresses the same immutable BLOB index and rebuilds all offsets every time. Workloads that repeatedly open the same BLOB files pay this CPU cost on each reader creation. Cache decoded indexes by their exact compressed bytes with a bounded 16-entry LRU. Cached values remain immutable, while every reader receives its own mutable lists. This change is extracted from #9466 so the generic reader optimization can be reviewed independently. A local paired ACT benchmark on an Apple M2 Pro showed Paimon batch-fetch throughput improving from 148.70 to 172.56 samples/s (+16.05%). With the same HDF5 baseline, the Paimon/HDF5 ratio improved from 87.88% to 101.98%; tensor fingerprints and three rounds of train/validation loss remained identical. ### Tests - `PYTHONPATH=. python -m pytest pypaimon/tests/blob_test.py -q` (`118 passed, 1 skipped`) - `ruff check pypaimon/read/reader/format_blob_reader.py pypaimon/tests/blob_test.py` - `flake8 --config=dev/cfg.ini pypaimon/read/reader/format_blob_reader.py pypaimon/tests/blob_test.py` - `git diff --check origin/master..HEAD` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
