linliu-code opened a new pull request, #649: URL: https://github.com/apache/hudi-rs/pull/649
**Stacked on #639–#648** — their commits appear here until they merge. **Review only the last commit.** The merge map the reader holds records in while merging a file slice. It tracks its own size and spills to RocksDB once past the configured budget instead of growing without bound; a separate peak ceiling fails the read loudly rather than letting it consume the host. ## This PR adds rocksdb as a required dependency — the thing to scrutinize That is a real cost: it bundles RocksDB 8.10 and needs **libclang** to build (bindgen), which affects CI images. The alternative was an in-memory-only tier behind a feature flag. That would have meant writing a spill backend that does not exist upstream — new design work rather than a port — and a merge map that silently cannot spill is worse than one that needs a build dependency. Flagging it explicitly because it is the kind of decision that should be made deliberately, not inherited. Licenses are compatible: `rocksdb` Apache-2.0, `librocksdb-sys` MIT/Apache-2.0/BSD-3-Clause. Also adds `foldhash` (map hasher), `uuid` (collision-free spill dir names), `tempfile` (their lifecycle), and `CoreError::MemoryLimitExceeded` for the peak ceiling. Build warning-free; 923 lib tests green. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
