JingsongLi opened a new pull request, #125: URL: https://github.com/apache/paimon-vector-index/pull/125
Follow-up to #123. Oversized IVF-RQ lists changed batch scans from parallel query execution to a single worker. The release preflight could also mistake a version example inside multiline TOML text for the actual Python project version. ## Changes - Share each streamed IVF-RQ chunk across active query workers for top-K and range search. Keep heaps and statistics exclusive to each query, preserve error propagation, and avoid payload copies or duplicate reads. - Parse the actual TOML project version with `tomllib`/`tomli`, including comments and quoted values, rather than matching text with a regex. Install `tomli` conditionally for the Python 3.9 CI checks. - Add regressions for multiline README version examples, valid TOML quoting/comments, and parallel streaming scans. Use condition-variable coordination to make the worker check deterministic. ## Validation - `cargo test --workspace --no-fail-fast`: 738 passed, 2 existing ignored. - Release/tool tests on real Python 3.9 and Python 3.13: 28 passed on each. - Clippy with warnings denied, Rust formatting, license headers, and binary legal inventory passed. - A synthetic partition above 64 MiB now uses four Rayon workers while sharing the same bounded reads. The new branch is tree-identical to the locally tested fixes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
