JingsongLi opened a new pull request, #123:
URL: https://github.com/apache/paimon-vector-index/pull/123

   Large IVF-RQ partitions could bypass the 64 MiB batch limit, malformed 
headers could drive large allocations, and JNI output was limited by a single 
Java byte array. Release jobs also lacked a shared source-version check and 
runtime validation of their final wheels.
   
   ## Changes
   
   - Stream oversized IVF-RQ lists in aligned scan blocks, including a bounded 
delta-ID window. Preserve top-K, range, filtered, batch, and continued-probe 
results while accounting for the ID window in the allocation budget.
   - Validate header-derived PQ/RQ memory before construction, use fallible 
allocations, defer rotation creation, and commit loaded metadata only after 
complete success so retries cannot retain partial arrays.
   - Write JNI output in reusable 1 MiB chunks, update position after each 
successful chunk, and preserve the original Java exception.
   - Gate the release orchestrator and independent language workflows on tag, 
Cargo workspace/path-dependency, Python, and Maven version consistency. RC tags 
use the existing clean source-version policy.
   - Install each final repaired wheel in an isolated environment and exercise 
native loading, training, writing, reading, and searching before artifact 
upload and publication.
   
   ## Validation
   
   - `cargo test --workspace --no-fail-fast`: 737 passed, 2 existing ignored.
   - `python3 -m unittest discover -s tools/tests -p 'test_*.py' -q`: 26 
passed, including final/RC tags, mismatches, and non-tag inputs.
   - `cargo clippy --all-targets --workspace -- -D warnings`, formatting, 
license-header, and binary legal-inventory checks passed.
   - Java API tests and native chunk/boundary tests passed, including 
`-Xcheck:jni`, buffer reuse, byte order, and original exception identity.
   - A locally built and delocate-repaired macOS arm64 wheel passed isolated 
installation and the end-to-end smoke test.
   
   Linux x86-64/aarch64 and Windows final-wheel runtime validation is 
configured as a required step in the release workflows; it still needs the 
corresponding CI runners.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to