JingsongLi opened a new pull request, #7: URL: https://github.com/apache/paimon-full-text/pull/7
## Summary Prepare the standalone Paimon full-text index for a 0.1 release by stabilizing the storage envelope, simplifying the public query API to JSON strings, and completing seek-on-demand read support across Rust, C, Java, and Python. ## Changes - Stabilize the v1 storage envelope with metadata validation, Tantivy version checks, a 16 MiB header limit, and a golden file for the binary envelope layout. - Replace eager in-memory Tantivy directory loading with a seek-on-demand archive directory, reader reuse, read metrics, prewarm support, and a small-block read cache. - Expand indexing/search support for multi-field documents, repeated named fields, fuzzy/phrase/boolean/multi-match/boost-demotion queries, tokenizer options, and 64-bit Roaring filters. - Simplify public search APIs to accept query JSON strings directly and remove Java/Python `FullTextQuery` helper classes. - Align C, JNI, Java, and Python I/O around single positional `pread` callbacks that must be safe for concurrent calls. - Remove stale design/plan documents and update current README/API/storage docs. ## Testing - [x] `cargo clippy --workspace --all-targets -- -D warnings` - [x] `cargo test --workspace` - [x] `cargo build -p paimon-ftindex-ffi -p paimon-ftindex-jni` - [x] `PYTHONPATH=python PAIMON_FTINDEX_LIB_PATH=/Users/lijingsong/IdeaProjects/paimon-full-text/target/debug/libpaimon_ftindex_ffi.dylib python3 -m pytest -q python/tests` - [x] `PAIMON_FTINDEX_JNI_LIB_PATH=/Users/lijingsong/IdeaProjects/paimon-full-text/target/debug/libpaimon_ftindex_jni.dylib mvn -q -f java/pom.xml test` ## Notes Breaking API cleanup before 0.1: Java and Python query builder classes were removed; callers now pass the query DSL as a JSON string to `search`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
