JingsongLi opened a new pull request, #8398: URL: https://github.com/apache/paimon/pull/8398
## Summary Builds on #8393 to add PyPaimon support for building paimon-vindex global vector indexes from Python, with behavior aligned to the Java default global index builder. Also updates the global index docs with SQL/Python SDK TabItem examples for BTree and vector index lifecycle APIs. ## Changes - Add PyPaimon vindex global index build for `ivf-flat`, `ivf-pq`, `ivf-hnsw-flat`, and `ivf-hnsw-sq`. - Align Python vindex build with Java behavior for row-count sharding, relative row ids, non-indexable schema boundaries, files without row ids, unaware-bucket validation, and deletion-vector rejection. - Add a native paimon-vindex writer that spills row ids/vectors to temporary files, trains/adds in batches, writes index files, and cleans temporary files on close/failure. - Document BTree and vector global index build/drop/query usage with SQL and Python SDK tabs. ## Testing - `pytest -q paimon-python/pypaimon/tests/global_index_build_test.py paimon-python/pypaimon/tests/vindex_vector_index_test.py paimon-python/pypaimon/tests/index_manifest_write_test.py paimon-python/pypaimon/tests/file_store_commit_test.py` - `pytest -q paimon-python/pypaimon/tests/global_index_update_action_test.py paimon-python/pypaimon/tests/global_index_test.py paimon-python/pypaimon/tests/external_paths_test.py` - `pytest -q paimon-python/pypaimon/tests/vector_search_filter_test.py` - `python -m compileall -q paimon-python/pypaimon/globalindex/create_global_index.py paimon-python/pypaimon/globalindex/vindex/vindex_vector_index_writer.py paimon-python/pypaimon/tests/global_index_build_test.py` - `git diff --check` - `yarn build` in `docs` (passes; reports existing `/docs/master/concepts/overview` broken-link warnings) ## Notes PyPaimon vindex build requires `paimon-vindex` or `pypaimon[vindex]` for the native writer. Lumina index build remains documented as SQL-only for now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
