JingsongLi opened a new pull request, #8283:
URL: https://github.com/apache/paimon/pull/8283
## Summary
This PR generalizes the BTree global-index build path into a sorted-index
build path and reuses it for Bitmap global indexes. It also exposes Bitmap
global-index creation through Flink and Spark procedures and updates the
related documentation.
## Changes
- Rename the BTree-specific global index builder/topology classes to
sorted-index builders.
- Route both `btree` and `bitmap` through the sorted build topology for
Flink and Spark.
- Add shared sorted build options:
- `sorted-index.records-per-range`
- `sorted-index.build.max-parallelism`
- Keep legacy `btree-index.records-per-range` and
`btree-index.build.max-parallelism` as fallback keys.
- Add Flink and Spark coverage for Bitmap global-index procedure creation.
- Update global-index docs and procedure docs to include Bitmap and the
shared sorted build options.
## Testing
- `mvn -pl paimon-common -Pfast-build
-Dtest=SortedIndexOptionsTest,BTreeIndexOptionsTest test`
- `mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false
-Dtest=SortedGlobalIndexBuilderSplitTest test`
- `mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false
-Dtest=SortedGlobalIndexBuilderTest test`
- `mvn -pl paimon-flink/paimon-flink-common -am -Pfast-build
-DfailIfNoTests=false
-Dtest=SortedIndexTopoBuilderTest,CreateGlobalIndexProcedureTest test`
- `mvn -pl paimon-flink/paimon-flink-common -am -Pfast-build
-DfailIfNoTests=false -Dtest=CreateGlobalIndexProcedureTest test`
- `mvn -pl paimon-flink/paimon-flink-common -am -Pfast-build
-DfailIfNoTests=false -Dtest=SortedGlobalIndexITCase#testBitmapIndex test`
- `mvn -pl paimon-flink/paimon-flink-common -am -Pfast-build
-DfailIfNoTests=false
-Dtest=DropGlobalIndexProcedureITCase#testDropBtreeGlobalIndex test`
- `mvn -pl paimon-spark/paimon-spark-common -am -Pfast-build
-DfailIfNoTests=false -Dtest=CreateGlobalIndexProcedureTest test`
- `git diff --check origin/master..HEAD`
## Notes
The Spark UT Scala suite command was started but interrupted while
downloading a large snapshot dependency, so it did not report a code failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
