TheR1sing3un opened a new pull request, #10426: URL: https://github.com/apache/paimon/pull/10426
### Purpose Allow `create_global_index` and multimodal `create_index` to build native vector-index shards with `execution="ray"`. The driver plans uncovered row ranges from one snapshot, workers construct shards with bounded concurrency, and the driver publishes all resulting entries in one commit. Local execution remains the default. Allocate output names before dispatch and disable automatic write-task retries. If a task fails, drain outstanding tasks before removing uncommitted shard files, including files whose commit messages were never returned. Document shared storage, dependency, snapshot-retention and driver-failure requirements. ### Tests 50 targeted tests passed with Ray 2.54.0 and paimon-vindex 0.5.0, covering distributed construction, incremental coverage, deleted rows, one-commit publication, build/commit separation, cleanup after a worker writes then fails, option validation, and existing global-index/multimodal APIs. Flake8 and `git diff --check` passed. No benchmark code included. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
