JingsongLi opened a new pull request, #10490: URL: https://github.com/apache/paimon/pull/10490
## Dependency Depends on apache/paimon-rust#1094. Keep this PR in draft until that change is merged. Native CI continues building Apache paimon-rust from `main`. ## What changes The Python local full-text/vector builder already supports `global-index.build.parallelism`. The accompanying Rust change enables that concurrency in native builds, retaining independent shard writers, logical ranges, relative row IDs and private preparation ownership. - Resolve effective parallelism through the existing Python integer ConfigOption conversion before passing it to Rust. This preserves typed inputs, build-over-table precedence and explicit default overrides without Python row processing. - Include the option name in conversion errors, before any index output is created. - Document native shard concurrency and commit-message file ownership. - Add REST tests for all six generic index families: serial/parallel output comparisons, plan order, sparse and empty shards, incremental coverage, partition filtering, snapshot responses, typed options, invalid controls, internal/external private cleanup, and file retention after source-conflict commits. The core performs scheduling and index construction. There is no Python retry after a native build failure. ## Verification - REST/native index regression suites: 253 passed, 85 actual native plans. - New REST tests plus native integration gates with every Native CI switch enabled: 94 passed; 45 plans, 15 reads, 10 writes, 55 commits, and every tracked native update operation exercised. - Existing Python global-index build suite: 38 passed, 19 subtests passed. - Flake8 and Python 3.6 syntax checks for changed Python files: passed. - Tests used an editable extension built from the paired Rust branch. No CI source pin or dependency was changed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
