JingsongLi opened a new pull request, #1202: URL: https://github.com/apache/paimon-rust/pull/1202
## Purpose Allow distributed Python execution to reuse Core plans and conflict semantics without exposing internal file indexes or re-planning in every worker. Rust Core remains independent of Ray. ## Changes - Add typed `Plan::from_splits` and `RowIdPlan` routing metadata for complete data-evolution file groups. `TableUpdate::with_plan` reuses a captured snapshot for row-ID updates and deletes. Other operations explicitly reject that configuration before consuming inputs. - Reauthorize mutation workers, validate snapshot/table paths and complete groups, and reject deleted row IDs before producing any update file. Preserve captured deletion vectors and conflict baselines. - Expose snapshot-bound `GlobalIndexBuildPlan` / `GlobalIndexBuildShard` through `GlobalIndexBuildBuilder::plan` and `build_shard`, covering the vindex types already supported by Python Ray builds. Each worker executes one captured shard with its own table/I/O environment; source metadata still protects final publication. - Add structured `RowIdExistenceConflict` details in Core and Python bindings. Align stale DELETE metadata checks with current Java `DataEvolutionConflictDetection`. - Add an independent `TableCommit::check_row_id_conflicts` and batch binding for history checks before a rewrite. Logical DML conflicts remain errors; historical compactions are ignored as in Java. Checking does not publish, abort, or consume the batch commit boundary. - Add bindings and type stubs for the new typed interfaces. No new dependencies. ## Verification - Rust 1.98.0: row-ID tests (79), captured-plan misuse test (1), vindex tests (32). - Actual PyO3 harness: error tests (10), planned input rejection (1), independent conflict check followed by a real commit (1). - Core and Python binding Clippy with `--all-targets -- -D warnings`. - `cargo fmt --all -- --check`. Regression cases include fixed snapshots after later appends, manifest-free worker execution, fresh authorization and masked/restricted grants, complete groups and DV checks, invalid/foreign plans, shard coverage and incremental builds, exception serialization, and preserving ordinary/unknown publication errors. The paired Apache Paimon Python PR will wire Ray scheduling to these APIs. Python's existing public API is unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
