JingsongLi opened a new pull request, #970: URL: https://github.com/apache/paimon-rust/pull/970
### Purpose Close the remaining core and binding gaps exposed by enabling PyPaimon native Parquet writers for data-evolution tables, merge engines, nested input and primary-key value statistics. ### Brief change log - Delegate Python batch validation and normalization to `TableWrite`, allowing compatible nested child names and fixed-size binary input. Preserve schema-error recovery for the shared format-table writer path. - Apply the core row-kind filter to both generated and explicit `_VALUE_KIND` input before bucketing and changelog writing. Accept global `ignore-delete` and Java fallback aliases for aggregation, and allow `rowkind.field` with supported merge engines. - Reuse `FirstRowMergeFunction` for multi-row key groups during flush so every retract is checked. Preserve Java reducer behavior for singleton groups and the insert-only fast path. - Honor `parquet.write-page-index.enabled` for both data and changelog files, retaining row-group statistics when page indexes are disabled and rejecting invalid settings before output creation. - Parse bare `VARCHAR`, `BINARY` and `VARBINARY` names with Java's default length of 1, including nested Variant shredding schemas. ### Tests - Rust targeted tests: KV writer (27), aggregation options (21), row-kind schema validation (7), row-kind integration (21), table writer (69), types (22), Variant shredding (7), Parquet writer and physical data/changelog page-index regression. - Python binding write tests: 29 passed, including normalization, explicit row kinds and schema-error recovery. - `cargo fmt --all --check`. - `cargo clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings`. - PyPaimon integration results will be recorded in the companion PR. ### API and Format Python write signatures remain unchanged; normalization and filtering live in Rust core. Adds a `CoreOptions` accessor for the existing Parquet option. Storage remains standard Paimon/Parquet. ### Documentation No new user-facing option: this implements existing Java/PyPaimon contracts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
