JunRuiLee opened a new pull request, #595: URL: https://github.com/apache/paimon-rust/pull/595
> **Depends on apache/paimon#8819** (the Java producer side of this contract). Draft until that PR's `TableDescriptor` JSON is finalized — the golden fixture here is byte-identical to the one produced there, so it must be updated in lockstep if the Java wire form changes. ### Purpose No dedicated issue yet — this is the Rust reader half of a cross-language contract; the Java producer half is apache/paimon#8819. Add a portable, versioned `TableDescriptor` and a catalog-free way to build a read-only `Table` from it, so a non-Java engine (e.g. Doris BE via the C FFI) can rebuild a table for scan/read **without a catalog lookup**. Today the only way to obtain a `Table` is `Catalog::get_table`, which re-resolves the table (risking schema/snapshot drift vs. what the planner used) and ties the reader to a filesystem-catalog layout. The descriptor carries only what a reader needs — `path` + the full `TableSchema` + optional `options`/`database`/`name`/`branch` — as language-neutral JSON. It is byte-compatible with the Java `TableDescriptorSerializer` output (apache/paimon#8819). ### Brief change log - `crates/paimon/src/table/descriptor.rs`: `TableDescriptor` (serde DTO) with `from_json` (rejects unknown `version` and non-`main` `branch`; wraps serde errors as `DataInvalid`), and `Table::from_descriptor_json` (top-level `path` wins over `schema.options["path"]`; identifier requires both `database`+`name` or neither; plain `FileIO`; `rest_env = None`; descriptor `options` are not merged into table semantics in v1). - `bindings/c/src/table.rs`: C FFI `paimon_table_from_descriptor(json, options, len)`, mirroring the existing `paimon_catalog_get_table` handle/free contract, with an ABI signature guard; freed via `paimon_table_free`. - Historical per-file schemas are still read from the immutable on-disk `schema/schema-N` by `SchemaManager`; the descriptor pins the planned/current schema only. ### Tests - `table::descriptor` unit tests (13): parse valid v1; reject unknown version / non-main / empty branch; missing-field & malformed JSON → `DataInvalid`; build table + path-option injection; identifier synthesis and partial-identifier rejection; descriptor options do not affect table semantics; and `parses_java_produced_golden_v1` — parses the **byte-identical Java-produced golden** and asserts the ARRAY field reconstructs to `DataType::Array`. - `paimon-c` FFI tests (3): build a table from a descriptor; reject unknown version; reject null JSON. ### API and Format - New public API: `paimon::table::TableDescriptor` + `Table::from_descriptor_json`, and C FFI `paimon_table_from_descriptor` (additive; existing symbols unchanged). - New JSON descriptor format, versioned (`version = 1`). No change to any existing on-disk or wire format. ### Documentation None yet (new, evolving cross-language contract); docs can follow once it stabilizes together with apache/paimon#8819. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
