shyjsarah opened a new pull request, #749:
URL: https://github.com/apache/paimon-rust/pull/749
## Summary
Add native, read-only Object Table support to Paimon Rust and the DataFusion
integration. The implementation follows Paimon Java's fixed Object Table schema
and REST path semantics, so Object Tables no longer require an externally
registered table engine for reads.
## Changes
- Add `ObjectTable` and `ObjectEntry` with the Java-compatible fixed schema:
`path`, `name`, `length`, `mtime`, `atime`, and `owner`.
- Recursively list object metadata with relative paths and deterministic
ordering.
- Route Object Tables natively from filesystem and REST catalogs.
- Build filesystem I/O from the Object Table's own path, including
cross-scheme locations.
- Use `GetTableResponse.path` for REST Object Tables, matching Paimon Java
and preserving REST data-token handling.
- Add a DataFusion table provider with projection and limit support.
- Keep Object Tables read-only and reject branches, time travel, and writes.
- Keep non-native table types on the existing external table-engine path.
## Testing
- [x] `cargo clippy --locked -p paimon -p paimon-datafusion --all-targets --
-D warnings`
- [x] `cargo test -p paimon-datafusion --test object_table`
- [x] `cargo test -p paimon-datafusion --test table_type_routing`
- [x] `cargo test -p paimon --lib catalog`
- [x] `cargo test -p paimon --test rest_catalog_test
test_load_table_constructs_native_object_table -- --exact`
- [x] `cargo test -p paimon --test rest_catalog_test test_catalog_get_table
-- --exact`
- [x] `cargo check -p paimon-rest-server --all-targets`
- [x] `cargo fmt --all -- --check`
- [x] `git diff --check`
## Notes
- OpenDAL does not expose access time or owner portably, so `atime` is
currently `0` and `owner` is `NULL`; modification time and length are populated
when available.
- Public API impact: adds `LoadedTable::Object`, `ObjectTable`, and
`ObjectEntry`. Existing Paimon and external-engine variants keep their behavior.
- This is complementary to #747, which keeps metadata queries available when
unsupported external engines are not registered; neither PR depends on the
other.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
