JingsongLi opened a new pull request, #846: URL: https://github.com/apache/paimon-rust/pull/846
### Purpose Closes #712. Allow embedding applications such as Milvus to reuse their shared OpenDAL backends through the existing FileIOProvider abstraction. Providers resolve original URIs to an operator and a relative object path, while continuing to own credential refresh. ### Brief change log - Make FileIOProvider public and expose with_provider on FileIOBuilder and FileIO. - Defer provider-backed input/output resolution to async operations, bypassing built-in storage construction and propagating provider errors without fallback. - Validate directory path mappings and return reusable full URIs from both listing APIs, including bucket roots and operators rooted below a bucket. - Reject object paths that OpenDAL would normalize and rename operations spanning different shared service instances. - Add regression coverage for literal object keys, multi-bucket routing, error propagation, open-handle lifetime, cache isolation/invalidation, and existing filesystem behavior. ### Tests - Reproduced the old synchronous filesystem-resolution failure before implementing the change. - `cargo test --locked --offline -p paimon --all-targets --features fulltext,vortex` — 3149 passed, 2 ignored. - `cargo test --locked --offline -p paimon --lib io:: --features storage-all,fulltext,vortex` — 175 passed, including existing REST token-refresh and OSS retry tests. - `cargo clippy --locked --offline --all-targets --workspace --features fulltext,vortex -- -D warnings` — passed. - `cargo check --locked --offline -p paimon --lib --no-default-features` — passed, with two existing unused import/variable warnings in storage.rs. - `cargo fmt --all -- --check` and `git diff --check` — passed. ### API and Format Adds public Rust provider injection APIs; no storage-format change. Provider-backed constructors remain synchronous and report resolution errors during subsequent async IO. Listing requires the resolved relative path to be a component suffix of the original URI. Existing readers/writers retain their operator, so credential refresh for open handles must happen inside the backend. Property-based storage configuration remains available. ### Documentation Adds a shared-backend integration example and documents URI mapping, credential lifetime, cache identity, and rename requirements. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
