jordepic opened a new issue, #616: URL: https://github.com/apache/paimon-rust/issues/616
### Motivation `Storage::build` resolves the backing opendal operator through a closed match on URI schemes, so an embedding engine cannot supply its own operator — for example one wrapped with a custom service, a metrics/caching layer, or a preconfigured object-store client. Java Paimon treats configurable file IO as a first-class concern: `FileIO` is an interface, `FileIOLoader` is a ServiceLoader SPI matched by scheme, and `CatalogContext` accepts `preferIO`/`fallbackIO` loaders programmatically. paimon-rust has no equivalent injection point. ### Proposal The minimal hook: `FileIOBuilder::with_operator(op)` accepting a prebuilt `opendal::Operator` that takes precedence over scheme resolution. A scheme-keyed loader registry mirroring Java's `FileIOLoader` could follow as a larger change, but the builder hook alone unblocks embedders. ### Use case StreamFusion (github.com/datafusion-contrib/StreamFusion) stores Flink operator state in local Paimon PK tables; its write path needs a thin custom fs service (the stock service's per-write `create_dir_all` dominated a checkpoint-path CPU profile). It currently carries this hook as a fork patch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
