meteorgan commented on code in PR #7130: URL: https://github.com/apache/opendal/pull/7130#discussion_r2659772456
########## core/core/src/docs/rfcs/7130_route_layer.md: ########## @@ -0,0 +1,214 @@ +- Proposal Name: `route_layer` +- Start Date: 2026-01-04 +- RFC PR: [apache/opendal#7130](https://github.com/apache/opendal/pull/7130) +- Tracking Issue: [apache/opendal#7131](https://github.com/apache/opendal/issues/7131) + +# Summary + +Introduce `RouteLayer`, a layer that dispatches each OpenDAL operation to one of multiple pre-built `Operator` stacks by matching the operation path against user-provided glob patterns. + +# Motivation + +In practice, OpenDAL users often want different policies for different parts of a namespace: + +- Apply caching only for frequently-read objects (e.g., `**/*.parquet`). +- Use a tighter timeout for latency-sensitive paths (e.g., `hot/**`). +- Attach different observability or throttling configurations per dataset. + +Today, users can achieve this by building multiple `Operator`s and routing requests in application code. However, this approach duplicates routing logic across projects and makes it harder to share a consistent, well-tested routing behavior across bindings and integrations. + +`RouteLayer` centralizes this routing as a reusable primitive while preserving OpenDAL's existing composition model: users still build independent `Operator` stacks for each policy, and `RouteLayer` only decides which stack should handle a request. + +# Guide-level explanation + +## Enable feature + +```toml +opendal = { version = "*", features = ["layers-route"] } +``` + +## Build per-route operators + +Build each routed `Operator` as usual, including its own service configuration and layer stack. + +```rust +use std::time::Duration; + +use opendal::layers::RouteLayer; +use opendal::services; +use opendal::Operator; +use opendal::Result; +use opendal_layer_timeout::TimeoutLayer; + +fn build_default() -> Result<Operator> { + Operator::new(services::Memory::default())?.finish() +} + +fn build_parquet_fast_path() -> Result<Operator> { + Operator::new(services::Memory::default())? + .layer(TimeoutLayer::default().with_timeout(Duration::from_secs(3))) + .finish() +} +``` + +## Combine them with `RouteLayer` + +`RouteLayer` is applied to the default operator. When none of the patterns match, the request is handled by the default operator. + +Patterns are evaluated in insertion order. The first matching pattern wins. + +```rust +use opendal::Operator; +use opendal::Result; + +fn build_routed_operator() -> Result<Operator> { + let default_op = build_default()?; + let parquet_op = build_parquet_fast_path()?; + + let routed = default_op.layer( + RouteLayer::builder() + .route("**/*.parquet", parquet_op) + .build()?, + ); + + Ok(routed) +} +``` + +## Pattern rules + +`RouteLayer` uses glob patterns (via `globset`) and does not perform any implicit expansion. + +- `*.parquet` matches `file.parquet` in the root, but does not match `dir/file.parquet`. +- `**/*.parquet` matches `dir/file.parquet` and any other depth. + +Paths are matched against OpenDAL normalized paths as seen by the accessor: + +- The root is represented as `/`. +- Non-root paths do not start with `/` (e.g., `dir/file`). +- Directory paths end with `/` (e.g., `dir/`). + +## Routing for `copy` and `rename` + +For `copy(from, to, ..)` and `rename(from, to, ..)`, routing is decided only by `from`. The `to` path is forwarded unchanged to the selected operator. + +# Reference-level explanation + +## Public API + +`RouteLayer` is constructed via a fallible builder, because glob compilation can fail and `Layer::layer()` cannot return `Result`. + +```rust +pub struct RouteLayer { /* compiled router */ } + +impl RouteLayer { + pub fn builder() -> RouteLayerBuilder; +} + +pub struct RouteLayerBuilder { /* patterns + operators */ } + +impl RouteLayerBuilder { + pub fn route(self, pattern: impl AsRef<str>, op: Operator) -> Self; + pub fn build(self) -> Result<RouteLayer>; +} +``` + +`RouteLayer` is intended to be applied to a fully-built `Operator` (dynamic dispatch). Routed operators are also fully-built and independent. + +## Internal structure + +`RouteLayer` holds: + +- `glob: globset::GlobSet`, compiled from patterns in insertion order. +- `targets: Vec<Accessor>`, where `targets[i]` is the accessor corresponding to the `i`-th inserted pattern. + +At `build()` time: + +1. Compile each pattern into a `globset::Glob`. +2. Insert each glob into a `globset::GlobSetBuilder` in insertion order. +3. Convert each routed `Operator` into its `Accessor` and store it in `targets` in the same order. + +Any glob compilation error results in `build()` returning an `ErrorKind::InvalidInput` error with the pattern and the underlying source error attached. + +## Dispatch algorithm (Scheme B) Review Comment: What does "Scheme B" mean here ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
