This is an automated email from the ASF dual-hosted git repository.
xuanwo pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/opendal.git
The following commit(s) were added to refs/heads/main by this push:
new 910572d2c RFC-6828: OpenDAL Core (#6828)
910572d2c is described below
commit 910572d2c68161e095f1f092bd5efe8232a2e9e3
Author: Xuanwo <[email protected]>
AuthorDate: Thu Nov 27 22:54:52 2025 +0800
RFC-6828: OpenDAL Core (#6828)
* rfc: core
Signed-off-by: Xuanwo <[email protected]>
* Assign number
Signed-off-by: Xuanwo <[email protected]>
* Add issue
Signed-off-by: Xuanwo <[email protected]>
---------
Signed-off-by: Xuanwo <[email protected]>
---
core/src/docs/rfcs/6828_core.md | 108 ++++++++++++++++++++++++++++++++++++++++
core/src/docs/rfcs/mod.rs | 4 ++
2 files changed, 112 insertions(+)
diff --git a/core/src/docs/rfcs/6828_core.md b/core/src/docs/rfcs/6828_core.md
new file mode 100644
index 000000000..0276dd515
--- /dev/null
+++ b/core/src/docs/rfcs/6828_core.md
@@ -0,0 +1,108 @@
+- Proposal Name: core
+- Start Date: 2025-11-27
+- RFC PR: [apache/opendal#6828](https://github.com/apache/opendal/pull/6828)
+- Tracking Issue:
[apache/opendal#6829](https://github.com/apache/opendal/issues/6829)
+
+# Summary
+
+Introduce a crate topology where `opendal-core` owns types, raw traits, and a
minimal set of critical layers; each storage backend lives in its own
`opendal-service-{backend}` crate; non-critical layers move to
`opendal-layer-{name}` crates; and a facade crate `opendal` re-exports core,
services, and layers while wiring dynamic service registration and feature
aliases.
+
+# Motivation
+
+- Decouple compilation units so users only build the services and layers they
need, reducing compile time and dependency footprint.
+- Make service delivery and versioning independent, enabling faster backend
evolution without touching core.
+- Allow layer evolution at its own pace while keeping correctness-critical
layers close to core for safety.
+- Prepare for future plugin ecosystems (e.g., third-party services/layers
published independently) without breaking existing API paths.
+
+# Guide-level explanation
+
+As a user you depend on the facade crate `opendal` as before. Differences:
+
+- To enable a backend, turn on the corresponding feature, for example `opendal
= { version = "x.y", features = ["service-s3"] }`. The feature pulls in
`opendal-service-s3`.
+- Critical layers (error context, simulate, completeness, correctness check,
capability check, immutable index, HTTP client) are always available via
`opendal::layers` because they live in `opendal-core`.
+- Non-critical layers are opt-in features, e.g., `"layer-logging"` pulls
`opendal-layer-logging` and re-exports `LoggingLayer` under
`opendal::layers::LoggingLayer`.
+- API surface stays the same: `opendal::Operator`, `opendal::services::S3`,
`opendal::layers::LoggingLayer` still work; only crate boundaries change under
the hood.
+
+Migration for existing users:
+
+```toml
+# Old
+opendal = { version = "0.55", features = ["services-s3", "layers-logging"] }
+
+# New (facade keeps aliases for one major version)
+opendal = { version = "0.56", features = ["service-s3", "layer-logging"] }
+```
+
+# Reference-level explanation
+
+## Crate layout
+
+- `opendal-core`
+ - Modules: `types`, `raw`, `blocking`, `docs`, critical layers
(ErrorContext, Simulate, Complete, CorrectnessCheck, CapabilityCheck,
ImmutableIndex, HttpClient, optional TailCut).
+ - No service implementations, no non-critical layers.
+ - Exposes `OperatorRegistry` and `Scheme` but performs no builtin
registrations.
+- `opendal-service-{backend}`
+ - Contains the backend’s `Builder`, `Config`, accessor implementation,
scheme constant, and a `register(&OperatorRegistry)` helper.
+ - Depends only on `opendal-core` (plus backend-specific deps).
+- `opendal-layer-{name}`
+ - Each non-critical layer in its own crate (or small groups when strongly
coupled, e.g., oteltrace/otelmetrics).
+ - Depends on `opendal-core` and layer-specific deps.
+- `opendal` facade
+ - Depends on `opendal-core` (mandatory), `opendal-layer-*` and
`opendal-service-*` as optional deps behind features.
+ - Re-exports `core::{Operator, Result, ...}`, critical layers, and optional
services/layers.
+ - On build, calls `register_builtin_services` that invokes each enabled
service crate’s `register` function to populate `DEFAULT_OPERATOR_REGISTRY`.
+ - Maintains feature aliases for one major version (e.g., `services-s3` ->
`service-s3`, `layers-logging` -> `layer-logging`).
+
+## Feature matrix
+
+- Core features: keep executor/runtime toggles (`executors-tokio`,
`blocking`), internal helpers (`internal-path-cache`, `internal-tokio-rt`),
plus minimal TLS/http knobs.
+- Service features: `service-{backend}` live in facade, map 1:1 to
`opendal-service-{backend}` optional dependency.
+- Layer features: `layer-{name}` live in facade, map 1:1 to
`opendal-layer-{name}` optional dependency; critical layers require no feature.
+
+## Registry and Scheme
+
+- `Operator::from_uri` remains, but core contains no builtin registrations.
+- Facade constructs registry at init by conditionally calling each service
crate’s `register` based on enabled features.
+- `Scheme::enabled()` moves to facade to aggregate enabled services; core
provides base enum and helper for custom schemes.
+
+## Testing and CI
+
+- Core tests remain for types/raw/critical layers.
+- Each service crate carries unit tests plus behavior tests triggered via
shared harness; a new `opendal-behavior` test crate can reuse the existing
fixtures.
+- Each layer crate keeps its own minimal tests; integration tests that combine
layers/services run in facade or dedicated smoke suites.
+- Workspace-level tasks (`cargo clippy`, `cargo fmt --check`) cover all
crates; behavior matrix splits by service features to control runtime cost.
+
+## Compatibility
+
+- Public API paths stay stable through facade re-exports.
+- Feature aliases maintained for one major release; deprecation warnings in
docs and cargo feature descriptions.
+- Crate names and versions remain under Apache OpenDAL namespace.
+
+# Drawbacks
+
+- More crates increase release coordination overhead and CI matrix size.
+- Users pinning to core-only without facade must manually enable
services/layers they need.
+- Potential longer resolve time in dependency solvers due to many optional
dependencies.
+
+# Rationale and alternatives
+
+- Keeping critical layers in core avoids breaking invariants and reduces
dependency churn on safety-sensitive code.
+- Per-layer and per-service crates maximize compile-time selectivity and allow
independent evolution.
+- Alternative considered: one `opendal-layers` mega-crate; rejected to keep
feature granularity and lighter dependency pull for single-layer use.
+
+# Prior art
+
+- Rust ecosystem crates like `tokio` vs `tokio-util`, `tower` vs `tower-*`
split utilities from core runtime.
+- Storage SDKs (AWS SDK for Rust) split per-service crates to reduce bloat and
compile time.
+
+# Unresolved questions
+
+- Exact set of critical layers that must stay in core (TailCut, Retry?) may
need tuning after performance and safety review.
+- Whether to group strongly coupled layers (otelmetrics/oteltrace) into one
crate or keep fully separate.
+- Behavior test harness packaging: shared crate vs duplicated fixtures.
+
+# Future possibilities
+
+- Allow third-party service or layer crates to self-register via cargo
features or a plugin registry.
+- Provide a `minimal` facade profile that ships only core + registry glue for
embedders.
+- Introduce codegen to auto-wire service registrations to shrink manual glue
in facade.
diff --git a/core/src/docs/rfcs/mod.rs b/core/src/docs/rfcs/mod.rs
index 36cdaa6b2..5c5d6b82d 100644
--- a/core/src/docs/rfcs/mod.rs
+++ b/core/src/docs/rfcs/mod.rs
@@ -288,3 +288,7 @@ pub mod rfc_6707_capability_override_layer {}
/// Checksum
#[doc = include_str!("6817_checksum.md")]
pub mod rfc_6817_checksum {}
+
+/// Core
+#[doc = include_str!("6828_core.md")]
+pub mod rfc_6828_core {}
