This is an automated email from the ASF dual-hosted git repository.
xuanwo pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-opendal.git
The following commit(s) were added to refs/heads/main by this push:
new f76b33cb docs(refactor): Add more detailed description of operator,
accessor, and builder (#2094)
f76b33cb is described below
commit f76b33cb8275f891938a49ce918789d20efcd9f8
Author: Yansongsongsong <[email protected]>
AuthorDate: Mon Apr 24 23:00:10 2023 +0800
docs(refactor): Add more detailed description of operator, accessor, and
builder (#2094)
* docs: fix some typos
* docs: fix typo
* docs: add more descriptions for builder API
* docs: fix commet typo of memory builder
* docs: fix typo of Accessor
* docs: refactor operator accessor and builder related doc
* fix: revert wrong fix
* docs: fix incorrect description
* docs: fix diagram representation
* docs: fix unproper description of builder
* docs: fix unproper decription of builder
* docs: remove inappropriate analogy
---
core/src/docs/concepts.rs | 33 ++++++++++++++++++++++-----------
core/src/docs/internals/accessor.rs | 2 +-
core/src/raw/accessor.rs | 13 +++++++++++--
core/src/raw/adapters/kv/backend.rs | 8 +++++++-
core/src/services/memory/backend.rs | 2 +-
core/src/types/builder.rs | 20 ++++++++++++++++----
core/src/types/operator/operator.rs | 8 +++++++-
7 files changed, 65 insertions(+), 21 deletions(-)
diff --git a/core/src/docs/concepts.rs b/core/src/docs/concepts.rs
index aa01e6e9..db4f06db 100644
--- a/core/src/docs/concepts.rs
+++ b/core/src/docs/concepts.rs
@@ -17,12 +17,15 @@
//! The core concepts of OpenDAL's public API.
//!
-//! OpenDAL provides a unified abstraction for all storage services.
+//! OpenDAL provides a unified abstraction that helps developers access all
storage services.
//!
//! There are two core concepts in OpenDAL:
//!
-//! - [`Builder`]: Build an instance of underlying services.
-//! - [`Operator`]: A bridge between underlying implementation detail and
unified abstraction.
+//! - [`Builder`]: Builder accepts a series of parameters to set up an
instance of underlying services.
+//! You can adjust the behaviour of underlying services with these parameters.
+//! - [`Operator`]: Developer can access underlying storage services with
manipulating one Operator.
+//! The Operator is a delegate for underlying implementation detail, and
provides one unified access interface,
+//! including `read`, `write`, `list` and so on.
//!
//! If you are interested in internal implementation details, please have a
look at [`internals`][super::internals].
//!
@@ -30,7 +33,9 @@
//!
//! Let's start with [`Builder`].
//!
-//! A `Builder` is a trait that is implemented by the underlying services. We
can use a `Builder` to configure and create a service. Builder is the only
public API provided by services, and the detailed implementation is hidden.
+//! A `Builder` is a trait that is implemented by the underlying services. We
can use a `Builder` to configure and create a service.
+//! Developer can only create one service via Builder, in other words, Builder
is the only public API provided by services.
+//! And other detailed implementation will be hidden.
//!
//! ```text
//! ┌───────────┐ ┌───────────┐
@@ -41,6 +46,8 @@
//! ```
//!
//! All [`Builder`] provided by OpenDAL is under
[`services`][crate::services], we can refer to them like
`opendal::services::S3`.
+//! By right the builder will be named like `OneServiceBuilder`, but usually
we will export it to public with renaming it as one
+//! general name. For example, we will rename `S3Builder` to `S3` and
developer will use `S3` finally.
//!
//! For example:
//!
@@ -53,15 +60,19 @@
//! ```
//!
//! # Operator
-//!
-//! The [`Operator`] is a bridge between the underlying implementation details
and the unified abstraction. OpenDAL will erase all generic types and higher
abstraction around it.
+//! The [`Operator`] is a delegate for Service, the underlying implementation
detail that implements [`Accessor`][crate::raw::Accessor],
+//! and it also provides one unified access interface.
+//! It will hold one reference of Service with its all generic types erased by
OpenDAL,
+//! which is the reason why we say the Operator is the delegate of one Service.
//!
//! ```text
-//! ┌───────────┐ ┌───────────┐ ┌─────────────────┐
-//! │ │ build() │ │ type erase │ │
-//! │ Builder ├──────────►│ Service ├─────────────►│ Operator │
-//! │ │ │ │ │ │
-//! └───────────┘ └───────────┘ └─────────────────┘
+//! ┌────────────────────┐
+//! │ Operator │
+//! │ │delegate │
+//! ┌─────────┐ build │ ▼ │ rely on ┌─────────────────────┐
+//! │ Builder ├───────┼──►┌────────────┐ │◄────────┤ business logic code │
+//! └─────────┘ │ │ Service │ │ └─────────────────────┘
+//! └───┴────────────┴───┘
//! ```
//!
//! `Operator` can be built from `Builder`:
diff --git a/core/src/docs/internals/accessor.rs
b/core/src/docs/internals/accessor.rs
index 463ac5c8..d149457e 100644
--- a/core/src/docs/internals/accessor.rs
+++ b/core/src/docs/internals/accessor.rs
@@ -148,7 +148,7 @@
//!
//! First of all, let's pick a good [`Scheme`] for our duck service. The
//! scheme should be unique and easy to understand. Normally we should
-//! use it's formal name.
+//! use its formal name.
//!
//! For example, we will use `s3` for AWS S3 Compatible Storage Service
//! instead of `aws` or `awss3`. This is because there are many storage
diff --git a/core/src/raw/accessor.rs b/core/src/raw/accessor.rs
index d551fda0..ee6e7618 100644
--- a/core/src/raw/accessor.rs
+++ b/core/src/raw/accessor.rs
@@ -26,6 +26,16 @@ use crate::*;
/// Underlying trait of all backends for implementors.
///
+/// The actual data access of storage service happens in Accessor layer.
+/// Every storage supported by OpenDAL must implement [`Accessor`] but not all
+/// methods of [`Accessor`] will be implemented according to how the storage
service is.
+///
+/// For example, user can not modify the content from one HTTP file server
directly.
+/// So [`Http`][crate::services::Http] implements and provides only read
related actions.
+///
+/// [`Accessor`] gives default implementation for all methods which will raise
[`ErrorKind::Unsupported`] error.
+/// And what action this [`Accessor`] supports will be pointed out in
[`AccessorInfo`].
+///
/// # Note
///
/// Visit [`internals`][crate::docs::internals] for more tutorials.
@@ -73,7 +83,6 @@ pub trait Accessor: Send + Sync + Debug + Unpin + 'static {
///
/// - scheme: declare the scheme of backend.
/// - capabilities: declare the capabilities of current backend.
- /// - hints: declare the hints of current backend
fn info(&self) -> AccessorInfo;
/// Invoke the `create` operation on the specified path
@@ -487,7 +496,7 @@ impl<T: Accessor + ?Sized> Accessor for Arc<T> {
}
}
-/// FusedAccessor is the type erased accessor with `Box<dyn Read>`.
+/// FusedAccessor is the type erased accessor with `Arc<dyn Accessor>`.
pub type FusedAccessor = Arc<
dyn Accessor<
Reader = oio::Reader,
diff --git a/core/src/raw/adapters/kv/backend.rs
b/core/src/raw/adapters/kv/backend.rs
index 855b9475..55273102 100644
--- a/core/src/raw/adapters/kv/backend.rs
+++ b/core/src/raw/adapters/kv/backend.rs
@@ -25,7 +25,13 @@ use crate::ops::*;
use crate::raw::*;
use crate::*;
-/// Backend of kv service.
+/// Backend of kv service. If the storage service is one k-v-like service, it
should implement this kv [`Backend`] by right.
+///
+/// `Backend` implements one general logic on how to read, write, scan the
data from one kv store efficiently.
+/// And the [`Adapter`] held by `Backend` will handle how to communicate with
one k-v-like service really and provides
+/// a series of basic operation for this service.
+///
+/// OpenDAL developer can implement one new k-v store backend easily with help
of this Backend.
#[derive(Debug, Clone)]
pub struct Backend<S: Adapter> {
kv: Arc<S>,
diff --git a/core/src/services/memory/backend.rs
b/core/src/services/memory/backend.rs
index f34b2b0d..34c77350 100644
--- a/core/src/services/memory/backend.rs
+++ b/core/src/services/memory/backend.rs
@@ -43,7 +43,7 @@ pub struct MemoryBuilder {
}
impl MemoryBuilder {
- /// Set the root for dashmap.
+ /// Set the root for BTreeMap.
pub fn root(&mut self, path: &str) -> &mut Self {
self.root = Some(path.into());
self
diff --git a/core/src/types/builder.rs b/core/src/types/builder.rs
index 252ce9cc..237c103c 100644
--- a/core/src/types/builder.rs
+++ b/core/src/types/builder.rs
@@ -21,16 +21,28 @@ use std::env;
use crate::raw::*;
use crate::*;
-/// Builder is used to build a storage accessor used by [`Operator`].
+/// Builder is used to set up a real underlying service, i.e. storage accessor.
+///
+/// One builder is usually used by [`Operator`] during its initialization.
+/// It can be created by accepting several k-v pairs from one HashMap, one
iterator and specific environment variables.
+///
+/// By default each builder of underlying service must support deriving from
one HashMap.
+/// Besides that, according to the implementation, each builder will have its
own special methods
+/// to control the behavior of initialization of the underlying service.
+/// It often provides semantic interface instead of using dynamic k-v strings
directly.
+/// Because the latter way is obscure and hard to remember how many parameters
it will have.
+///
+/// So it is recommended that developer should read related doc of builder
carefully when you are working with one service.
+/// We also promise that every public API will provide detailed documentation.
///
/// It's recommended to use [`Operator::new`] to avoid use `Builder` trait
directly.
pub trait Builder: Default {
- /// Associated scheme for this builder.
+ /// Associated scheme for this builder. It indicates what underlying
service is.
const SCHEME: Scheme;
/// The accessor that built by this builder.
type Accessor: Accessor;
- /// Construct a builder from given map.
+ /// Construct a builder from given map which contains several parameters
needed by underlying service.
fn from_map(map: HashMap<String, String>) -> Self;
/// Construct a builder from given iterator.
@@ -41,7 +53,7 @@ pub trait Builder: Default {
Self::from_map(iter.collect())
}
- /// Construct a builder from envs.
+ /// Construct a builder from envs. Please note that the env should begin
with `opendal_{schema_name}_`.
fn from_env() -> Self
where
Self: Sized,
diff --git a/core/src/types/operator/operator.rs
b/core/src/types/operator/operator.rs
index 8187bdb1..8fdd9bf7 100644
--- a/core/src/types/operator/operator.rs
+++ b/core/src/types/operator/operator.rs
@@ -33,7 +33,10 @@ use crate::raw::*;
use crate::*;
/// Operator is the entry for all public async APIs.
+/// Developer should manipulate the data from storage service through Operator
only by right.
///
+/// We will usually do some general checks and data transformations in this
layer,
+/// like normalizing path from input, checking whether the path refers to one
file or one directory, and so on.
/// Read [`concepts`][docs::concepts] for know more about [`Operator`].
///
/// # Examples
@@ -61,8 +64,10 @@ use crate::*;
/// ```
#[derive(Clone, Debug)]
pub struct Operator {
+ // accessor is what Operator delegates for
accessor: FusedAccessor,
+ // limit is usually the maximum size of data that operator will handle in
one operation
limit: usize,
}
@@ -85,7 +90,8 @@ impl Operator {
self.accessor
}
- /// Get current operator's limit
+ /// Get current operator's limit.
+ /// Limit is usually the maximum size of data that operator will handle in
one operation.
pub fn limit(&self) -> usize {
self.limit
}
