+1, it's an interesting project. I'd like to be a mentor of this project. BTW, even though there are 37 contributors, I found Xuanwo did most of the development, which is not a good bus factor[1] of this project. We can make it better during the incubating process.
[1]https://en.wikipedia.org/wiki/Bus_factor Willem Jiang Twitter: willemjiang Weibo: 姜宁willem On Fri, Feb 17, 2023 at 7:22 AM tison <wander4...@gmail.com> wrote: > > Hi IPMC, > > I would like to propose OpenDAL[1] as a new apache incubator project, you > can find the proposal[2] of OpenDAL for more detail. > > [1] https://github.com/datafuselabs/opendal > [2] https://cwiki.apache.org/confluence/display/INCUBATOR/OpenDAL+Proposal > > Best, > tison. > > Full text of the proposal below: > > ## Abstract > > OpenDAL means “Open Data Access Layer”. It’s a Rust library that helps > developers access data freely, painlessly, and efficiently over multiple > services, including AWS S3, HDFS, POSIX-compatible file systems, and so on. > > ## Proposal > > OpenDAL provides the following features to support developers accessing > data freely, painlessly, and efficiently: > > - Freely > - Access different storage services in the same way > - Behavior tests for all services > - Cross-language/project bindings (working in progress) > - Painlessly > - **100%** of documents covered > - Powerful [Layers]( > https://opendal.databend.rs/opendal/layers/index.html) > - Automatic [retry]( > https://opendal.databend.rs/opendal/layers/struct.RetryLayer.html) support > - Full observability support: [logging]( > https://opendal.databend.rs/opendal/layers/struct.LoggingLayer.html > ), [tracing]( > https://opendal.databend.rs/opendal/layers/struct.TracingLayer.html), and > [metrics]( > https://opendal.databend.rs/opendal/layers/struct.MetricsLayer.html). > - Native chaos testing > - Native service-side encryption support > - Efficiently > - Zero cost: mapping to underlying API calls directly > - Best effort: auto-pick the best `read`/`seek`/`next` implementations > based on services > - [Auto metadata reuse]( > https://opendal.databend.rs/rfcs/0561-list-metadata-reuse.html): avoid > extra `metadata` calls > > OpenDAL was originally designed to be used by the Databend project but is > now being used by Mozilla's sccache, DeepETH's mars, and several database > startups. > > We believe that the OpenDAL project will provide diversity value to the > community if OpenDAL is brought into the Apache incubator. > > ## Background > > OpenDAL is being developed by an open-source community from day one and is > owned by DatafuseLabs. The project has been launched in February 2022. > > ## Rationale > > OpenDAL provides a unified storage abstraction layer that simplifies the > interfacing of different storage services. In addition, OpenDAL provides > further advanced storage encapsulation, enabling enhancements such as > automatic retry, request optimization, and observability. OpenDAL makes it > possible to develop once and run on any storage service. > > ## Initial Goals > > By transferring ownership of the project to the ASF, OpenDAL expects to > ensure its neutrality and further encourage and facilitate the adoption of > OpenDAL by the community. > > Some of the areas we would like to focus on during the Apache incubation > phase include: > > - A healthier community: more maintainers and contributors will be able to > participate in OpenDAL and own different modules. > - Wider adoption: OpenDAL can be adopted by more open source/commercial > projects, which in turn drives its own functionality. > - Richer integration: OpenDAL enables greater integration of storage > services and offers a wider range of language bindings. > > ## Current Status > > ### Meritocracy > > We intend to radically expand the initial developer and user community by > running the project the 'Apache way'. Users and new contributors will be > respected and welcomed. They will earn credit by participating in the > community and providing quality patches/support to move the project > forward. They will also be encouraged to provide non-code contributions > (documentation, events, community management, etc.) and will be rewarded > accordingly. Those with a proven track record of support and quality will > be encouraged to become committers. > > ### Community > > Contributors: 37 > > Users: > > - Databend: A cloud data warehouse > - GreptimeDB: A time-series database > - Sccache: ccache with cloud storage > - RisingWave: A Distributed SQL Database for Stream Processing > > ### Core Developers > > The core developers are all experienced open-source developers. They have > been running the OpenDAL community for 1 year. > > ### Alignment > > ## Known Risks > > ### Project Name > > We have checked and believe that the name is appropriate and that the > project has legal permission to continue using its current name. There are > no other projects with this name found in a Google search. > > ### Orphan Products > > ### Inexperience with Open Source > > OpenDAL's core developers are all experienced open source contributors, and > its main maintainer Xuanwo has 10 years of open source experience, having > worked on a number of open source projects including Hexo, TiDB, TiKV, > Databend, Sccache, and others. > > ### Length of Incubation > > Expect to enter incubation in two months and graduate in about two years. > > ### Homogenous Developers > > OpenDAL developers come from a variety of backgrounds and contribute to the > OpenDAL project for different usage scenarios. > > ### Reliance on Salaried Developers > > Most developers are paid by their employers to contribute to this project. > It's a big risk indeed, we expect to attract more maintainers and > contributors from outside the DatafuseLabs to address this. > > ### Relationships with Other Apache Products > > - OpenDAL can be used to operate files such as parquet, avro, … > - OpenDAL provides a shim that can be used with arrow object_store > > ### An Excessive Fascination with the Apache Brand > > ## Documentation > > The document of OpenDAL is hosted at [ > https://opendal.databend.rs/](https://opendal.databend.rs/). And opendal’s > document is self-contained, all its current and historical versions could > be found at [ > https://docs.rs/opendal/latest/opendal/](https://docs.rs/opendal/latest/opendal/) > . > > ## Initial Source > > The project currently holds a GitHub repository and a Cargo crate: > > - [ > https://github.com/datafuselabs/opendal](https://github.com/datafuselabs/opendal) > - [https://crates.io/crates/opendal](https://crates.io/crates/opendal) > - [https://opendal.databend.rs/](https://opendal.databend.rs/) > > The crate will retain its name, while the repository will be moved to > apache org, and the website will be permanently moved to [opendal.apache.org > ](http://opendal.apache.org) if the proposal gets accepted. > > ## Source and Intellectual Property Submission Plan > > ### External Dependencies > > > Generated by `cargo deny list` > > - Apache-2.0 (192): > - ahash@0.7.6 > - anes@0.1.6 > - anyhow@1.0.68 > - async-channel@1.8.0 > - async-compat@0.2.1 > - async-trait@0.1.60 > - autocfg@1.1.0 > - backon@0.2.0 > - base64@0.13.1 > - base64@0.21.0 > - bitflags@1.3.2 > - block-buffer@0.10.3 > - bumpalo@3.11.1 > - cast@0.3.0 > - cc@1.0.78 > - cfg-if@1.0.0 > - ciborium@0.2.0 > - ciborium-io@0.2.0 > - ciborium-ll@0.2.0 > - clap@3.2.23 > - clap_lex@0.2.4 > - concurrent-queue@2.0.0 > - core-foundation@0.9.3 > - core-foundation-sys@0.8.3 > - cpufeatures@0.2.5 > - criterion@0.4.0 > - criterion-plot@0.5.0 > - crossbeam-channel@0.5.6 > - crossbeam-deque@0.8.2 > - crossbeam-epoch@0.9.13 > - crossbeam-utils@0.8.14 > - crypto-common@0.1.6 > - ctor@0.1.26 > - deadpool@0.9.5 > - deadpool-runtime@0.1.2 > - diff@0.1.13 > - digest@0.10.6 > - dirs@4.0.0 > - dirs-sys@0.3.7 > - either@1.8.0 > - encoding_rs@0.8.31 > - env_logger@0.10.0 > - event-listener@2.5.3 > - fastrand@1.8.0 > - flagset@0.4.3 > - fnv@1.0.7 > - form_urlencoded@1.1.0 > - futures@0.3.25 > - futures-channel@0.3.25 > - futures-core@0.3.25 > - futures-executor@0.3.25 > - futures-io@0.3.25 > - futures-lite@1.12.0 > - futures-macro@0.3.25 > - futures-sink@0.3.25 > - futures-task@0.3.25 > - futures-timer@3.0.2 > - futures-util@0.3.25 > - getrandom@0.1.16 > - getrandom@0.2.8 > - half@1.8.2 > - hashbrown@0.12.3 > - hermit-abi@0.1.19 > - hermit-abi@0.2.6 > - hex@0.4.3 > - hmac@0.12.1 > - http@0.2.8 > - http-types@2.12.0 > - httparse@1.8.0 > - httpdate@1.0.2 > - humantime@2.1.0 > - hyper-rustls@0.23.2 > - idna@0.3.0 > - indexmap@1.9.2 > - io-lifetimes@1.0.3 > - ipnet@2.7.0 > - itertools@0.10.5 > - itoa@1.0.5 > - js-sys@0.3.60 > - lazy_static@1.4.0 > - libc@0.2.139 > - linux-raw-sys@0.1.4 > - lock_api@0.4.9 > - log@0.4.17 > - md-5@0.10.5 > - mime@0.3.16 > - num-bigint@0.4.3 > - num-integer@0.1.45 > - num-traits@0.2.15 > - num_cpus@1.15.0 > - once_cell@1.17.0 > - opendal@0.25.1 > - openssl-probe@0.1.5 > - opentelemetry@0.17.0 > - opentelemetry-jaeger@0.16.0 > - opentelemetry-semantic-conventions@0.9.0 > - os_str_bytes@6.4.1 > - parking@2.0.0 > - parking_lot@0.12.1 > - parking_lot_core@0.9.5 > - paste@1.0.11 > - percent-encoding@2.2.0 > - pin-project@1.0.12 > - pin-project-internal@1.0.12 > - pin-project-lite@0.2.9 > - pin-utils@0.1.0 > - ppv-lite86@0.2.17 > - pretty_assertions@1.3.0 > - proc-macro2@1.0.49 > - quote@1.0.23 > - rand@0.7.3 > - rand@0.8.5 > - rand_chacha@0.2.2 > - rand_chacha@0.3.1 > - rand_core@0.5.1 > - rand_core@0.6.4 > - rand_hc@0.2.0 > - rayon@1.6.1 > - rayon-core@1.10.1 > - regex@1.7.0 > - regex-syntax@0.6.28 > - reqsign@0.8.1 > - reqwest@0.11.13 > - rustix@0.36.6 > - rustls@0.20.7 > - rustls-native-certs@0.6.2 > - rustls-pemfile@1.0.1 > - ryu@1.0.12 > - scopeguard@1.1.0 > - sct@0.7.0 > - security-framework@2.7.0 > - security-framework-sys@2.6.1 > - serde@1.0.152 > - serde_derive@1.0.152 > - serde_json@1.0.91 > - serde_qs@0.8.5 > - serde_urlencoded@0.7.1 > - sha1@0.10.5 > - sha2@0.10.6 > - smallvec@1.10.0 > - socket2@0.4.7 > - syn@1.0.107 > - thiserror@1.0.38 > - thiserror-impl@1.0.38 > - thread_local@1.1.4 > - threadpool@1.8.1 > - thrift@0.15.0 > - time@0.3.17 > - time-core@0.1.0 > - time-macros@0.2.6 > - tinytemplate@1.2.1 > - tinyvec@1.6.0 > - tinyvec_macros@0.1.0 > - tokio-rustls@0.23.4 > - typenum@1.16.0 > - unicase@2.6.0 > - unicode-bidi@0.3.8 > - unicode-ident@1.0.6 > - unicode-normalization@0.1.22 > - ureq@2.6.1 > - url@2.3.1 > - uuid@1.2.2 > - version_check@0.9.4 > - waker-fn@1.1.0 > - wasi@0.11.0+wasi-snapshot-preview1 > - wasi@0.9.0+wasi-snapshot-preview1 > - wasm-bindgen@0.2.83 > - wasm-bindgen-backend@0.2.83 > - wasm-bindgen-futures@0.4.33 > - wasm-bindgen-macro@0.2.83 > - wasm-bindgen-macro-support@0.2.83 > - wasm-bindgen-shared@0.2.83 > - web-sys@0.3.60 > - winapi@0.3.9 > - winapi-i686-pc-windows-gnu@0.4.0 > - winapi-x86_64-pc-windows-gnu@0.4.0 > - windows-sys@0.36.1 > - windows-sys@0.42.0 > - windows_aarch64_gnullvm@0.42.0 > - windows_aarch64_msvc@0.36.1 > - windows_aarch64_msvc@0.42.0 > - windows_i686_gnu@0.36.1 > - windows_i686_gnu@0.42.0 > - windows_i686_msvc@0.36.1 > - windows_i686_msvc@0.42.0 > - windows_x86_64_gnu@0.36.1 > - windows_x86_64_gnu@0.42.0 > - windows_x86_64_gnullvm@0.42.0 > - windows_x86_64_msvc@0.36.1 > - windows_x86_64_msvc@0.42.0 > - wiremock@0.5.16 > - yansi@0.5.1 > - Apache-2.0 WITH LLVM-exception (5) > - io-lifetimes@1.0.3 > - linux-raw-sys@0.1.4 > - rustix@0.36.6 > - wasi@0.11.0+wasi-snapshot-preview1 > - wasi@0.9.0+wasi-snapshot-preview1 > - BSD-3-Clause (3) > - encoding_rs@0.8.31 > - instant@0.1.12 > - subtle@2.4.1 > - BSL-1.0 (1): > - ryu@1.0.12 > - ISC (8): > - hyper-rustls@0.23.2 > - rustls@0.20.7 > - rustls-native-certs@0.6.2 > - rustls-pemfile@1.0.1 > - sct@0.7.0 > - simple_asn1@0.6.2 > - untrusted@0.7.1 > - webpki@0.22.0 > - MIT (242) > - ahash@0.7.6 > - aho-corasick@0.7.20 > - anes@0.1.6 > - anyhow@1.0.68 > - assert-json-diff@2.0.2 > - async-channel@1.8.0 > - async-compat@0.2.1 > - async-trait@0.1.60 > - atty@0.2.14 > - autocfg@1.1.0 > - base64@0.13.1 > - base64@0.21.0 > - bincode@2.0.0-rc.2 > - bincode_derive@2.0.0-rc.2 > - bitflags@1.3.2 > - block-buffer@0.10.3 > - bumpalo@3.11.1 > - byteorder@1.4.3 > - bytes@1.3.0 > - cast@0.3.0 > - cc@1.0.78 > - cfg-if@1.0.0 > - clap@3.2.23 > - clap_lex@0.2.4 > - concurrent-queue@2.0.0 > - core-foundation@0.9.3 > - core-foundation-sys@0.8.3 > - cpufeatures@0.2.5 > - criterion@0.4.0 > - criterion-plot@0.5.0 > - crossbeam-channel@0.5.6 > - crossbeam-deque@0.8.2 > - crossbeam-epoch@0.9.13 > - crossbeam-utils@0.8.14 > - crypto-common@0.1.6 > - ctor@0.1.26 > - deadpool@0.9.5 > - deadpool-runtime@0.1.2 > - diff@0.1.13 > - digest@0.10.6 > - dirs@4.0.0 > - dirs-sys@0.3.7 > - dlv-list@0.3.0 > - dotenvy@0.15.6 > - either@1.8.0 > - encoding_rs@0.8.31 > - env_logger@0.10.0 > - event-listener@2.5.3 > - fastrand@1.8.0 > - fnv@1.0.7 > - form_urlencoded@1.1.0 > - futures@0.3.25 > - futures-channel@0.3.25 > - futures-core@0.3.25 > - futures-executor@0.3.25 > - futures-io@0.3.25 > - futures-lite@1.12.0 > - futures-macro@0.3.25 > - futures-sink@0.3.25 > - futures-task@0.3.25 > - futures-timer@3.0.2 > - futures-util@0.3.25 > - generic-array@0.14.6 > - getrandom@0.1.16 > - getrandom@0.2.8 > - h2@0.3.15 > - half@1.8.2 > - hashbrown@0.12.3 > - hermit-abi@0.1.19 > - hermit-abi@0.2.6 > - hex@0.4.3 > - hmac@0.12.1 > - http@0.2.8 > - http-body@0.4.5 > - http-types@2.12.0 > - httparse@1.8.0 > - httpdate@1.0.2 > - humantime@2.1.0 > - hyper@0.14.23 > - hyper-rustls@0.23.2 > - idna@0.3.0 > - indexmap@1.9.2 > - infer@0.2.3 > - integer-encoding@3.0.4 > - io-lifetimes@1.0.3 > - ipnet@2.7.0 > - is-terminal@0.4.2 > - itertools@0.10.5 > - itoa@1.0.5 > - js-sys@0.3.60 > - jsonwebtoken@8.2.0 > - lazy_static@1.4.0 > - libc@0.2.139 > - linux-raw-sys@0.1.4 > - lock_api@0.4.9 > - log@0.4.17 > - matchers@0.1.0 > - md-5@0.10.5 > - memchr@2.5.0 > - memoffset@0.7.1 > - mime@0.3.16 > - mime_guess@2.0.4 > - mio@0.8.5 > - nu-ansi-term@0.46.0 > - num-bigint@0.4.3 > - num-integer@0.1.45 > - num-traits@0.2.15 > - num_cpus@1.15.0 > - once_cell@1.17.0 > - oorandom@11.1.3 > - openssl-probe@0.1.5 > - ordered-float@1.1.1 > - ordered-multimap@0.4.3 > - os_str_bytes@6.4.1 > - output_vt100@0.1.3 > - overload@0.1.1 > - parking@2.0.0 > - parking_lot@0.12.1 > - parking_lot_core@0.9.5 > - paste@1.0.11 > - pem@1.1.0 > - percent-encoding@2.2.0 > - pin-project@1.0.12 > - pin-project-internal@1.0.12 > - pin-project-lite@0.2.9 > - pin-utils@0.1.0 > - plotters@0.3.4 > - plotters-backend@0.3.4 > - plotters-svg@0.3.3 > - ppv-lite86@0.2.17 > - pretty_assertions@1.3.0 > - proc-macro2@1.0.49 > - quick-xml@0.27.1 > - quote@1.0.23 > - rand@0.7.3 > - rand@0.8.5 > - rand_chacha@0.2.2 > - rand_chacha@0.3.1 > - rand_core@0.5.1 > - rand_core@0.6.4 > - rand_hc@0.2.0 > - rayon@1.6.1 > - rayon-core@1.10.1 > - redox_syscall@0.2.16 > - redox_users@0.4.3 > - regex@1.7.0 > - regex-automata@0.1.10 > - regex-syntax@0.6.28 > - reqwest@0.11.13 > - retain_mut@0.1.9 > - rust-ini@0.18.0 > - rustix@0.36.6 > - rustls@0.20.7 > - rustls-native-certs@0.6.2 > - rustls-pemfile@1.0.1 > - same-file@1.0.6 > - schannel@0.1.20 > - scopeguard@1.1.0 > - sct@0.7.0 > - security-framework@2.7.0 > - security-framework-sys@2.6.1 > - serde@1.0.152 > - serde_derive@1.0.152 > - serde_json@1.0.91 > - serde_qs@0.8.5 > - serde_urlencoded@0.7.1 > - sha1@0.10.5 > - sha2@0.10.6 > - sharded-slab@0.1.4 > - size@0.4.1 > - slab@0.4.7 > - smallvec@1.10.0 > - socket2@0.4.7 > - spin@0.5.2 > - syn@1.0.107 > - termcolor@1.1.3 > - textwrap@0.16.0 > - thiserror@1.0.38 > - thiserror-impl@1.0.38 > - thread_local@1.1.4 > - threadpool@1.8.1 > - time@0.3.17 > - time-core@0.1.0 > - time-macros@0.2.6 > - tinytemplate@1.2.1 > - tinyvec@1.6.0 > - tinyvec_macros@0.1.0 > - tokio@1.23.1 > - tokio-macros@1.8.2 > - tokio-rustls@0.23.4 > - tokio-util@0.7.4 > - tower-service@0.3.2 > - tracing@0.1.37 > - tracing-core@0.1.30 > - tracing-log@0.1.3 > - tracing-opentelemetry@0.17.4 > - tracing-subscriber@0.3.16 > - try-lock@0.2.3 > - typenum@1.16.0 > - unicase@2.6.0 > - unicode-bidi@0.3.8 > - unicode-ident@1.0.6 > - unicode-normalization@0.1.22 > - ureq@2.6.1 > - url@2.3.1 > - uuid@1.2.2 > - valuable@0.1.0 > - version_check@0.9.4 > - virtue@0.0.8 > - waker-fn@1.1.0 > - walkdir@2.3.2 > - want@0.3.0 > - wasi@0.11.0+wasi-snapshot-preview1 > - wasi@0.9.0+wasi-snapshot-preview1 > - wasm-bindgen@0.2.83 > - wasm-bindgen-backend@0.2.83 > - wasm-bindgen-futures@0.4.33 > - wasm-bindgen-macro@0.2.83 > - wasm-bindgen-macro-support@0.2.83 > - wasm-bindgen-shared@0.2.83 > - web-sys@0.3.60 > - winapi@0.3.9 > - winapi-i686-pc-windows-gnu@0.4.0 > - winapi-util@0.1.5 > - winapi-x86_64-pc-windows-gnu@0.4.0 > - windows-sys@0.36.1 > - windows-sys@0.42.0 > - windows_aarch64_gnullvm@0.42.0 > - windows_aarch64_msvc@0.36.1 > - windows_aarch64_msvc@0.42.0 > - windows_i686_gnu@0.36.1 > - windows_i686_gnu@0.42.0 > - windows_i686_msvc@0.36.1 > - windows_i686_msvc@0.42.0 > - windows_x86_64_gnu@0.36.1 > - windows_x86_64_gnu@0.42.0 > - windows_x86_64_gnullvm@0.42.0 > - windows_x86_64_msvc@0.36.1 > - windows_x86_64_msvc@0.42.0 > - winreg@0.10.1 > - wiremock@0.5.16 > - yansi@0.5.1 > - MPL-2.0 (1): > - webpki-roots@0.22.6 > - Unicode-DFS-2016 (1): > - unicode-ident@1.0.6 > - Unlicense (8): > - aho-corasick@0.7.20 > - byteorder@1.4.3 > - memchr@2.5.0 > - regex-automata@0.1.10 > - same-file@1.0.6 > - termcolor@1.1.3 > - walkdir@2.3.2 > - winapi-util@0.1.5 > - Zlib (2): > - tinyvec@1.6.0 > - tinyvec_macros@0.1.0 > - Unlicensed (1): > - ring@0.16.20 > > ring is licensed under a mix of [MIT, ISC, and OpenSSL’s licenses]( > https://github.com/briansmith/ring/blob/main/LICENSE). OpenSSL itself is > licensed under [Apache 2.0]( > https://www.openssl.org/source/apache-license-2.0.txt). So ring is Apache > 2.0 compatible. > > > ### Cryptography > > N/A > > ## Required Resources > > ### Mailing Lists > > - priv...@opendal.apache.org > - d...@opendal.apache.org > - comm...@opendal.apache.org > - iss...@opendal.apache.org > - bu...@opendal.apache.org > > ### Subversion Directory > > N/A > > ### Git Repositories > > From [ > https://github.com/datafuselabs/opendal](https://github.com/datafuselabs/opendal) > > - [ > https://gitbox.apache.org/asf/repos/incubator-opendal](https://gitbox.apache.org/asf/repos/incubator-opendal) > - [ > https://github.com/apache/incubator-opendal](https://github.com/apache/incubator-opendal) > > ### Issue Tracking > > The community would like to continue using GitHub Issues. > > ### Other Resources > > The community has already chosen GitHub actions as continuous integration > tools. > > ## Initial Committers > > - Hao Ding [[git...@xuanwo.io](mailto:git...@xuanwo.io)] > - Lue Cai [[cai...@bupt.edu.cn](mailto:cai...@bupt.edu.cn)] > - Zhuoran Shang [[psi...@outlook.com](mailto:psi...@outlook.com)] > - Benwang Li [[libenw1...@gmail.com](mailto:libenw1...@gmail.com)] > > ## Sponsors > > ### Champion > > - tison [ti...@apache.org] > > ### Nominated Mentors > > - hexiaoqiao [hexiaoq...@apache.org] > - tedliu [ted...@apache.org] > - tison [ti...@apache.org] > - wusheng [wush...@apache.org] > > ### Sponsor Entity > > The Incubator --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
