kevinjqliu opened a new issue, #744:
URL: https://github.com/apache/arrow-rs-object-store/issues/744
## Context
PR #724 makes `reqwest` optional by introducing base provider features such
as:
- `cloud-base`
- `aws-base`
- `azure-base`
- `gcp-base`
- `http-base`
PR #707 makes the crypto provider configurable by separating the provider
implementation from the default bundled crypto choice.
Together, these changes introduce two independent configuration axes:
1. **HTTP transport**
- use the built-in `reqwest` transport
- or provide a custom HTTP connector
2. **Crypto provider**
- use the default bundled crypto provider
- or provide/configure crypto externally
This raises a naming question for the feature model before the next release.
## Proposal
Use `*-base` for provider implementation features, and compose transport and
crypto independently.
```toml
cloud-base = [
# shared cloud implementation deps, no default reqwest or bundled crypto
choice
]
reqwest = [
"dep:reqwest",
"reqwest/stream",
]
aws-lc-rs = [
"dep:aws-lc-rs",
]
ring = [
"dep:ring",
]
cloud = [
"cloud-base",
"reqwest",
"aws-lc-rs",
]
aws-base = [
"cloud-base",
"crc-fast",
"md-5",
]
aws = [
"aws-base",
"reqwest",
"aws-lc-rs",
]
azure-base = [
"cloud-base",
"httparse",
]
azure = [
"azure-base",
"reqwest",
"aws-lc-rs",
]
gcp-base = [
"cloud-base",
"rustls-pki-types",
]
gcp = [
"gcp-base",
"reqwest",
"aws-lc-rs",
]
http-base = [
"cloud-base",
]
http = [
"http-base",
"reqwest",
"aws-lc-rs",
]
```
## Rationale
- `cloud-base` means shared cloud implementation without the default
transport or bundled crypto choice.
- `*-base` means provider-specific implementation without the default
transport or bundled crypto choice.
- `reqwest` means enabling the built-in HTTP transport.
- `aws-lc-rs` and `ring` represent explicit crypto provider choices.
- `aws`, `azure`, `gcp`, and `http` preserve the existing batteries-included
behavior.
- `cloud` is preserved for backwards compatibility.
- Provider features compose directly from `*-base + reqwest +
crypto-provider` for clarity.
The core model is:
```text
cloud-base = shared cloud implementation without default reqwest or bundled
crypto
aws-base = cloud-base + AWS/S3-specific implementation deps
azure-base = cloud-base + Azure-specific implementation deps
gcp-base = cloud-base + GCS-specific implementation deps
http-base = cloud-base + HTTP-specific implementation deps
reqwest = built-in reqwest HTTP transport
aws-lc-rs = bundled aws-lc-rs crypto provider
ring = bundled ring crypto provider
cloud = cloud-base + reqwest + default crypto provider
compatibility alias for the old cloud behavior
aws = aws-base + reqwest + default crypto provider
azure = azure-base + reqwest + default crypto provider
gcp = gcp-base + reqwest + default crypto provider
http = http-base + reqwest + default crypto provider
```
## Example advanced compositions
```toml
# S3 implementation only; user provides HTTP connector and crypto provider
object_store = { default-features = false, features = ["aws-base"] }
# S3 implementation + built-in reqwest, but user controls crypto provider
object_store = { default-features = false, features = ["aws-base",
"reqwest"] }
# S3 implementation + custom HTTP connector, but object_store provides
default crypto
object_store = { default-features = false, features = ["aws-base",
"aws-lc-rs"] }
# S3 implementation + built-in reqwest + explicit crypto provider
object_store = { default-features = false, features = ["aws-base",
"reqwest", "ring"] }
```
## Open question
Should we standardize on this naming convention before the next release, so
optional transport and optional crypto use the same composable feature model?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
