AryanVBW opened a new pull request, #5499:
URL: https://github.com/apache/opendal/pull/5499
# Add TLS Support for AWS ElastiCache Memcached
This PR adds TLS support for Memcached connections, particularly targeting
AWS ElastiCache serverless instances which require TLS connections. Instead of
using plain TCP conn
This pull request introduces several changes to the
`core/src/services/memcached` module to add support for TLS connections, which
is required for AWS ElastiCache Memcached serverless instances. The most
important changes include adding TLS-related dependencies, updating the
`MemcachedBuilder` and `MemcacheConnectionManager` to support TLS
configuration, and modifying the `Connection` struct to handle both plain and
TLS connections.
### TLS Support:
*
[`core/src/services/memcached/backend.rs`](diffhunk://#diff-786bd730c99162ea9ee3f086fb571e522159e9d6623f5e207dbde76ff30bf855R23-R25):
Added `tokio_native_tls` dependencies and updated `MemcachedBuilder` to
include methods for enabling TLS and setting the CA certificate file path.
[[1]](diffhunk://#diff-786bd730c99162ea9ee3f086fb571e522159e9d6623f5e207dbde76ff30bf855R23-R25)
[[2]](diffhunk://#diff-786bd730c99162ea9ee3f086fb571e522159e9d6623f5e207dbde76ff30bf855R88-R103)
*
[`core/src/services/memcached/backend.rs`](diffhunk://#diff-786bd730c99162ea9ee3f086fb571e522159e9d6623f5e207dbde76ff30bf855L245-R239):
Modified `MemcacheConnectionManager` to handle TLS connections by adding
`enable_tls` and `ca_cert` fields and updating the `connect` method to
establish a TLS connection if enabled.
[[1]](diffhunk://#diff-786bd730c99162ea9ee3f086fb571e522159e9d6623f5e207dbde76ff30bf855L245-R239)
[[2]](diffhunk://#diff-786bd730c99162ea9ee3f086fb571e522159e9d6623f5e207dbde76ff30bf855L266-R298)
*
[`core/src/services/memcached/config.rs`](diffhunk://#diff-afc9b800e90af29db42063799acfd3636266fcd4b4fb952d3d0bf9c30fd2c330R43-R48):
Updated `MemcachedConfig` to include `enable_tls` and `ca_cert` fields for TLS
configuration.
### Connection Handling:
*
[`core/src/services/memcached/backend.rs`](diffhunk://#diff-786bd730c99162ea9ee3f086fb571e522159e9d6623f5e207dbde76ff30bf855L103-R147):
Refactored `Adapter` to use a connection pool instead of managing individual
connections, and updated the `Builder` implementation to create the connection
pool with the new `MemcacheConnectionManager`.
[[1]](diffhunk://#diff-786bd730c99162ea9ee3f086fb571e522159e9d6623f5e207dbde76ff30bf855L103-R147)
[[2]](diffhunk://#diff-786bd730c99162ea9ee3f086fb571e522159e9d6623f5e207dbde76ff30bf855L165-R163)
*
[`core/src/services/memcached/binary.rs`](diffhunk://#diff-fe1fb8d6eb49bb8d1262dc29c15b5b44f69402757d787a40883b78fc3cdabd62R23):
Modified the `Connection` struct to use a dynamic stream (`Box<dyn AsyncRead +
AsyncWrite + Send + Unpin>`) to support both plain and TLS connections, and
updated related methods to work with the new stream type.
[[1]](diffhunk://#diff-fe1fb8d6eb49bb8d1262dc29c15b5b44f69402757d787a40883b78fc3cdabd62R23)
[[2]](diffhunk://#diff-fe1fb8d6eb49bb8d1262dc29c15b5b44f69402757d787a40883b78fc3cdabd62L64-R65)
[[3]](diffhunk://#diff-fe1fb8d6eb49bb8d1262dc29c15b5b44f69402757d787a40883b78fc3cdabd62L101-R111)
ections (`telnet`), users can now establish secure TLS connections (`openssl
s_client`) with their Memcached instances.
## Changes
### 1. Configuration Options
Added new configuration options to `MemcachedConfig`:
```rust
pub struct MemcachedConfig {
// ... existing fields ...
/// Enable TLS for the connection.
/// Required for AWS ElastiCache Memcached serverless instances.
pub enable_tls: Option<bool>,
/// Path to CA certificate file for TLS verification.
pub ca_cert: Option<String>,
}
```
### 2. Builder Methods
Added new builder methods to `MemcachedBuilder`:
```rust
impl MemcachedBuilder {
/// Enable TLS for the connection.
pub fn enable_tls(mut self, enable: bool) -> Self {
self.config.enable_tls = Some(enable);
self
}
/// Set the CA certificate file path for TLS verification.
pub fn ca_cert(mut self, ca_cert: &str) -> Self {
if !ca_cert.is_empty() {
self.config.ca_cert = Some(ca_cert.to_string());
}
self
}
}
```
### 3. TLS Connection Handling
- Added TLS support using `tokio-native-tls`
- Implemented dynamic stream type handling for both TLS and non-TLS
connections
- Added support for custom CA certificates for AWS ElastiCache verification
- Updated the binary protocol implementation to work with TLS streams
## Usage Example
```rust
let builder = MemcachedBuilder::default()
.endpoint("your-elasticache-endpoint:11211")
.enable_tls(true)
.ca_cert("/path/to/ca.pem") // Optional: provide CA cert if needed
.build()?;
```
## Implementation Details
1. **Stream Abstraction**: Updated `Connection` to use a boxed trait object
that can handle both TLS and non-TLS streams:
```rust
pub struct Connection {
stream: Box<dyn AsyncRead + AsyncWrite + Send + Unpin>,
}
```
2. **TLS Configuration**: Added TLS configuration to
`MemcacheConnectionManager`:
```rust
pub struct MemcacheConnectionManager {
address: String,
username: Option<String>,
password: Option<String>,
enable_tls: bool,
ca_cert: Option<String>,
}
```
3. **Connection Logic**: Implemented smart connection handling that
automatically uses TLS when enabled:
```rust
async fn connect(&self) -> Result<Self::Connection, Self::Error> {
if self.enable_tls {
// TLS connection logic with certificate handling
} else {
// Regular TCP connection logic
}
}
```
## Testing
- [ ] Added unit tests for TLS configuration
- [ ] Added integration tests with AWS ElastiCache
- [ ] Tested backward compatibility with non-TLS connections
- [ ] Verified error handling for invalid certificates
## Dependencies
Added `tokio-native-tls` for TLS support:
```toml
[dependencies]
tokio-native-tls = "0.3"
```
## Related Issues
https://github.com/apache/opendal/issues/5419
@killme2008 @drmingdrmer @jayvdb @viirya @qrilka @Xuanwo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
