This is an automated email from the ASF dual-hosted git repository. HTHou pushed a commit to branch codex/use-rustls in repository https://gitbox.apache.org/repos/asf/iotdb-client-rust.git
commit 5c9a4d80f95365f420563b50cbf2e9b5a50b97c5 Author: HTHou <[email protected]> AuthorDate: Wed Aug 12 17:31:54 2026 +0800 Use rustls for TLS --- .github/workflows/ci.yml | 33 ++++++ Cargo.toml | 19 +++- README.md | 11 +- README_ZH.md | 10 +- src/connection/mod.rs | 265 ++++++++++++++++++++++++++++++++++--------- src/error.rs | 5 +- tests/fixtures/tls/README.md | 14 +-- 7 files changed, 287 insertions(+), 70 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6bec0ae..23b6d6f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -65,6 +65,39 @@ jobs: - name: Unit tests (tls) run: cargo test --features tls + tls-platforms: + name: tls (${{ matrix.os }}) + runs-on: ${{ matrix.os }} + strategy: + fail-fast: false + matrix: + os: [macos-latest, windows-latest] + steps: + - uses: actions/checkout@v4 + + - uses: dtolnay/rust-toolchain@4cda84d5c5c54efe2404f9d843567869ab1699d4 # stable + + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + + # Exercises each OS verifier and native root-store integration. + - name: Unit tests (tls) + run: cargo test --features tls + + msrv: + name: msrv (rust 1.75) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: dtolnay/rust-toolchain@4cda84d5c5c54efe2404f9d843567869ab1699d4 + with: + toolchain: 1.75.0 + + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + + - name: Check (tls) + run: cargo check --features tls + integration: runs-on: ubuntu-latest strategy: diff --git a/Cargo.toml b/Cargo.toml index 8d9b168..c025128 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -36,13 +36,24 @@ thrift = "0.23" byteorder = "1.5" chrono = "0.4" log = "0.4" -native-tls = { version = "0.2", optional = true } +# thrift 0.23 allows any uuid 1.x; 1.21+ raises MSRV to Rust 1.85. +uuid = "=1.20.0" +rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"], optional = true } +rustls-pemfile = { version = "2.2", optional = true } +# 0.7 raises MSRV to 1.85; the crate currently supports Rust 1.75. +rustls-platform-verifier = { version = "0.6.2", optional = true } +# Keep Cargo 1.75 from resolving zeroize 1.9, whose manifest uses edition 2024. +zeroize = { version = "=1.8.2", optional = true } + +[target.'cfg(any(target_vendor = "apple"))'.dependencies] +# rustls-platform-verifier 0.6 allows 3.x; 3.7 raises MSRV to Rust 1.85. +security-framework = { version = "=3.5.1", optional = true } [dev-dependencies] env_logger = "0.11" [features] default = [] -# TLS support via the platform-native TLS stack (SecureTransport / -# SChannel / OpenSSL). Adds `use_ssl` & friends to SessionConfig. -tls = ["dep:native-tls"] +# TLS 1.2/1.3 via rustls, using the ring crypto provider and platform +# certificate verification. Adds `use_ssl` & friends to SessionConfig. +tls = ["dep:rustls", "dep:rustls-pemfile", "dep:rustls-platform-verifier", "dep:zeroize", "dep:security-framework"] diff --git a/README.md b/README.md index ef06b58..5691ab7 100644 --- a/README.md +++ b/README.md @@ -183,7 +183,12 @@ let config = SessionConfig { enable_rpc_compression: true, ..Default::default() It must match the **server** setting `dn_rpc_thrift_compression_enable` (default `false`). The server speaks exactly one protocol — there is no per-connection negotiation, so a mismatch in either direction fails at the first RPC with a transport error. -**TLS** is behind the `tls` cargo feature (platform-native TLS via [`native-tls`](https://crates.io/crates/native-tls)): +**TLS** is behind the `tls` cargo feature. It uses [`rustls`](https://crates.io/crates/rustls) +with the `ring` crypto provider and supports TLS 1.2/1.3. Server certificates are +verified with [`rustls-platform-verifier`](https://crates.io/crates/rustls-platform-verifier), +so the platform trust store and verification policy are used where available; +on Linux/BSD, platform roots are loaded and verified with WebPKI. A CA supplied +through `ca_cert_path` is added to those platform roots. ```toml iotdb-client-rust = { version = "0.1", features = ["tls"] } @@ -200,6 +205,10 @@ let config = SessionConfig { // or: TableSession::builder().use_ssl(true).ca_cert_path("ca.pem")... ``` +`accept_invalid_certs` disables certificate-chain and hostname verification, +but TLS handshake signatures are still cryptographically verified. It should +only be used with controlled test servers. + For **mutual TLS** (server has `thrift_ssl_client_auth=true`), add a PEM client certificate and its PKCS#8 key — the analogue of the Node.js `sslOptions.cert`/`sslOptions.key`: ```rust diff --git a/README_ZH.md b/README_ZH.md index 8161bf1..f3e454a 100644 --- a/README_ZH.md +++ b/README_ZH.md @@ -183,7 +183,12 @@ let config = SessionConfig { enable_rpc_compression: true, ..Default::default() 必须与**服务端**配置 `dn_rpc_thrift_compression_enable`（默认 `false`）一致。服务端只讲一种协议——没有按连接协商的机制，任意方向的不匹配都会在第一个 RPC 上以传输错误失败。 -**TLS** 位于 `tls` cargo feature 之后（基于 [`native-tls`](https://crates.io/crates/native-tls) 的平台原生 TLS）： +**TLS** 位于 `tls` cargo feature 之后。其底层使用 +[`rustls`](https://crates.io/crates/rustls) 和 `ring` 密码学 provider，支持 +TLS 1.2/1.3。服务端证书通过 +[`rustls-platform-verifier`](https://crates.io/crates/rustls-platform-verifier) +校验：在支持的平台上使用系统信任库及校验策略；在 Linux/BSD 上加载系统根证书并通过 +WebPKI 校验。`ca_cert_path` 指定的 CA 会追加到系统根证书中。 ```toml iotdb-client-rust = { version = "0.1", features = ["tls"] } @@ -200,6 +205,9 @@ let config = SessionConfig { // 或：TableSession::builder().use_ssl(true).ca_cert_path("ca.pem")... ``` +`accept_invalid_certs` 会关闭证书链和主机名校验，但 TLS 握手签名仍会经过密码学校验。 +该选项只应对受控的测试服务使用。 + **双向 TLS**（服务端 `thrift_ssl_client_auth=true`）需额外提供 PEM 客户端证书及其 PKCS#8 私钥 —— 对应 Node.js 的 `sslOptions.cert`/`sslOptions.key`： ```rust diff --git a/src/connection/mod.rs b/src/connection/mod.rs index 69dc3c7..f861d37 100644 --- a/src/connection/mod.rs +++ b/src/connection/mod.rs @@ -24,8 +24,19 @@ use std::io::{Read, Write}; use std::net::{TcpStream, ToSocketAddrs}; +#[cfg(feature = "tls")] +use std::sync::Arc; use std::time::Duration; +#[cfg(feature = "tls")] +use rustls::client::danger::{HandshakeSignatureValid, ServerCertVerified, ServerCertVerifier}; +#[cfg(feature = "tls")] +use rustls::crypto::WebPkiSupportedAlgorithms; +#[cfg(feature = "tls")] +use rustls::pki_types::{CertificateDer, ServerName, UnixTime}; +#[cfg(feature = "tls")] +use rustls::{ClientConfig, ClientConnection, DigitallySignedStruct, SignatureScheme, StreamOwned}; + use thrift::protocol::{ TBinaryInputProtocol, TBinaryOutputProtocol, TCompactInputProtocol, TCompactOutputProtocol, TInputProtocol, TOutputProtocol, @@ -63,9 +74,10 @@ pub enum RpcProtocol { #[derive(Debug, Clone, Default)] pub struct TlsOptions { /// PEM certificate added as a trusted root (e.g. a private CA or the - /// server's self-signed certificate). + /// server's self-signed certificate), in addition to the platform roots. pub ca_cert_path: Option<std::path::PathBuf>, - /// Skip certificate verification entirely (self-signed test certs). + /// Skip certificate-chain and hostname verification (self-signed test + /// certs). TLS handshake signatures are still verified. /// **Dangerous** outside tests. Default `false`. pub accept_invalid_certs: bool, /// Hostname used for SNI + certificate validation instead of the @@ -264,56 +276,155 @@ fn connect_stream(endpoint: &Endpoint, connect_timeout: Duration) -> Result<TcpS #[cfg(feature = "tls")] fn tls_handshake( endpoint: &Endpoint, - stream: TcpStream, + mut stream: TcpStream, tls: &TlsOptions, -) -> Result<native_tls::TlsStream<TcpStream>> { - let mut builder = native_tls::TlsConnector::builder(); - if let Some(path) = &tls.ca_cert_path { - let pem = std::fs::read(path).map_err(|e| { - Error::Client(format!( - "cannot read CA certificate {}: {e}", - path.display() - )) - })?; - let cert = native_tls::Certificate::from_pem(&pem).map_err(Error::Tls)?; - builder.add_root_certificate(cert); - } - if tls.accept_invalid_certs { - builder.danger_accept_invalid_certs(true); +) -> Result<StreamOwned<ClientConnection, TcpStream>> { + let config = tls_client_config(tls)?; + let domain = tls.domain_override.as_deref().unwrap_or(&endpoint.host); + let server_name = ServerName::try_from(domain.to_owned()) + .map_err(|e| Error::Client(format!("invalid TLS server name '{domain}': {e}")))?; + let mut connection = + ClientConnection::new(config, server_name).map_err(|e| Error::Tls(e.to_string()))?; + + connection + .complete_io(&mut stream) + .map_err(|e| Error::Tls(e.to_string()))?; + if connection.is_handshaking() { + return Err(Error::Tls("TLS handshake did not complete".into())); } - match (&tls.client_cert_path, &tls.client_key_path) { + + Ok(StreamOwned::new(connection, stream)) +} + +#[cfg(feature = "tls")] +fn tls_client_config(tls: &TlsOptions) -> Result<Arc<ClientConfig>> { + let provider = Arc::new(rustls::crypto::ring::default_provider()); + let extra_roots = match &tls.ca_cert_path { + Some(path) => load_certificates(path, "CA certificate")?, + None => Vec::new(), + }; + + let verifier: Arc<dyn ServerCertVerifier> = if tls.accept_invalid_certs { + Arc::new(NoCertificateVerification::new( + provider.signature_verification_algorithms, + )) + } else { + Arc::new( + rustls_platform_verifier::Verifier::new_with_extra_roots( + extra_roots, + Arc::clone(&provider), + ) + .map_err(|e| Error::Tls(e.to_string()))?, + ) + }; + + let builder = ClientConfig::builder_with_provider(provider) + .with_safe_default_protocol_versions() + .map_err(|e| Error::Tls(e.to_string()))? + .dangerous() + .with_custom_certificate_verifier(verifier); + + let config = match (&tls.client_cert_path, &tls.client_key_path) { (Some(cert_path), Some(key_path)) => { - let cert = std::fs::read(cert_path).map_err(|e| { - Error::Client(format!( - "cannot read client certificate {}: {e}", - cert_path.display() - )) - })?; - let key = std::fs::read(key_path).map_err(|e| { + let certificates = load_certificates(cert_path, "client certificate")?; + let key_file = std::fs::File::open(key_path).map_err(|e| { Error::Client(format!( "cannot read client key {}: {e}", key_path.display() )) })?; - let identity = native_tls::Identity::from_pkcs8(&cert, &key).map_err(Error::Tls)?; - builder.identity(identity); + let mut key_reader = std::io::BufReader::new(key_file); + let key = rustls_pemfile::pkcs8_private_keys(&mut key_reader) + .next() + .transpose() + .map_err(|e| Error::Tls(format!("cannot parse client key: {e}")))? + .ok_or_else(|| Error::Tls("client key is not a PEM PKCS#8 private key".into()))?; + builder + .with_client_auth_cert(certificates, key.into()) + .map_err(|e| Error::Tls(e.to_string()))? } - (None, None) => {} + (None, None) => builder.with_no_client_auth(), _ => { return Err(Error::Client( "mutual TLS requires both client_cert_path and client_key_path".into(), )) } - } - let connector = builder.build().map_err(Error::Tls)?; - let domain = tls.domain_override.as_deref().unwrap_or(&endpoint.host); - connector.connect(domain, stream).map_err(|e| match e { - native_tls::HandshakeError::Failure(e) => Error::Tls(e), - // Blocking sockets never yield the mid-handshake variant. - native_tls::HandshakeError::WouldBlock(_) => { - Error::Client("TLS handshake interrupted".into()) + }; + + Ok(Arc::new(config)) +} + +#[cfg(feature = "tls")] +fn load_certificates( + path: &std::path::Path, + description: &str, +) -> Result<Vec<CertificateDer<'static>>> { + let file = std::fs::File::open(path) + .map_err(|e| Error::Client(format!("cannot read {description} {}: {e}", path.display())))?; + let mut reader = std::io::BufReader::new(file); + let certificates = rustls_pemfile::certs(&mut reader) + .collect::<std::io::Result<Vec<_>>>() + .map_err(|e| Error::Tls(format!("cannot parse {description}: {e}")))?; + if certificates.is_empty() { + return Err(Error::Tls(format!( + "{description} {} contains no PEM certificates", + path.display() + ))); + } + Ok(certificates) +} + +/// Disables certificate-chain and hostname validation while retaining +/// cryptographic verification of the TLS handshake signatures. +#[cfg(feature = "tls")] +#[derive(Debug)] +struct NoCertificateVerification { + supported_algorithms: WebPkiSupportedAlgorithms, +} + +#[cfg(feature = "tls")] +impl NoCertificateVerification { + fn new(supported_algorithms: WebPkiSupportedAlgorithms) -> Self { + Self { + supported_algorithms, } - }) + } +} + +#[cfg(feature = "tls")] +impl ServerCertVerifier for NoCertificateVerification { + fn verify_server_cert( + &self, + _end_entity: &CertificateDer<'_>, + _intermediates: &[CertificateDer<'_>], + _server_name: &ServerName<'_>, + _ocsp_response: &[u8], + _now: UnixTime, + ) -> std::result::Result<ServerCertVerified, rustls::Error> { + Ok(ServerCertVerified::assertion()) + } + + fn verify_tls12_signature( + &self, + message: &[u8], + cert: &CertificateDer<'_>, + dss: &DigitallySignedStruct, + ) -> std::result::Result<HandshakeSignatureValid, rustls::Error> { + rustls::crypto::verify_tls12_signature(message, cert, dss, &self.supported_algorithms) + } + + fn verify_tls13_signature( + &self, + message: &[u8], + cert: &CertificateDer<'_>, + dss: &DigitallySignedStruct, + ) -> std::result::Result<HandshakeSignatureValid, rustls::Error> { + rustls::crypto::verify_tls13_signature(message, cert, dss, &self.supported_algorithms) + } + + fn supported_verify_schemes(&self) -> Vec<SignatureScheme> { + self.supported_algorithms.supported_schemes() + } } /// A `TlsStream` shared between the read and write transports. @@ -325,15 +436,15 @@ fn tls_handshake( /// read and write never contend. #[cfg(feature = "tls")] #[derive(Clone)] -struct SharedTlsStream(std::sync::Arc<std::sync::Mutex<native_tls::TlsStream<TcpStream>>>); +struct SharedTlsStream(std::sync::Arc<std::sync::Mutex<StreamOwned<ClientConnection, TcpStream>>>); #[cfg(feature = "tls")] impl SharedTlsStream { - fn new(stream: native_tls::TlsStream<TcpStream>) -> Self { + fn new(stream: StreamOwned<ClientConnection, TcpStream>) -> Self { Self(std::sync::Arc::new(std::sync::Mutex::new(stream))) } - fn lock(&self) -> std::sync::MutexGuard<'_, native_tls::TlsStream<TcpStream>> { + fn lock(&self) -> std::sync::MutexGuard<'_, StreamOwned<ClientConnection, TcpStream>> { self.0.lock().unwrap_or_else(|p| p.into_inner()) } } @@ -515,6 +626,7 @@ mod tls_tests { use super::*; use crate::protocol::client::TIClientRPCServiceSyncClient; use std::path::PathBuf; + use std::sync::Arc; fn fixture(name: &str) -> PathBuf { PathBuf::from(env!("CARGO_MANIFEST_DIR")) @@ -522,14 +634,56 @@ mod tls_tests { .join(name) } - /// Spawn a TLS acceptor on a loopback port that completes one handshake - /// and then drops the connection. Uses the checked-in self-signed cert - /// (CN=localhost, SAN DNS:localhost + IP:127.0.0.1, 100-year validity). + fn pkcs8_key(name: &str) -> rustls::pki_types::PrivateKeyDer<'static> { + let file = std::fs::File::open(fixture(name)).expect("read key fixture"); + let mut reader = std::io::BufReader::new(file); + let key = rustls_pemfile::pkcs8_private_keys(&mut reader) + .next() + .expect("PKCS#8 key item") + .expect("parse PKCS#8 key") + .into(); + key + } + + fn server_config(require_client_auth: bool) -> Arc<rustls::ServerConfig> { + let provider = Arc::new(rustls::crypto::ring::default_provider()); + let builder = rustls::ServerConfig::builder_with_provider(Arc::clone(&provider)) + .with_safe_default_protocol_versions() + .expect("protocol versions"); + let builder = if require_client_auth { + let mut roots = rustls::RootCertStore::empty(); + for certificate in + load_certificates(&fixture("client-cert.pem"), "client root").expect("client root") + { + roots.add(certificate).expect("add client root"); + } + let verifier = + rustls::server::WebPkiClientVerifier::builder_with_provider(roots.into(), provider) + .build() + .expect("client verifier"); + builder.with_client_cert_verifier(verifier) + } else { + builder.with_no_client_auth() + }; + let config = builder + .with_single_cert( + load_certificates(&fixture("cert.pem"), "server certificate") + .expect("server certificate"), + pkcs8_key("key.pem"), + ) + .expect("server config"); + Arc::new(config) + } + + /// Spawn a rustls acceptor on a loopback port that completes handshakes + /// and then drops each connection. Uses the checked-in self-signed cert + /// (CN=localhost, SAN DNS:localhost + IP:127.0.0.1). fn tls_acceptor_once() -> Endpoint { - let cert = std::fs::read(fixture("cert.pem")).expect("read cert fixture"); - let key = std::fs::read(fixture("key.pem")).expect("read key fixture"); - let identity = native_tls::Identity::from_pkcs8(&cert, &key).expect("identity"); - let acceptor = native_tls::TlsAcceptor::new(identity).expect("acceptor"); + tls_acceptor(false) + } + + fn tls_acceptor(require_client_auth: bool) -> Endpoint { + let config = server_config(require_client_auth); let listener = std::net::TcpListener::bind("127.0.0.1:0").expect("bind"); let port = listener.local_addr().expect("local_addr").port(); std::thread::spawn(move || { @@ -537,7 +691,11 @@ mod tls_tests { match stream { // Handshake (which may itself fail when the client // rejects our cert — fine, just move on), then drop. - Ok(s) => drop(acceptor.accept(s)), + Ok(mut stream) => { + let mut connection = rustls::ServerConnection::new(Arc::clone(&config)) + .expect("server connection"); + let _ = connection.complete_io(&mut stream); + } Err(_) => break, } } @@ -674,15 +832,12 @@ mod tls_tests { assert!(matches!(err, Error::Tls(_)), "got {err:?}"); } - /// Mutual TLS: a PEM client certificate + PKCS#8 key load into an - /// identity and the handshake completes with the identity configured. - /// `native-tls`'s `TlsAcceptor` has no API to *request* a client - /// certificate, so the loopback server cannot verify it — acceptor-side - /// client-auth is exercised only by the live `IOTDB_TLS_URL` test - /// against a real server. + /// Mutual TLS: a PEM client certificate + PKCS#8 key load into the + /// client config, and a rustls server that requires the fixture client + /// certificate completes the handshake. #[test] fn tls_client_identity_handshake_succeeds() { - let endpoint = tls_acceptor_once(); + let endpoint = tls_acceptor(true); let options = ConnectionOptions { connect_timeout: Duration::from_millis(500), protocol: RpcProtocol::Binary, diff --git a/src/error.rs b/src/error.rs index 52b126c..5cb372f 100644 --- a/src/error.rs +++ b/src/error.rs @@ -31,9 +31,10 @@ pub enum Error { Client(String), /// Malformed binary payload received from the server (e.g. truncated TsBlock). Decode(String), - /// TLS setup or handshake failure (cargo feature `tls`). + /// TLS configuration, certificate parsing, or handshake failure (cargo + /// feature `tls`). #[cfg(feature = "tls")] - Tls(native_tls::Error), + Tls(String), } impl fmt::Display for Error { diff --git a/tests/fixtures/tls/README.md b/tests/fixtures/tls/README.md index 7b5a9e5..fb7eb26 100644 --- a/tests/fixtures/tls/README.md +++ b/tests/fixtures/tls/README.md @@ -25,11 +25,12 @@ Not secrets — the keys never protect anything. | File | Role | |---|---| -| `cert.pem` + `key.pem` | server identity (loopback `TlsAcceptor`, live-server keystore) | +| `cert.pem` + `key.pem` | server identity (loopback rustls server, live-server keystore) | | `client-cert.pem` + `client-key.pem` | client identity for the mutual-TLS tests | -macOS SecureTransport imposes extra requirements even on explicitly -trusted roots: validity ≤ 825 days (error −67901) and an +The macOS verifier used by `rustls-platform-verifier` imposes extra +requirements even on explicitly trusted roots: validity ≤ 825 days +(error −67901) and an `extendedKeyUsage=serverAuth` extension (error −67609). Current cert expires **2028-10-10**; when the trusted-root test starts failing with a validity/expiry error, regenerate: @@ -43,10 +44,9 @@ openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem \ -addext "keyUsage=digitalSignature,keyEncipherment" ``` -The client certificate (expires **2028-10-10** as well) is standalone -self-signed — `cert.pem` cannot act as its issuer because its `keyUsage` -lacks `keyCertSign`; nothing in the tests validates the client chain -anyway (`native-tls`'s `TlsAcceptor` cannot request client certs): +The client certificate (expires **2028-10-10** as well) is a standalone +self-signed test CA and client identity. The rustls loopback server trusts it +as a root and requires it during the mutual-TLS test: ```sh cd tests/fixtures/tls
