Gabriel39 opened a new pull request, #70693:
URL: https://github.com/apache/doris/pull/70693

   ### Background
   
   FE sends query fragments to BE using gRPC/BRPC. BE retrieves lazy external 
scan splits from FE over Thrift. OSS implementations expose TLS provider 
interfaces, while their private clients use plaintext and their servers reject 
TLS activation.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: #70675 (separate encrypted table/cache ownership hardening)
   PR Type: Enhancement
   
   Problem Summary:
   
   OSS deployments need authenticated, encrypted internal RPC before 
transporting private file-key descriptors. Implement mutual TLS for fragment 
channels and blocking Thrift clients/listeners, using the existing TLS 
configuration and explicit protocol exclusions. Preserve the original certified 
hostname after DNS resolution, keep reconnects encrypted, and bound 
authentication IO independently of normal RPC read timeouts. Connection and 
server lifecycle tests also cover failed initialization and repeated cleanup.
   
   ### Details
   
   - **Change map:** 930 changed production lines, 802 test/fixture lines, 97 
documentation lines, and no generated files. Native TLS setup and bounded 
sockets live in `private_rpc_tls`; Java contexts and Thrift transports live in 
`tls`; OSS providers select them, and the shared Thrift server accepts a 
provider listener. Tests use ephemeral OpenSSL identities and real listeners.
   - **Before and after:** Included private protocols now require a trusted 
peer certificate and encrypt their traffic. Authentication failures propagate 
to callers. Java and native Thrift reconnects retain TLS, and a peer that 
stalls before authentication cannot inherit an unlimited RPC idle timeout. 
Cleanup handles servers whose TLS validation failed before start, as well as a 
server already joined during explicit shutdown.
   - **Configs and defaults:** `enable_tls=false` stays the default. Private 
TLS requires `verify_fail_if_no_peer_cert`, certificate/key/CA files, static 
refresh interval zero, and a positive Thrift connect timeout. Unsupported SAN 
allowlists and BE private-key passwords are rejected. See 
`docs/private-rpc-tls.md` for public-protocol exclusions, certificate rotation 
and dedicated-cluster-CA requirements. Native BRPC clients authenticate the 
configured CA trust domain; FE gRPC and Thrift clients additionally verify the 
target hostname/IP.
   - **Impact on existing users:** Default plaintext deployments keep their 
transport behavior. Operators can opt into OSS private RPC mutual TLS with 
coordinated node configuration. Existing encrypted Iceberg SQL 
capability/admission checks remain in place.
   
   ### Testing
   
   Local verification used the transport patch together with the separate 
cache/lifetime hardening in #70675. All 17 transport files are identical on 
this independent PR branch, and their preimages are unchanged by that cache 
patch.
   
   - Local repository `run-fe-ut.sh`: 11 tests passed, including real 
gRPC/Thrift authentication, hostname and CA rejection, plaintext control, 
reconnects in both transport modes, and stalled-handshake cleanup.
   - Local repository `run-be-ut.sh`: 158 ASAN tests in 11 suites passed, 
including 11 TLS tests and native encrypted Parquet/data/delete/cancellation 
cases. New lifecycle tests reproduced null cleanup and repeated-join failures 
before the fixes; stalled-peer tests failed before bounded authentication was 
added.
   - Full ASAN `build.sh --be --fe` passed with FE Checkstyle; clang-format 16, 
header hygiene and clang-tidy passed on the affected C++ files. The shipped BE 
ELF has no GLIBC requirement newer than 2.17.
   - An isolated local FE/BE cluster completed authenticated heartbeats, a BE 
fragment query returning count 100 and sum 4950, an HMS data scan returning IDs 
1–4, and a lazy V2 delete scan returning 1,3,4,6,8. The FE `fetchSplitBatch` 
RPC counter confirmed BE-to-FE split fetches.
   - All four live private RPC endpoints accepted trusted identities and 
rejected missing/untrusted identities, wrong CA, wrong hostname and plaintext. 
Trusted connections succeeded again after rejection checks.
   - Independent SDK 1.12.0 and Spark reference reads agreed on historical 
snapshots with real encrypted manifests, manifest lists and file keys. Default 
encrypted SQL still rejected admission with `READ_DISABLED`.
   
   ### Release note
   
   OSS deployments can enable mutual TLS for internal gRPC/BRPC and blocking 
Thrift using the existing TLS settings. Private RPC mutual authentication, 
static certificate configuration and explicit exclusions for unsupported public 
protocols are required. The default remains disabled.
   
   ### Check List (For Author)
   
   - Behavior changed: Yes (see Details)
   - Does this need documentation: Yes; operator and trust-boundary notes are 
included in `docs/private-rpc-tls.md`.
   - Pre-submission status: Ready for transport review; encrypted SQL 
capability negotiation and broader encrypted-read acceptance are separate work.
   
   ### AI Disclosure
   
   - Tool: Codex (GPT-6)
   - Generated: Implementation, tests, fixture script, documentation and this 
description.
   - Human review: Pending maintainer review; agent self-review and local 
verification completed.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to