[
https://issues.apache.org/jira/browse/RATIS-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18107303#comment-18107303
]
Haonan Hou edited comment on RATIS-2638 at 8/24/26 10:47 AM:
-------------------------------------------------------------
There are two possible implementation approaches:
*Approach 1: A Ratis-specific TLS handshake failure listener*
Add {{TlsHandshakeFailureListener}} and event APIs to {{{}ratis-grpc{}}}, and
let Ratis wrap the shaded gRPC {{ServerCredentials}} and protocol negotiator to
report inbound TLS and ALPN handshake failures. This keeps TLS internals and
shaded gRPC compatibility inside Ratis and provides callers with a simple
callback, but it is more transport-specific and requires a larger Ratis change.
*Approach 2: A generic gRPC server builder creation hook*
Add a default method to {{GrpcServices.Customizer}} that creates the
{{NettyServerBuilder}} from the server address and {{SslContext}} before Ratis
applies its options and registers services. The default implementation
preserves the existing behavior. Callers may override this hook to provide
custom {{ServerCredentials}} and observe TLS handshake failures. This keeps the
Ratis change small and general-purpose, but the caller must maintain the shaded
gRPC protocol-negotiator integration across upgrades.
PR #1561 currently implements Approach 1. If the community prefers the more
generic server builder creation hook described in Approach 2, I can update the
PR accordingly.
was (Author: haonan):
There are two possible implementation approaches:
*Approach 1: A Ratis-specific TLS handshake failure listener*
Add {{TlsHandshakeFailureListener}} and event APIs to {{{}ratis-grpc{}}}, and
let Ratis wrap the shaded gRPC {{ServerCredentials}} and protocol negotiator to
report inbound TLS and ALPN handshake failures. This keeps TLS internals and
shaded gRPC compatibility inside Ratis and provides callers with a simple
callback, but it is more transport-specific and requires a larger Ratis change.
*Approach 2: A generic gRPC server builder creation hook*
Add a default method to {{GrpcServices.Customizer}} that creates the
{{NettyServerBuilder}} from the server address and {{SslContext}} before Ratis
applies its options and registers services. The default implementation
preserves the existing behavior. Callers may override this hook to provide
custom {{ServerCredentials}} and observe TLS handshake failures. This keeps the
Ratis change small and general-purpose, but the caller must maintain the shaded
gRPC protocol-negotiator integration across upgrades.
PR #1561 currently implements Approach 2. If the community prefers Approach 1
and considers a first-class TLS handshake failure listener appropriate for
Ratis, I can update the PR accordingly.
> Add a TLS handshake failure listener for gRPC transports
> --------------------------------------------------------
>
> Key: RATIS-2638
> URL: https://issues.apache.org/jira/browse/RATIS-2638
> Project: Ratis
> Issue Type: Improvement
> Components: gRPC
> Reporter: Haonan Hou
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Motivation
> Applications embedding Ratis currently cannot reliably observe a gRPC TLS
> handshake failure.
> The failure is usually exposed later through an RPC as a
> StatusRuntimeException. This does not reliably provide:
> * the original TLS handshake exception;
> * the local socket address;
> * the remote socket address;
> * whether the failure occurred on an inbound or outbound connection.
> This information is needed by applications that audit failures to establish
> trusted communication channels.
> CC:2022 Part 2, section 18.2.4 (FTP_ITC.1), identifies the following as
> minimum-level auditable information when FAU_GEN is included:
> * failure of trusted channel functions;
> * identification of the initiator and target of failed trusted channel
> functions.
> Reference:
> https://www.commoncriteriaportal.org/files/ccfiles/CC2022PART2R1.pdf
> For an inbound connection, the remote endpoint is the initiator and the local
> endpoint is the target. For an outbound connection, the local endpoint is the
> initiator and the remote endpoint is the target.
> Ratis itself does not need to implement application-specific audit logging.
> It only needs to expose enough transport information for an embedding
> application to generate the audit event.
> h3. Current limitations
> ServerInterceptor and ClientInterceptor cannot observe this failure because a
> TLS handshake may fail before an RPC is created.
> ServerTransportFilter also does not provide the original TLS handshake
> failure in a form suitable for this use case.
> Inferring TLS failure from StatusRuntimeException messages is unreliable and
> does not provide the server-side remote socket address.
> h3. Proposed API
> Ratis could provide an optional listener for TLS handshake failures.
> For example:
> {code:java}
> @FunctionalInterface
> public interface TlsHandshakeFailureListener {
> void onFailure(TlsHandshakeFailureEvent event);
> }
> {code}
> {code:java}
> public interface TlsHandshakeFailureEvent {
> Throwable getCause();
> SocketAddress getLocalAddress();
> SocketAddress getRemoteAddress();
> boolean isInbound();
> }
> {code}
> The exact API shape is open for discussion.
> Since this is specific to the gRPC transport, the listener and event could be
> placed in the ratis-grpc module. The listener could be registered through
> Parameters or GrpcConfigKeys instead of being stored with certificate and key
> material in GrpcTlsConfig.
> h3. Required semantics
> The notification should have the following behavior:
> * Report only an initial TLS handshake failure.
> * Preserve the original Throwable and its cause chain.
> * Provide addresses from the actual physical connection.
> * Report at most once per physical connection.
> * A new failed reconnect may generate another event.
> * Do not report RPC failures, RPC timeouts, connection refusal, HTTP/2 errors,
> normal channel close, or successful TLS handshakes.
> * An exception thrown by the listener must not replace the TLS exception or
> change the channel closing behavior.
> * The listener execution context should be documented.
> * If invoked on a Netty event-loop thread, the listener must perform only
> non-blocking work, such as enqueueing the event for another thread.
> * No private key, password, trust store content, or other credential material
> should be exposed.
> * Existing behavior should remain unchanged when no listener is configured.
> h3. Connection paths
> The notification should cover TLS handshakes for all Ratis gRPC connection
> paths, including:
> * SERVER, CLIENT and ADMIN server listeners, whether separate or sharing the
> same port;
> * RaftClient connections;
> * GrpcServerProtocolClient peer connections;
> * heartbeat connections;
> * GrpcStubPool connections.
> The event does not need to identify SERVER, CLIENT or ADMIN because multiple
> services may share one physical listener or ManagedChannel, and the TLS
> handshake may fail before a specific RPC service is selected.
> The event also does not need to provide a RaftPeerId. On an inbound connection
> that fails during TLS negotiation, the peer identity may not yet be known
> reliably.
> h3. Implementation considerations
> The preferred failure signal is the Netty SslHandler handshake future:
> {code:java}
> sslHandler.handshakeFuture().addListener(future -> {
> if (!future.isSuccess()) {
> Throwable cause = future.cause();
> SocketAddress localAddress = channel.localAddress();
> SocketAddress remoteAddress = channel.remoteAddress();
> // Notify the application listener.
> }
> });
> {code}
> The listener should be attached to the SslHandler created for each physical
> connection. This ensures that a shared SslContext still produces
> connection-specific local and remote addresses.
> A handshake future may also fail because the channel was closed before the
> handshake completed. To avoid false reports, the implementation should verify
> that the cause chain contains an SSLException. This includes errors such as
> SSLHandshakeException and NotSslRecordException. A ClosedChannelException by
> itself should not be reported as a TLS handshake failure.
> A Ratis-only implementation may be possible by decorating the shaded gRPC
> protocol negotiator and observing the SslHandler added to each Netty pipeline.
> This would avoid changing ratis-thirdparty, although it may depend on shaded
> gRPC internal APIs.
> If using those internal APIs is considered too fragile, a stable handshake
> listener extension in ratis-thirdparty could be considered instead.
> A Ratis-only implementation would be preferred if it can preserve the
> existing TLS executor and protocol negotiation behavior.
> h3. Compatibility
> The feature should be opt-in.
> When no listener is configured:
> * existing GrpcTlsConfig constructors should remain compatible;
> * existing server and channel creation should remain unchanged;
> * no audit-specific dependencies should be introduced;
> * there should be no additional application-visible behavior.
> h3. Suggested tests
> The tests should use real gRPC/Netty TLS handshakes and cover:
> * successful TLS handshake does not invoke the listener;
> * client does not trust the server certificate;
> * mTLS server does not trust the client certificate;
> * mTLS client does not provide a certificate;
> * plaintext client connects to a TLS server port;
> * listener throws an exception;
> * local and remote addresses are correct;
> * inbound and outbound direction is correct;
> * server, client, admin, heartbeat and pooled connection paths;
> * reconnect reports once for each new physical connection;
> * one physical connection never reports the same handshake failure twice.
> I can contribute an implementation and integration tests after confirming the
> preferred API and implementation approach.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
