Repository: kudu Updated Branches: refs/heads/master 72b2ffdac -> 0a4541bba
TLS-negotiation [8/n]: TLS negotiation This commit adds TLS negotiation to the KRPC protocol, and removes the existing non-negotiated SSL support. Change-Id: I4f7d36ed8ecf6067a3043344557f45ebd3cdcf9d Reviewed-on: http://gerrit.cloudera.org:8080/5762 Reviewed-by: Alexey Serbin <[email protected]> Reviewed-by: Todd Lipcon <[email protected]> Tested-by: Kudu Jenkins Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/0a4541bb Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/0a4541bb Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/0a4541bb Branch: refs/heads/master Commit: 0a4541bbad3817be568b9bc95604d46a0318f7bd Parents: 72b2ffd Author: Dan Burkert <[email protected]> Authored: Fri Jan 20 16:15:18 2017 -0800 Committer: Todd Lipcon <[email protected]> Committed: Thu Jan 26 18:17:46 2017 +0000 ---------------------------------------------------------------------- docs/design-docs/rpc.md | 257 +++++++++++++++++++------------- src/kudu/rpc/client_negotiation.cc | 73 ++++++++- src/kudu/rpc/client_negotiation.h | 19 +++ src/kudu/rpc/connection.cc | 1 - src/kudu/rpc/constants.cc | 5 +- src/kudu/rpc/messenger.cc | 24 +-- src/kudu/rpc/messenger.h | 17 ++- src/kudu/rpc/negotiation.cc | 21 +-- src/kudu/rpc/reactor.cc | 16 +- src/kudu/rpc/rpc_header.proto | 18 ++- src/kudu/rpc/server_negotiation.cc | 68 ++++++++- src/kudu/rpc/server_negotiation.h | 19 +++ src/kudu/security/CMakeLists.txt | 2 - src/kudu/security/ssl_factory.cc | 121 --------------- src/kudu/security/ssl_factory.h | 65 -------- src/kudu/security/ssl_socket.cc | 179 ---------------------- src/kudu/security/ssl_socket.h | 57 ------- 17 files changed, 372 insertions(+), 590 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/docs/design-docs/rpc.md ---------------------------------------------------------------------- diff --git a/docs/design-docs/rpc.md b/docs/design-docs/rpc.md index 5c3608f..c565b30 100644 --- a/docs/design-docs/rpc.md +++ b/docs/design-docs/rpc.md @@ -12,15 +12,12 @@ See the License for the specific language governing permissions and limitations under the License. --> -=============================================================================== -RPC -=============================================================================== +# RPC + +## Intro -------------------------------------------------------------------------------- -Intro -------------------------------------------------------------------------------- The RPC layer makes communication with remote processes look like local -function calls. You can make either asynchronous calls, in which you provide a +function calls. You can make either asynchronous calls, in which you provide a callback which is invoked later, or synchronous calls, where your thread blocks until the remote system responds. @@ -35,9 +32,8 @@ We use protocol buffers for serialization, and libev for non-blocking I/O. For some code examples, look in rpc-test.cc and rpc_stub-test. -------------------------------------------------------------------------------- -Overview -------------------------------------------------------------------------------- +## Overview + ``` +------------------------------------+ | AcceptorPool | @@ -70,19 +66,18 @@ Overview | handles new inbound RPCs | +------------------------------------+ ``` + Each reactor has a thread which uses epoll to handle many sockets using -non-blocking I/O. Blocking calls are implemented by the Proxy using +non-blocking I/O. Blocking calls are implemented by the Proxy using non-blocking calls-- from the point of view of the Messenger, all calls are non-blocking. -The acceptor pool and the service pool are optional components. If you don't +The acceptor pool and the service pool are optional components. If you don't expect anyone to be connecting to you, you do not have to start them. If a server expects to listen on multiple ports (eg for different protocols), multiple AcceptorPools may be attached. -------------------------------------------------------------------------------- -Proxy classes -------------------------------------------------------------------------------- +## Proxy classes Proxy classes are used by the client to send calls to a remote service. Calls may be made synchronously or asynchronously -- the synchronous calls are simply @@ -120,9 +115,7 @@ Please see the accompanying documentation in the Proxy and RpcController classes for more information on the specific API, as well as the test cases in rpc-test.cc for example usage. -------------------------------------------------------------------------------- -Generated Code -------------------------------------------------------------------------------- +## Generated Code In general, clients will use auto-generated subclasses of Proxy and ServiceIf to get additional type safety and nicer APIs. @@ -155,19 +148,17 @@ and not much else. See rpc/rpc-test-base.h for an example service implementation, as well as the documentation comments in rpc/service_if.h. -------------------------------------------------------------------------------- -ServiceIf classes -------------------------------------------------------------------------------- +## `ServiceIf` classes + ServiceIf classes are abstract interfaces that the server implements to handle -incoming RPCs. In general, each generated service has several virtual methods +incoming RPCs. In general, each generated service has several virtual methods which you can override in order to implement the relevant function call. There is a ServicePool which you can use to coordinate several worker threads handling callbacks. -------------------------------------------------------------------------------- -Exactly once semantics -------------------------------------------------------------------------------- +## Exactly once semantics + Exactly once semantics can be enabled for RPCs that require them by using the 'track_rpc_result' option when declaring a service interface method. @@ -193,9 +184,8 @@ the lifetime of the server, making sure that responses survive crashes and are also available in replicas (for replicated RPCs, like writes) needs that actions be taken outside of the RPC subsystem. -------------------------------------------------------------------------------- -RPC Sidecars -------------------------------------------------------------------------------- +## RPC Sidecars + RPC sidecars are used to avoid excess copies for large volumes of data. Prior to RPC sidecars, the sequence of steps for creating an RPC response on the server side would be as follows: @@ -217,6 +207,7 @@ protobuf), the sidecar's data is directly written to the socket. The data is appended directly after the main message protobuf. Here's what a typical message looks like without sidecars: + ``` +------------------------------------------------+ | Total message length (4 bytes) | @@ -230,11 +221,13 @@ a typical message looks like without sidecars: | Main message protobuf | +------------------------------------------------+ ``` + In this case, the main message length is equal to the protobuf's byte size. Since there are no sidecars, the header protobuf's sidecar_offsets list will will be empty. Here's what it looks like with the sidecars: + ``` +------------------------------------------------+ | Total message length (4 bytes) | @@ -256,6 +249,7 @@ Here's what it looks like with the sidecars: | ... | +------------------------------------------------+ ``` + When there are sidecars, the sidecar_offsets member in the header will be a nonempty list, whose values indicate the offset, measured from the beginning of the main message protobuf, of the start of each sidecar. The number @@ -268,16 +262,14 @@ sidecar array is correct and in-bounds. More information is available in rpc/rpc_sidecar.h. -------------------------------------------------------------------------------- -Wire Protocol -------------------------------------------------------------------------------- +## Wire Protocol -Connection establishment and connection header ----------------------------------------------- +### Connection establishment and connection header After the client connects to a server, the client first sends a connection header. The connection header consists of a magic number "hrpc" and three byte flags, for a total of 7 bytes: + ``` +----------------------------------+ | "hrpc" 4 bytes | @@ -292,8 +284,8 @@ for a total of 7 bytes: Currently, the RPC version is 9. The ServiceClass and AuthProtocol fields are unused. -Message framing and request/response headers --------------------------------------------- +### Message framing and request/response headers + Aside from the initial connection header described above, all other messages are serialized as follows: ``` @@ -305,15 +297,15 @@ serialized as follows: - server->client messages use the ResponseHeader protobuf body: varint-prefixed protobuf - - for typical RPC calls, this is the user-specified request or response - protobuf + - for typical RPC calls, this is the user-specified request or response protobuf - for RPC calls which caused an error, the response is an ErrorStatusPB - - during SASL negotiation, this is a SaslMessagePB + - during negotiation, this is a NegotiatePB ``` -Example packet capture --------------------------- +### Example packet capture + An example call (captured with strace on rpc-test.cc) follows: + ``` "\x00\x00\x00\x17" (total_size: 23 bytes to follow) "\x09" RequestHeader varint: 9 bytes @@ -330,79 +322,136 @@ An example call (captured with strace on rpc-test.cc) follows: y: 1303455736 ``` +## Negotiation + +After the initial connection header is sent, negotiation begins. Negotiation +consists of a sequence of request/response messages sent between the client and +server. Each message is of type `NegotiatePB` and includes a `step` field which +identifies the negotiation protocol step. Each `NegotiatePB` is framed as usual +using `RequestHeader` or `ResponseHeader` messages with `call_id` -33. -SASL negotiation ------------------- -After the initial connection header is sent, SASL negotiation begins. -Kudu always uses SASL regardless of security settings. In the case that -no strong authentication is required, SASL PLAIN is used with no password. +The Kudu negotiation protocol allows the client and server to communicate +supported RPC feature flags and supported SASL authentication mechanisms, +initiate an optional TLS handshake, and perform SASL negotiation. -This SASL negotiation protocol matches the Hadoop protocol. -The negotiation proceeds as described in this diagram: +#### Step 1: Negotiate + +The client and server swap RPC feature flags and supported SASL mechanisms. This +step always takes exactly one round trip. + +``` +Client Server + | | + | +----NegotiatePB-----------------------------+ | + | | step = NEGOTIATE | | + | | supported_features = <client RPC features> | ----------------------> | + | | auths = <client SASL mechanisms> | | + | +--------------------------------------------+ | + | | + | +----NegotiatePB-----------------------------+ | + | | step = NEGOTIATE | | + | <---------------------- | supported_features = <server RPC features> | | + | | auths = <server SASL mechanisms> | | + | +--------------------------------------------+ | +``` + +RPC feature flags allow the client and server to know what optional features the +other end can support, for example TLS encryption. There are several advantages +of using feature flags over version numbers for this purpose: + +* since we have both a Java and C++ client, this allows us to add features in + different orders, or decide to not support a feature at all in one client or + the other. For example, the C++ client is likely to gain support for a + shared-memory transport long before the Java one. +* this allows much more flexibility in backporting RPC system features across + versions. For example, if we introduce feature 'A' in Kudu 2.0, and feature + 'B' in Kudu 2.1, we are still able to selectively backport 'B' without 'A' to + Kudu 1.5. +* the set of supported features can be determined by code-level support as well + as conditionally based on configuration or machine capability. + +#### Step 2: TLS Handshake + +If both the server and client support the `TLS` RPC feature flag, the client +initiates a TLS handshake, after which both sides wrap the socket in the TLS +protected channel. If either the client or server does not support the `TLS` +flag, then this step is skipped. This step takes as many round trips as +necessary to complete the TLS handshake. + +``` +Client Server + | | + | +----NegotiatePB------------------------+ | + | | step = TLS_HANDSHAKE | | + | | tls_handshake = <TLS handshake token> | ---------------------------> | + | +---------------------------------------+ | + | | + | +----NegotiatePB------------------------+ | + | | step = TLS_HANDSHAKE | | + | <--------------------------- | tls_handshake = <TLS handshake token> | | + | +---------------------------------------+ | + | | + | <...repeat until TLS handshake is complete...> | ``` - CLIENT | | SERVER - | | -(1) SaslMessagePB } | | -state=NEGOTIATE } --------------------------> | - | | - | | { (2) SaslMessagePB - | | { state=NEGOTIATE - | <------- { auths=<list of supported mechanisms> - | | -(3) SaslMessagePB } | | -state=INITIATE } | | -auths[0]=<chosen mechanism> } | | -token=<challenge response, if any> } ---------> | - | | - | | { (4) SaslMessagePB - | | { state=CHALLENGE (or SUCCESS) - | <------- { token=<challenge token, if applicable> - | | -(5) SaslMessagePB } | | -state=RESPONSE } | | -token=<challenge response> } -----------------> | - | | - | | { GOTO (4) above - | | + +The client and server repeat `TLS_HANDSHAKE` round-trips until the TLS handshake +is complete, at which point both ends wrap their respective TCP socket in the +new TLS channel. All subsequent messages will be encrypted. + +#### Step 3: SASL Negotiation + +The client and server now initiate a SASL handshake. The client is responsible +for choosing which SASL mechanism is used, with the restriction that it must be +in the set of mutually supported SASL mechanisms exchanged in the `NEGOTIATE` +step. Depending on the mechanism, SASL negotiation may serve to authenticate the +client to the server, and vice versa. + +SASL negotiation may take one or more round trips. The first and last messages +are always a `SASL_INITIATE` from the client and a `SASL_SUCCESS` from the +server, respectively. In between `SASL_INITIATE` and `SASL_SUCCESS`, zero or +more pairs of `SASL_CHALLENGE` and `SASL_RESPONSE` messages from server and +client, respectively, may occur depending on the mechanism. ``` -Each of the SaslMessagePBs above is framed as usual using RequestHeader or ResponseHeader -protobufs. For each SASL message, the CallId should be set to '-33'. - -RPC Feature Flags ------------------ - -During connection negotiation the client and server exchange the set of RPC -feature flags, so that subsequent RPCs request and responses are aware of what -is supported. There are several advantages of feature flags over version numbers: - -* since we have both a Java and C++ client, this allows us to add - features in different orders, or decide to not support a feature - at all in one client or the other. For example, the C++ client - is likely to gain support for a shared-memory transport long before - the Java one. -* this allows much more flexibility in backporting RPC system features - across versions. For example, if we introduce feature 'A' in Kudu - 2.0, and feature 'B' in Kudu 2.1, we are still able to selectively - backport 'B' without 'A' to Kudu 1.5. -* currently, the set of supported features is determined only by - code-level support, but we could later decide to conditionally - enable features based on configuration or machine capability. - -Connection Context: ------------------- +Client Server + | | + | +----NegotiatePB----------------+ | + | | step = SASL_INITIATE | | + | | auths[0] = <chosen mechanism> | -----------------------------------> | + | | token = <SASL token> | | + | +-------------------------------+ | + | | + | <...SASL_INITIATE is followed by 0 or more SASL_CHALLENGE + | + | SASL_RESPONSE steps...> | + | | + | +----NegotiatePB--------+ | + | | step = SASL_CHALLENGE | | + | <------------------------------------------- | token = <SASL token> | | + | +-----------------------+ | + | | + | +----NegotiatePB-------+ | + | | step = SASL_RESPONSE | | + | | token = <SASL token> | --------------------------------------------> | + | +----------------------+ | + | | + | +----NegotiatePB------+ | + | <--------------------------------------------- | step = SASL_SUCCESS | | + | +---------------------+ | +``` + +## Connection Context + Once the SASL negotiation is complete, before the first request, the client -sends the server a special call with call_id -3. The body of this call is a +sends the server a special call with `call_id` -3. The body of this call is a ConnectionContextPB. The server should not respond to this call. +## Steady state -Steady state ------------- During steady state operation, the client sends call protobufs prefixed by -RequestHeader protobufs. The server sends responses prefixed by ResponseHeader -protobufs. +`RequestHeader` protobufs. The server sends responses prefixed by +`ResponseHeader` protobufs. -The client must send calls in strictly increasing 'call_id' order. The server +The client must send calls in strictly increasing `call_id` order. The server may reject repeated calls or calls with lower IDs. The server's responses may -arrive out-of-order, and use the 'call_id' in the response to associate a response -with the correct call. +arrive out-of-order, and use the `call_id` in the response to associate a +response with the correct call. http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/rpc/client_negotiation.cc ---------------------------------------------------------------------- diff --git a/src/kudu/rpc/client_negotiation.cc b/src/kudu/rpc/client_negotiation.cc index a7c6c63..9210864 100644 --- a/src/kudu/rpc/client_negotiation.cc +++ b/src/kudu/rpc/client_negotiation.cc @@ -39,20 +39,22 @@ #include "kudu/rpc/sasl_common.h" #include "kudu/rpc/sasl_helper.h" #include "kudu/rpc/serialization.h" +#include "kudu/security/tls_context.h" +#include "kudu/security/tls_handshake.h" #include "kudu/util/faststring.h" #include "kudu/util/net/sockaddr.h" #include "kudu/util/net/socket.h" #include "kudu/util/scoped_cleanup.h" #include "kudu/util/trace.h" -namespace kudu { -namespace rpc { - using std::map; using std::set; using std::string; using std::unique_ptr; +namespace kudu { +namespace rpc { + static int ClientNegotiationGetoptCb(ClientNegotiation* client_negotiation, const char* plugin_name, const char* option, @@ -95,6 +97,7 @@ static Status StatusFromRpcError(const ErrorStatusPB& error) { ClientNegotiation::ClientNegotiation(unique_ptr<Socket> socket) : socket_(std::move(socket)), helper_(SaslHelper::CLIENT), + tls_context_(nullptr), negotiated_mech_(SaslMechanism::INVALID), deadline_(MonoTime::Max()) { callbacks_.push_back(SaslBuildCallback(SASL_CB_GETOPT, @@ -133,6 +136,10 @@ void ClientNegotiation::set_server_fqdn(const string& domain_name) { helper_.set_server_fqdn(domain_name); } +void ClientNegotiation::EnableTls(const security::TlsContext* tls_context) { + tls_context_ = DCHECK_NOTNULL(tls_context); +} + void ClientNegotiation::set_deadline(const MonoTime& deadline) { deadline_ = deadline; } @@ -155,7 +162,28 @@ Status ClientNegotiation::Negotiate() { RETURN_NOT_OK(HandleNegotiate(response)); } - // Step 3: SASL negotiation. + // Step 3: if both ends support TLS, do a TLS handshake. + // TODO(dan): allow the client to require TLS. + if (tls_context_ && ContainsKey(server_features_, TLS)) { + RETURN_NOT_OK(tls_context_->InitiateHandshake(security::TlsHandshakeType::CLIENT, + &tls_handshake_)); + + // To initiate the TLS handshake, we pretend as if the server sent us an + // empty TLS_HANDSHAKE token. + NegotiatePB initial; + initial.set_step(NegotiatePB::TLS_HANDSHAKE); + initial.set_tls_handshake(""); + Status s = HandleTlsHandshake(initial); + + while (s.IsIncomplete()) { + NegotiatePB response; + RETURN_NOT_OK(RecvNegotiatePB(&response, &recv_buf)); + s = HandleTlsHandshake(response); + } + RETURN_NOT_OK(s); + } + + // Step 4: SASL negotiation. RETURN_NOT_OK(InitSaslClient()); RETURN_NOT_OK(SendSaslInitiate()); for (bool cont = true; cont; ) { @@ -177,7 +205,7 @@ Status ClientNegotiation::Negotiate() { } } - // Step 4: Send connection context. + // Step 5: Send connection context. RETURN_NOT_OK(SendConnectionContext()); TRACE("Negotiation successful"); @@ -307,6 +335,41 @@ Status ClientNegotiation::HandleNegotiate(const NegotiatePB& response) { return Status::OK(); } +Status ClientNegotiation::SendTlsHandshake(string tls_token) { + TRACE("Sending TLS_HANDSHAKE message to server"); + NegotiatePB msg; + msg.set_step(NegotiatePB::TLS_HANDSHAKE); + msg.mutable_tls_handshake()->swap(tls_token); + return SendNegotiatePB(msg); +} + +Status ClientNegotiation::HandleTlsHandshake(const NegotiatePB& response) { + if (PREDICT_FALSE(response.step() != NegotiatePB::TLS_HANDSHAKE)) { + return Status::NotAuthorized("expected TLS_HANDSHAKE step", + NegotiatePB::NegotiateStep_Name(response.step())); + } + TRACE("Received TLS_HANDSHAKE response from server"); + + if (PREDICT_FALSE(!response.has_tls_handshake())) { + return Status::NotAuthorized("No TLS handshake token in TLS_HANDSHAKE response from server"); + } + + string token; + Status s = tls_handshake_.Continue(response.tls_handshake(), &token); + if (s.IsIncomplete()) { + // Another roundtrip is required to complete the handshake. + RETURN_NOT_OK(SendTlsHandshake(std::move(token))); + } + + // Check that the handshake step didn't produce an error. Will also propagate + // an Incomplete status. + RETURN_NOT_OK(s); + + // TLS handshake is finished. + DCHECK(token.empty()); + return tls_handshake_.Finish(&socket_); +} + Status ClientNegotiation::SendSaslInitiate() { string matching_mechs_str = JoinElements(common_mechs_, " "); TRACE("Matching mech list: $0", matching_mechs_str); http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/rpc/client_negotiation.h ---------------------------------------------------------------------- diff --git a/src/kudu/rpc/client_negotiation.h b/src/kudu/rpc/client_negotiation.h index 6677d33..5ed4467 100644 --- a/src/kudu/rpc/client_negotiation.h +++ b/src/kudu/rpc/client_negotiation.h @@ -28,12 +28,17 @@ #include "kudu/rpc/rpc_header.pb.h" #include "kudu/rpc/sasl_common.h" #include "kudu/rpc/sasl_helper.h" +#include "kudu/security/tls_handshake.h" #include "kudu/util/monotime.h" #include "kudu/util/net/socket.h" #include "kudu/util/status.h" namespace kudu { +namespace security { +class TlsContext; +} + namespace rpc { class NegotiatePB; @@ -88,6 +93,10 @@ class ClientNegotiation { // Must be called before Negotiate(). Required for some mechanisms. void set_server_fqdn(const std::string& domain_name); + // Allow TLS to be used on the connection. 'tls_context' must outlive this + // ClientNegotiation. + void EnableTls(const security::TlsContext* tls_context); + // Set deadline for connection negotiation. void set_deadline(const MonoTime& deadline); @@ -140,6 +149,12 @@ class ClientNegotiation { // Handle NEGOTIATE step response from the server. Status HandleNegotiate(const NegotiatePB& response) WARN_UNUSED_RESULT; + // Send a TLS_HANDSHAKE request message to the server with the provided token. + Status SendTlsHandshake(std::string tls_token) WARN_UNUSED_RESULT; + + // Handle a TLS_HANDSHAKE response message from the server. + Status HandleTlsHandshake(const NegotiatePB& response) WARN_UNUSED_RESULT; + // Send an SASL_INITIATE message to the server. Status SendSaslInitiate() WARN_UNUSED_RESULT; @@ -167,6 +182,10 @@ class ClientNegotiation { gscoped_ptr<sasl_conn_t, SaslDeleter> sasl_conn_; SaslHelper helper_; + // TLS state. + const security::TlsContext* tls_context_; + security::TlsHandshake tls_handshake_; + // Authentication state. std::string plain_auth_user_; std::string plain_pass_; http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/rpc/connection.cc ---------------------------------------------------------------------- diff --git a/src/kudu/rpc/connection.cc b/src/kudu/rpc/connection.cc index 30d3720..d8edd08 100644 --- a/src/kudu/rpc/connection.cc +++ b/src/kudu/rpc/connection.cc @@ -41,7 +41,6 @@ #include "kudu/rpc/rpc_header.pb.h" #include "kudu/rpc/rpc_introspection.pb.h" #include "kudu/rpc/transfer.h" -#include "kudu/security/ssl_socket.h" #include "kudu/util/debug-util.h" #include "kudu/util/flag_tags.h" #include "kudu/util/logging.h" http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/rpc/constants.cc ---------------------------------------------------------------------- diff --git a/src/kudu/rpc/constants.cc b/src/kudu/rpc/constants.cc index ef3175c..7f91a51 100644 --- a/src/kudu/rpc/constants.cc +++ b/src/kudu/rpc/constants.cc @@ -25,8 +25,11 @@ namespace rpc { const char* const kMagicNumber = "hrpc"; const char* const kSaslAppName = "kudu"; const char* const kSaslProtoName = "kudu"; + +// The server supports the TLS flag if there is a TLS certificate available. +// The flag is added during negotiation if this is the case. set<RpcFeatureFlag> kSupportedServerRpcFeatureFlags = { APPLICATION_FEATURE_FLAGS }; -set<RpcFeatureFlag> kSupportedClientRpcFeatureFlags = { APPLICATION_FEATURE_FLAGS }; +set<RpcFeatureFlag> kSupportedClientRpcFeatureFlags = { APPLICATION_FEATURE_FLAGS, TLS }; } // namespace rpc } // namespace kudu http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/rpc/messenger.cc ---------------------------------------------------------------------- diff --git a/src/kudu/rpc/messenger.cc b/src/kudu/rpc/messenger.cc index 5a5ed7a..38c5eaa 100644 --- a/src/kudu/rpc/messenger.cc +++ b/src/kudu/rpc/messenger.cc @@ -43,7 +43,7 @@ #include "kudu/rpc/sasl_common.h" #include "kudu/rpc/server_negotiation.h" #include "kudu/rpc/transfer.h" -#include "kudu/security/ssl_factory.h" +#include "kudu/security/tls_context.h" #include "kudu/util/errno.h" #include "kudu/util/flag_tags.h" #include "kudu/util/metrics.h" @@ -183,7 +183,7 @@ void Messenger::Shutdown() { for (Reactor* reactor : reactors_) { reactor->Shutdown(); } - ssl_factory_.reset(); + tls_context_.reset(); } Status Messenger::AddAcceptorPool(const Sockaddr &accept_addr, @@ -298,15 +298,17 @@ Reactor* Messenger::RemoteToReactor(const Sockaddr &remote) { Status Messenger::Init() { Status status; - ssl_enabled_ = !FLAGS_rpc_ssl_server_certificate.empty() || !FLAGS_rpc_ssl_private_key.empty() - || !FLAGS_rpc_ssl_certificate_authority.empty(); - - if (ssl_enabled_) { - ssl_factory_.reset(new SSLFactory()); - RETURN_NOT_OK(ssl_factory_->Init()); - RETURN_NOT_OK(ssl_factory_->LoadCertificate(FLAGS_rpc_ssl_server_certificate)); - RETURN_NOT_OK(ssl_factory_->LoadPrivateKey(FLAGS_rpc_ssl_private_key)); - RETURN_NOT_OK(ssl_factory_->LoadCertificateAuthority(FLAGS_rpc_ssl_certificate_authority)); + server_tls_enabled_ = !FLAGS_rpc_ssl_server_certificate.empty() + || !FLAGS_rpc_ssl_private_key.empty() + || !FLAGS_rpc_ssl_certificate_authority.empty(); + + // Enable TLS unconditionally for client connections. + tls_context_.reset(new security::TlsContext()); + RETURN_NOT_OK(tls_context_->Init()); + if (server_tls_enabled_) { + RETURN_NOT_OK(tls_context_->LoadCertificate(FLAGS_rpc_ssl_server_certificate)); + RETURN_NOT_OK(tls_context_->LoadPrivateKey(FLAGS_rpc_ssl_private_key)); + RETURN_NOT_OK(tls_context_->LoadCertificateAuthority(FLAGS_rpc_ssl_certificate_authority)); } for (Reactor* r : reactors_) { RETURN_NOT_OK(r->Init()); http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/rpc/messenger.h ---------------------------------------------------------------------- diff --git a/src/kudu/rpc/messenger.h b/src/kudu/rpc/messenger.h index 23b39cb..1d2623f 100644 --- a/src/kudu/rpc/messenger.h +++ b/src/kudu/rpc/messenger.h @@ -17,12 +17,12 @@ #ifndef KUDU_RPC_MESSENGER_H #define KUDU_RPC_MESSENGER_H -#include <memory> #include <stdint.h> -#include <unordered_map> #include <list> +#include <memory> #include <string> +#include <unordered_map> #include <vector> #include <gtest/gtest_prod.h> @@ -39,9 +39,12 @@ namespace kudu { class Socket; -class SSLFactory; class ThreadPool; +namespace security { +class TlsContext; +} + namespace rpc { class AcceptorPool; @@ -182,7 +185,7 @@ class Messenger { void ScheduleOnReactor(const boost::function<void(const Status&)>& func, MonoDelta when); - SSLFactory* ssl_factory() const { return ssl_factory_.get(); } + const security::TlsContext& tls_context() const { return *tls_context_; } ThreadPool* negotiation_pool() const { return negotiation_pool_.get(); } @@ -194,7 +197,7 @@ class Messenger { return name_; } - bool ssl_enabled() const { return ssl_enabled_; } + bool server_tls_enabled() const { return server_tls_enabled_; } bool closing() const { shared_lock<rw_spinlock> l(lock_.get_lock()); @@ -226,7 +229,7 @@ class Messenger { bool closing_; - bool ssl_enabled_; + bool server_tls_enabled_; // Pools which are listening on behalf of this messenger. // Note that the user may have called Shutdown() on one of these @@ -241,7 +244,7 @@ class Messenger { gscoped_ptr<ThreadPool> negotiation_pool_; - gscoped_ptr<SSLFactory> ssl_factory_; + std::unique_ptr<security::TlsContext> tls_context_; std::unique_ptr<RpczStore> rpcz_store_; http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/rpc/negotiation.cc ---------------------------------------------------------------------- diff --git a/src/kudu/rpc/negotiation.cc b/src/kudu/rpc/negotiation.cc index 4b387ca..0f490a2 100644 --- a/src/kudu/rpc/negotiation.cc +++ b/src/kudu/rpc/negotiation.cc @@ -35,7 +35,6 @@ #include "kudu/rpc/rpc_header.pb.h" #include "kudu/rpc/sasl_common.h" #include "kudu/rpc/server_negotiation.h" -#include "kudu/security/ssl_socket.h" #include "kudu/util/errno.h" #include "kudu/util/flag_tags.h" #include "kudu/util/logging.h" @@ -155,18 +154,11 @@ static Status DoClientNegotiation(Connection* conn, MonoTime deadline) { } RETURN_NOT_OK(client_negotiation.EnablePlain(conn->user_credentials().real_user(), "")); + client_negotiation.EnableTls(&conn->reactor_thread()->reactor()->messenger()->tls_context()); client_negotiation.set_deadline(deadline); RETURN_NOT_OK(WaitForClientConnect(client_negotiation.socket(), deadline)); RETURN_NOT_OK(client_negotiation.socket()->SetNonBlocking(false)); - - // Do SSL handshake. - // TODO(dan): This is a messy place to do this. - if (conn->reactor_thread()->reactor()->messenger()->ssl_enabled()) { - SSLSocket* ssl_socket = down_cast<SSLSocket*>(client_negotiation.socket()); - RETURN_NOT_OK(ssl_socket->DoHandshake()); - } - RETURN_NOT_OK(client_negotiation.Negotiate()); RETURN_NOT_OK(DisableSocketTimeouts(client_negotiation.socket())); @@ -192,16 +184,13 @@ static Status DoServerNegotiation(Connection* conn, const MonoTime& deadline) { } else { RETURN_NOT_OK(server_negotiation.EnablePlain()); } - + if (conn->reactor_thread()->reactor()->messenger()->server_tls_enabled()) { + server_negotiation.EnableTls(&conn->reactor_thread()->reactor()->messenger()->tls_context()); + } server_negotiation.set_deadline(deadline); + RETURN_NOT_OK(server_negotiation.socket()->SetNonBlocking(false)); - // Do SSL handshake. - // TODO(dan): This is a messy place to do this. - if (conn->reactor_thread()->reactor()->messenger()->ssl_enabled()) { - SSLSocket* ssl_socket = down_cast<SSLSocket*>(server_negotiation.socket()); - RETURN_NOT_OK(ssl_socket->DoHandshake()); - } RETURN_NOT_OK(server_negotiation.Negotiate()); RETURN_NOT_OK(DisableSocketTimeouts(server_negotiation.socket())); http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/rpc/reactor.cc ---------------------------------------------------------------------- diff --git a/src/kudu/rpc/reactor.cc b/src/kudu/rpc/reactor.cc index c8f523c..4a78f8c 100644 --- a/src/kudu/rpc/reactor.cc +++ b/src/kudu/rpc/reactor.cc @@ -41,8 +41,6 @@ #include "kudu/rpc/rpc_introspection.pb.h" #include "kudu/rpc/server_negotiation.h" #include "kudu/rpc/transfer.h" -#include "kudu/security/ssl_factory.h" -#include "kudu/security/ssl_socket.h" #include "kudu/util/countdown_latch.h" #include "kudu/util/debug/sanitizer_scopes.h" #include "kudu/util/errno.h" @@ -342,12 +340,7 @@ Status ReactorThread::FindOrStartConnection(const ConnectionId& conn_id, bool connect_in_progress; RETURN_NOT_OK(StartConnect(&sock, conn_id.remote(), &connect_in_progress)); - std::unique_ptr<Socket> new_socket; - if (reactor()->messenger()->ssl_enabled()) { - new_socket = reactor()->messenger()->ssl_factory()->CreateSocket(sock.Release(), false); - } else { - new_socket.reset(new Socket(sock.Release())); - } + std::unique_ptr<Socket> new_socket(new Socket(sock.Release())); // Register the new connection in our map. *conn = new Connection(this, conn_id.remote(), std::move(new_socket), Connection::CLIENT); @@ -612,12 +605,7 @@ class RegisterConnectionTask : public ReactorTask { void Reactor::RegisterInboundSocket(Socket *socket, const Sockaddr& remote) { VLOG(3) << name_ << ": new inbound connection to " << remote.ToString(); - std::unique_ptr<Socket> new_socket; - if (messenger()->ssl_enabled()) { - new_socket = messenger()->ssl_factory()->CreateSocket(socket->Release(), true); - } else { - new_socket.reset(new Socket(socket->Release())); - } + std::unique_ptr<Socket> new_socket(new Socket(socket->Release())); auto task = new RegisterConnectionTask( new Connection(&thread_, remote, std::move(new_socket), Connection::SERVER)); ScheduleReactorTask(task); http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/rpc/rpc_header.proto ---------------------------------------------------------------------- diff --git a/src/kudu/rpc/rpc_header.proto b/src/kudu/rpc/rpc_header.proto index 26e56ed..e72b92c 100644 --- a/src/kudu/rpc/rpc_header.proto +++ b/src/kudu/rpc/rpc_header.proto @@ -71,6 +71,11 @@ enum RpcFeatureFlag { // The RPC system is required to support application feature flags in the // request and response headers. APPLICATION_FEATURE_FLAGS = 1; + + // The RPC system supports TLS protected connections. If both sides support + // this flag, the connection will automatically be wrapped in a TLS protected + // channel following a TLS handshake. + TLS = 2; }; // Message type passed back & forth for the SASL negotiation. @@ -82,6 +87,7 @@ message NegotiatePB { SASL_INITIATE = 2; SASL_CHALLENGE = 3; SASL_RESPONSE = 4; + TLS_HANDSHAKE = 5; } message SaslAuth { @@ -95,7 +101,7 @@ message NegotiatePB { // When the client sends its NEGOTIATE step message, it sends its set of // supported RPC system features. In the response to this message, the server - // sends back its own. This allows the two peers to agree on whether newer + // sends back its own. This allows the two peers to agree on whether newer // extensions of the RPC system may be used on this connection. We use a list // of features rather than a simple version number to make it easier for the // Java and C++ clients to implement features in different orders while still @@ -104,15 +110,18 @@ message NegotiatePB { repeated RpcFeatureFlag supported_features = 1; // The current negotiation step. - required NegotiateStep step = 2; + required NegotiateStep step = 2; // The SASL token, containing either the challenge during the SASL_CHALLENGE // step, or the response during the SASL_RESPONSE step. - optional bytes token = 3 [(REDACT) = true]; + optional bytes token = 3 [(REDACT) = true]; + + // During the TLS_HANDSHAKE step, contains the TLS handshake message. + optional bytes tls_handshake = 5 [(REDACT) = true]; // During the NEGOTIATE step, contains the supported SASL mechanisms. // During the SASL_INITIATE step, contains the single chosen SASL mechanism. - repeated SaslAuth auths = 4; + repeated SaslAuth auths = 4; } message RemoteMethodPB { @@ -200,7 +209,6 @@ message ResponseHeader { // These offsets are counted AFTER the message header, i.e., offset 0 // is the first byte after the bytes for this protobuf. repeated uint32 sidecar_offsets = 3; - } // Sent as response when is_error == true. http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/rpc/server_negotiation.cc ---------------------------------------------------------------------- diff --git a/src/kudu/rpc/server_negotiation.cc b/src/kudu/rpc/server_negotiation.cc index 837fb47..dad6194 100644 --- a/src/kudu/rpc/server_negotiation.cc +++ b/src/kudu/rpc/server_negotiation.cc @@ -32,6 +32,8 @@ #include "kudu/rpc/blocking_ops.h" #include "kudu/rpc/constants.h" #include "kudu/rpc/serialization.h" +#include "kudu/security/tls_context.h" +#include "kudu/security/tls_handshake.h" #include "kudu/util/net/sockaddr.h" #include "kudu/util/net/socket.h" #include "kudu/util/scoped_cleanup.h" @@ -64,6 +66,7 @@ static int ServerNegotiationPlainAuthCb(sasl_conn_t* conn, ServerNegotiation::ServerNegotiation(unique_ptr<Socket> socket) : socket_(std::move(socket)), helper_(SaslHelper::SERVER), + tls_context_(nullptr), negotiated_mech_(SaslMechanism::INVALID), deadline_(MonoTime::Max()) { callbacks_.push_back(SaslBuildCallback(SASL_CB_GETOPT, @@ -102,6 +105,10 @@ void ServerNegotiation::set_server_fqdn(const string& domain_name) { helper_.set_server_fqdn(domain_name); } +void ServerNegotiation::EnableTls(const security::TlsContext* tls_context) { + tls_context_ = DCHECK_NOTNULL(tls_context); +} + void ServerNegotiation::set_deadline(const MonoTime& deadline) { deadline_ = deadline; } @@ -123,7 +130,21 @@ Status ServerNegotiation::Negotiate() { RETURN_NOT_OK(HandleNegotiate(request)); } - // Step 3: SASL negotiation. + // Step 3: if both ends support TLS, do a TLS handshake. + // TODO(dan): allow the server to require TLS. + if (tls_context_ && ContainsKey(client_features_, TLS)) { + RETURN_NOT_OK(tls_context_->InitiateHandshake(security::TlsHandshakeType::SERVER, + &tls_handshake_)); + while (true) { + NegotiatePB request; + RETURN_NOT_OK(RecvNegotiatePB(&request, &recv_buf)); + Status s = HandleTlsHandshake(request); + if (s.ok()) break; + if (!s.IsIncomplete()) return s; + } + } + + // Step 4: SASL negotiation. RETURN_NOT_OK(InitSaslServer()); { NegotiatePB request; @@ -143,7 +164,7 @@ Status ServerNegotiation::Negotiate() { CHECK(rc == SASL_OK && username != nullptr) << "No username on authenticated connection"; authenticated_user_ = username; - // Step 4: Receive connection context. + // Step 5: Receive connection context. RETURN_NOT_OK(RecvConnectionContext(&recv_buf)); TRACE("Negotiation successful"); @@ -332,10 +353,53 @@ Status ServerNegotiation::SendNegotiate(const set<string>& server_mechs) { response.add_supported_features(feature); } + if (tls_context_) { + response.add_supported_features(TLS); + } + RETURN_NOT_OK(SendNegotiatePB(response)); return Status::OK(); } +Status ServerNegotiation::HandleTlsHandshake(const NegotiatePB& request) { + if (PREDICT_FALSE(request.step() != NegotiatePB::TLS_HANDSHAKE)) { + Status s = Status::NotAuthorized("expected TLS_HANDSHAKE step", + NegotiatePB::NegotiateStep_Name(request.step())); + RETURN_NOT_OK(SendError(ErrorStatusPB::FATAL_UNAUTHORIZED, s)); + return s; + } + + if (PREDICT_FALSE(!request.has_tls_handshake())) { + Status s = Status::NotAuthorized( + "No TLS handshake token in TLS_HANDSHAKE request from client"); + RETURN_NOT_OK(SendError(ErrorStatusPB::FATAL_UNAUTHORIZED, s)); + return s; + } + + string token; + Status s = tls_handshake_.Continue(request.tls_handshake(), &token); + + if (PREDICT_FALSE(!s.IsIncomplete() && !s.ok())) { + RETURN_NOT_OK(SendError(ErrorStatusPB::FATAL_UNAUTHORIZED, s)); + return s; + } + + // Regardless of whether this is the final handshake roundtrip (in which case + // Continue would have returned OK), we still need to return a response. + RETURN_NOT_OK(SendTlsHandshake(std::move(token))); + RETURN_NOT_OK(s); + + // TLS handshake is finished. + return tls_handshake_.Finish(&socket_); +} + +Status ServerNegotiation::SendTlsHandshake(string tls_token) { + NegotiatePB msg; + msg.set_step(NegotiatePB::TLS_HANDSHAKE); + msg.mutable_tls_handshake()->swap(tls_token); + return SendNegotiatePB(msg); +} + Status ServerNegotiation::HandleSaslInitiate(const NegotiatePB& request) { if (PREDICT_FALSE(request.step() != NegotiatePB::SASL_INITIATE)) { Status s = Status::NotAuthorized("expected SASL_INITIATE step", http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/rpc/server_negotiation.h ---------------------------------------------------------------------- diff --git a/src/kudu/rpc/server_negotiation.h b/src/kudu/rpc/server_negotiation.h index 089bc0b..fa6adeb 100644 --- a/src/kudu/rpc/server_negotiation.h +++ b/src/kudu/rpc/server_negotiation.h @@ -28,6 +28,7 @@ #include "kudu/rpc/rpc_header.pb.h" #include "kudu/rpc/sasl_common.h" #include "kudu/rpc/sasl_helper.h" +#include "kudu/security/tls_handshake.h" #include "kudu/util/monotime.h" #include "kudu/util/net/socket.h" #include "kudu/util/status.h" @@ -36,6 +37,10 @@ namespace kudu { class Slice; +namespace security { +class TlsContext; +} + namespace rpc { // Class for doing KRPC negotiation with a remote client over a bidirectional socket. @@ -91,6 +96,10 @@ class ServerNegotiation { // Must be called before Negotiate(). Required for some mechanisms. void set_server_fqdn(const std::string& domain_name); + // Allow TLS to be used on the connection. 'tls_context' must outlive this + // ServerNegotiation. + void EnableTls(const security::TlsContext* tls_context); + // Set deadline for connection negotiation. void set_deadline(const MonoTime& deadline); @@ -146,6 +155,12 @@ class ServerNegotiation { // Send a NEGOTIATE response to the client with the list of available mechanisms. Status SendNegotiate(const std::set<std::string>& server_mechs) WARN_UNUSED_RESULT; + // Handle a TLS_HANDSHAKE request message from the server. + Status HandleTlsHandshake(const NegotiatePB& request) WARN_UNUSED_RESULT; + + // Send a TLS_HANDSHAKE response message to the server with the provided token. + Status SendTlsHandshake(std::string tls_token) WARN_UNUSED_RESULT; + // Handle case when client sends SASL_INITIATE request. // Returns Status::OK if the SASL negotiation is complete, or // Status::Incomplete if a SASL_RESPONSE step is expected. @@ -171,6 +186,10 @@ class ServerNegotiation { gscoped_ptr<sasl_conn_t, SaslDeleter> sasl_conn_; SaslHelper helper_; + // TLS state. + const security::TlsContext* tls_context_; + security::TlsHandshake tls_handshake_; + // The set of features supported by the client. Filled in during negotiation. std::set<RpcFeatureFlag> client_features_; http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/security/CMakeLists.txt ---------------------------------------------------------------------- diff --git a/src/kudu/security/CMakeLists.txt b/src/kudu/security/CMakeLists.txt index ce06fb4..d49f6eb 100644 --- a/src/kudu/security/CMakeLists.txt +++ b/src/kudu/security/CMakeLists.txt @@ -34,8 +34,6 @@ set(SECURITY_SRCS init.cc openssl_util.cc ${PORTED_X509_CHECK_HOST_CC} - ssl_factory.cc - ssl_socket.cc tls_context.cc tls_handshake.cc tls_socket.cc) http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/security/ssl_factory.cc ---------------------------------------------------------------------- diff --git a/src/kudu/security/ssl_factory.cc b/src/kudu/security/ssl_factory.cc deleted file mode 100644 index 2d011bf..0000000 --- a/src/kudu/security/ssl_factory.cc +++ /dev/null @@ -1,121 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -#include "kudu/security/ssl_factory.h" - -#include <mutex> -#include <vector> - -#include <openssl/err.h> -#include <openssl/ssl.h> -#include <openssl/x509v3.h> - -#include "kudu/security/openssl_util.h" -#include "kudu/security/ssl_socket.h" -#include "kudu/util/debug/leakcheck_disabler.h" -#include "kudu/util/mutex.h" -#include "kudu/util/thread.h" - -namespace kudu { - -SSLFactory::SSLFactory() : ctx_(nullptr, SSL_CTX_free) { - security::InitializeOpenSSL(); -} - -SSLFactory::~SSLFactory() { -} - -Status SSLFactory::Init() { - CHECK(!ctx_.get()); - // NOTE: 'SSLv23 method' sounds like it would enable only SSLv2 and SSLv3, but in fact - // this is a sort of wildcard which enables all methods (including TLSv1 and later). - // We explicitly disable SSLv2 and SSLv3 below so that only TLS methods remain. - // See the discussion on https://trac.torproject.org/projects/tor/ticket/11598 for more - // info. - ctx_.reset(SSL_CTX_new(SSLv23_method())); - if (!ctx_) { - return Status::RuntimeError("Could not create SSL context"); - } - SSL_CTX_set_mode(ctx_.get(), SSL_MODE_AUTO_RETRY); - - // Disable SSLv2 and SSLv3 which are vulnerable to various issues such as POODLE. - // We support versions back to TLSv1.0 since OpenSSL on RHEL 6.4 and earlier does not - // not support TLSv1.1 or later. - // - // Disable SSL/TLS compression to free up CPU resources and be less prone - // to attacks exploiting the compression feature: - // https://tools.ietf.org/html/rfc7525#section-3.3 - SSL_CTX_set_options(ctx_.get(), - SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | - SSL_OP_NO_COMPRESSION); - SSL_CTX_set_verify(ctx_.get(), - SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT | SSL_VERIFY_CLIENT_ONCE, nullptr); - return Status::OK(); -} - -std::string SSLFactory::GetLastError(int errno_copy) { - int error_code = ERR_get_error(); - if (error_code == 0) return kudu::ErrnoToString(errno_copy); - const char* error_reason = ERR_reason_error_string(error_code); - if (error_reason != NULL) return error_reason; - return strings::Substitute("SSL error $0", error_code); -} - -Status SSLFactory::LoadCertificate(const std::string& certificate_path) { - ERR_clear_error(); - errno = 0; - if (SSL_CTX_use_certificate_file(ctx_.get(), certificate_path.c_str(), SSL_FILETYPE_PEM) != 1) { - return Status::NotFound( - "Failed to load certificate file '" + certificate_path + "': " + GetLastError(errno)); - } - return Status::OK(); -} - -Status SSLFactory::LoadPrivateKey(const std::string& key_path) { - ERR_clear_error(); - errno = 0; - if (SSL_CTX_use_PrivateKey_file(ctx_.get(), key_path.c_str(), SSL_FILETYPE_PEM) != 1) { - return Status::NotFound( - "Failed to load private key file '" + key_path + "': " + GetLastError(errno)); - } - return Status::OK(); -} - -Status SSLFactory::LoadCertificateAuthority(const std::string& certificate_path) { - ERR_clear_error(); - errno = 0; - if (SSL_CTX_load_verify_locations(ctx_.get(), certificate_path.c_str(), nullptr) != 1) { - return Status::NotFound( - "Failed to load certificate authority file '" + certificate_path + "': " + - GetLastError(errno)); - } - return Status::OK(); -} - -std::unique_ptr<SSLSocket> SSLFactory::CreateSocket(int socket_fd, bool is_server) { - CHECK(ctx_); - // Create SSL object and transfer ownership to the SSLSocket object created. - SSL* ssl = SSL_new(ctx_.get()); - if (ssl == nullptr) { - return nullptr; - } - std::unique_ptr<SSLSocket> socket(new SSLSocket(socket_fd, ssl, is_server)); - return socket; - //return new SSLSocket(socket_fd, ssl, is_server); -} - -} // namespace kudu http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/security/ssl_factory.h ---------------------------------------------------------------------- diff --git a/src/kudu/security/ssl_factory.h b/src/kudu/security/ssl_factory.h deleted file mode 100644 index 02a3afb..0000000 --- a/src/kudu/security/ssl_factory.h +++ /dev/null @@ -1,65 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -#pragma once - -#include <string> -#include <memory> - -#include "kudu/gutil/macros.h" -#include "kudu/gutil/strings/substitute.h" -#include "kudu/security/openssl_util.h" -#include "kudu/util/errno.h" -#include "kudu/util/status.h" - -namespace kudu { - -class Sockaddr; -class SSLSocket; - -class SSLFactory { - public: - SSLFactory(); - - ~SSLFactory(); - - // Set up the SSL_CTX and choose encryption preferences. - Status Init(); - - // Load the server certificate. - Status LoadCertificate(const std::string& certificate_path); - - // Load the private key for the server certificate. - Status LoadPrivateKey(const std::string& key_path); - - // Load the certificate authority. - Status LoadCertificateAuthority(const std::string& certificate_path); - - // Create an SSLSocket wrapped around the file descriptor 'socket_fd'. 'is_server' denotes if it's - // a server socket. The 'socket_fd' is closed when this object is destroyed. - std::unique_ptr<SSLSocket> CreateSocket(int socket_fd, bool is_server); - - private: - friend class SSLSocket; - std::unique_ptr<SSL_CTX, std::function<void(SSL_CTX*)>> ctx_; - - // Gets the last error from the thread local SSL error queue. If no error exists, it returns - // the error corresponding to 'errno_copy'. - static std::string GetLastError(int errno_copy); -}; - -} // namespace kudu http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/security/ssl_socket.cc ---------------------------------------------------------------------- diff --git a/src/kudu/security/ssl_socket.cc b/src/kudu/security/ssl_socket.cc deleted file mode 100644 index c2725c2..0000000 --- a/src/kudu/security/ssl_socket.cc +++ /dev/null @@ -1,179 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -#include "kudu/security/ssl_socket.h" - -#include <errno.h> - -#include <vector> - -#include <openssl/err.h> -#include <openssl/ssl.h> -#include <openssl/x509.h> -#include <openssl/x509v3.h> - -#include "kudu/gutil/strings/split.h" -#include "kudu/gutil/strings/util.h" -#include "kudu/security/ssl_factory.h" -#include "kudu/util/errno.h" -#include "kudu/util/net/sockaddr.h" - -#if OPENSSL_VERSION_NUMBER < 0x10002000L -#include "kudu/security/x509_check_host.h" -#endif // OPENSSL_VERSION_NUMBER - -namespace kudu { - -SSLSocket::SSLSocket(int fd, SSL* ssl, bool is_server) : - Socket(fd), ssl_(ssl), is_server_(is_server) { - SSL_set_fd(ssl_, fd); -} - -SSLSocket::~SSLSocket() { - WARN_NOT_OK(Close(), "unable to close SSL socket in destructor"); -} - -Status SSLSocket::DoHandshake() { - CHECK(ssl_); - ERR_clear_error(); - errno = 0; - int ret; - if (is_server_) { - ret = SSL_accept(ssl_); - } else { - ret = SSL_connect(ssl_); - } - if (ret <= 0) return Status::NetworkError(SSLFactory::GetLastError(errno)); - - // Verify if the handshake was successful. - int rc = SSL_get_verify_result(ssl_); - if (rc != X509_V_OK) { - return Status::NetworkError("SSL_get_verify_result()", X509_verify_cert_error_string(rc)); - } - - // Get the peer certificate. - std::unique_ptr<X509, void(*)(X509*)> cert( - SSL_get_peer_certificate(ssl_), [](X509* x) { X509_free(x); }); - cert.reset(SSL_get_peer_certificate(ssl_)); - if (cert == nullptr) { - if (SSL_get_verify_mode(ssl_) & SSL_VERIFY_FAIL_IF_NO_PEER_CERT) { - return Status::NetworkError("Handshake failed: Could not retreive peer certificate"); - } - } - - // Get the peer's hostname - Sockaddr peer_addr; - if (!GetPeerAddress(&peer_addr).ok()) { - return Status::NetworkError("Handshake failed: Could not retrieve peer address"); - } - std::string peer_hostname; - RETURN_NOT_OK(peer_addr.LookupHostname(&peer_hostname)); - - // Check if the hostname matches with either the Common Name or any of the Subject Alternative - // Names of the certificate. - int match; - if ((match = X509_check_host( - cert.get(), peer_hostname.c_str(), peer_hostname.length(), 0, nullptr)) == 0) { - return Status::NetworkError("Handshake failed: Could not verify host with certificate"); - } - if (match < 0) { - return Status::NetworkError("Handshake failed:", SSLFactory::GetLastError(errno)); - } - CHECK(match == 1); - return Status::OK(); -} - -Status SSLSocket::Write(const uint8_t *buf, int32_t amt, int32_t *nwritten) { - CHECK(ssl_); - ERR_clear_error(); - errno = 0; - int32_t bytes_written = SSL_write(ssl_, buf, amt); - if (bytes_written <= 0) { - if (SSL_get_error(ssl_, bytes_written) == SSL_ERROR_WANT_WRITE) { - // Socket not ready to write yet. - *nwritten = 0; - return Status::OK(); - } - return Status::NetworkError("SSL_write", SSLFactory::GetLastError(errno)); - } - *nwritten = bytes_written; - return Status::OK(); -} - -Status SSLSocket::Writev(const struct ::iovec *iov, int iov_len, - int32_t *nwritten) { - CHECK(ssl_); - ERR_clear_error(); - int32_t total_written = 0; - // Allows packets to be aggresively be accumulated before sending. - RETURN_NOT_OK(SetTcpCork(1)); - Status write_status = Status::OK(); - for (int i = 0; i < iov_len; ++i) { - int32_t frame_size = iov[i].iov_len; - // Don't return before unsetting TCP_CORK. - write_status = Write(static_cast<uint8_t*>(iov[i].iov_base), frame_size, nwritten); - total_written += *nwritten; - if (*nwritten < frame_size) break; - } - RETURN_NOT_OK(SetTcpCork(0)); - *nwritten = total_written; - return write_status; -} - -Status SSLSocket::Recv(uint8_t *buf, int32_t amt, int32_t *nread) { - CHECK(ssl_); - ERR_clear_error(); - errno = 0; - int32_t bytes_read = SSL_read(ssl_, buf, amt); - if (bytes_read <= 0) { - if (bytes_read == 0 && SSL_get_shutdown(ssl_) == SSL_RECEIVED_SHUTDOWN) { - return Status::NetworkError("SSLSocket::Recv() for EOF from remote", Slice(), ESHUTDOWN); - } - if (SSL_get_error(ssl_, bytes_read) == SSL_ERROR_WANT_READ) { - // Nothing available to read yet. - *nread = 0; - return Status::OK(); - } - return Status::NetworkError("SSL_read", SSLFactory::GetLastError(errno)); - } - *nread = bytes_read; - return Status::OK(); -} - -Status SSLSocket::Close() { - if (!ssl_) { - // Socket is already closed. - return Status::OK(); - } - ERR_clear_error(); - errno = 0; - int32_t ret = SSL_shutdown(ssl_); - Status shutdown_status; - if (ret < 0 && errno != EAGAIN) { - // We still need to close the underlying socket, so don't return just yet. - shutdown_status = Status::NetworkError("SSL_Shutdown", SSLFactory::GetLastError(errno)); - } - SSL_free(ssl_); - ssl_ = nullptr; - ERR_remove_state(0); - - Status close_status = Socket::Close(); - if (!close_status.ok()) return close_status.CloneAndPrepend(shutdown_status.message()); - return shutdown_status; -} - -} // namespace kudu http://git-wip-us.apache.org/repos/asf/kudu/blob/0a4541bb/src/kudu/security/ssl_socket.h ---------------------------------------------------------------------- diff --git a/src/kudu/security/ssl_socket.h b/src/kudu/security/ssl_socket.h deleted file mode 100644 index f7570fb..0000000 --- a/src/kudu/security/ssl_socket.h +++ /dev/null @@ -1,57 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -#pragma once - -#include <sys/uio.h> - -#include <string> - -#include "kudu/gutil/macros.h" -#include "kudu/security/openssl_util.h" -#include "kudu/util/net/socket.h" -#include "kudu/util/status.h" - -namespace kudu { - -class Sockaddr; - -class SSLSocket : public Socket { - public: - - SSLSocket(int fd, SSL* ssl, bool is_server); - - ~SSLSocket() override; - - // Do the SSL handshake as a client or a server and verify that the credentials were correctly - // verified. - Status DoHandshake(); - - Status Write(const uint8_t *buf, int32_t amt, int32_t *nwritten) override; - - Status Writev(const struct ::iovec *iov, int iov_len, int32_t *nwritten) override; - - Status Recv(uint8_t *buf, int32_t amt, int32_t *nread) override; - - // Shutdown the connection and free the SSL state for this connection. - Status Close() override; - private: - SSL* ssl_; // Owned. - bool is_server_; -}; - -} // namespace kudu
