This is an automated email from the ASF dual-hosted git repository. szetszwo pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/ratis-hadoop-projects.git
commit 8bdfcedef762448a2c1d7548ed4b9ab6022a653c Author: Josh Elser <[email protected]> AuthorDate: Mon Aug 19 19:22:44 2019 -0400 RATIS-636 Generate website for logservice security doc --- index.xml | 12 +++ logservice.html | 118 +++++++++++++++++++++ logservice/index.html | 1 + logservice/index.xml | 12 +++ logservice/security/index.html | 231 +++++++++++++++++++++++++++++++++++++++++ sitemap.xml | 4 + 6 files changed, 378 insertions(+) diff --git a/index.xml b/index.xml index f03dec4..14092f9 100644 --- a/index.xml +++ b/index.xml @@ -101,6 +101,18 @@ Like Ratis, the LogService is designed to be embedded into another application a </item> <item> + <title>LogService Security</title> + <link>https://ratis.incubator.apache.org/logservice/security/</link> + <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate> + + <guid>https://ratis.incubator.apache.org/logservice/security/</guid> + <description>This document aims to describe what the intended security deployment model of the Ratis LogService. +We will use integration into Apache HBase as an exemplar. +Background TLS is technology capable of giving us &ldquo;strong authentication&rdquo; over network communication. One-way TLS can provide encrypted communication while two-way or &ldquo;mutual&rdquo; TLS can provide encrypted communication and authentication. +One feature of Ratis is that it is decoupled from the RPC transport in use.</description> + </item> + + <item> <title>LogService Testing</title> <link>https://ratis.incubator.apache.org/logservice/testing/</link> <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate> diff --git a/logservice.html b/logservice.html index 47b1fe6..9cc9576 100644 --- a/logservice.html +++ b/logservice.html @@ -233,10 +233,128 @@ daemons provided for the LogService, but these are solely to be used for testing <ul> <li><a href="https://ratis.incubator.apache.org/logservice/testing/";>Testing</a></li> <li><a href="https://ratis.incubator.apache.org/logservice/lifecycle.html";>Log Lifecycle</a></li> +<li><a href="https://ratis.incubator.apache.org/logservice/security/";>Security</a></li> </ul> + <h1><a href="/logservice/security/">LogService Security</a></h1> + <p><small>0001 Jan 1 </small></p> + + + +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<p>This document aims to describe what the intended security deployment model of the Ratis LogService.</p> + +<p>We will use integration into Apache HBase as an exemplar.</p> + +<h2 id="background">Background</h2> + +<p>TLS is technology capable of giving us “strong authentication” over network communication. One-way TLS can provide +encrypted communication while two-way or “mutual” TLS can provide encrypted communication and authentication.</p> + +<p>One feature of Ratis is that it is decoupled from the RPC transport in use. gRPC is the foremost transport, and +can be configured to use one-way or two-way/mutual TLS. gRPC is the only transport for Ratis which +supports TLS today.</p> + +<p>However, the majority of components under the “Hadoop Umbrella” rely on Kerberos to guarantee strong authentication. +In this respect, use of TLS is jarring. However, gRPC does not support SPNEGO (which allows Kerberos authentication) +which all but requires the use of two authentication mechanisms when combining Ratis with other projects (like HBase).</p> + +<p>We anticipate the use of the Ratis LogService as an “embedded WAL” inside of HBase RegionServers and Masters +will result in HBase services using Kerberos authentication to talk to HDFS as well as TLS for Ratis-internal +communication (intra-server Ratis communication and client-server Ratis communication).</p> + +<h2 id="mutual-tls">Mutual TLS</h2> + +<p>Mutual TLS relies on a common certificate authority (CA) to issue all certificates which forms a circle +of trust. Certificates generated by the same CA can be used to set up a mutual TLS connection. A certificate +generated by one CA cannot be used to set up a mutal TLS connection to a service using a certificate +generated by a different CA outside of the circle of trust. [1]</p> + +<p>To control the clients and servers with one instance of the LogService, we want to use a single CA to generate +certificates for clients and servers. We will consider this as an invariant going forward.</p> + +<h2 id="hbase-examplar">HBase Examplar</h2> + +<p>We expect the following material to be provided for every HBase service using Ratis:</p> + +<ul> +<li>File containing an X.509 certificate in PEM format</li> +<li>File containing the PKCS private key in PEM format</li> +<li>File containing the X.509 certificate for the CA</li> +</ul> + +<p>OpenSSL is capable of creating each of these; however, for this document, we will assume +that you already have these pre-made. The server certificate and private key are unique to every +host participating in the HBase cluster. The server certificate and truststore are not sensitive, +but the private key is sensitive and should be protected like a password.</p> + +<p>Every component in HBase using the Ratis LogService would need to ensure that each LogService StateMachine is +configured to use the server keystore and truststore. The LogService state machines would need to constructed +with the appropriate configuration options to specify this TLS material:</p> + +<pre><code class="language-java">RaftProperties properties = ...; + +GrpcConfigKeys.TLS.tlsEnabled(properties); +GrpcConfigKeys.TLS.mutualAuthnEnabled(properties); +properties.set(GrpcConfigKeys.TLS.PRIVATE_KEY_FILE_KEY, "/path/to/server-private-key.pem"); +properties.set(GrpcConfigKeys.TLS.TRUST_STORE_KEY, "/path/to/ca.crt"); +properties.set(GrpcConfigKeys.TLS.CERT_CHAIN_FILE_KEY, "/path/to/server.crt"); + +RaftServer.Builder builder = RaftServer.newBuilder(); +... +builder.setProperties(properties); + +RaftServer server = builder.build(); +</code></pre> + +<p>Clients to the StateMachine would construct a similar configuration:</p> + +<pre><code class="language-java">RaftProperties properties = ...; + +GrpcConfigKeys.TLS.tlsEnabled(properties); +GrpcConfigKeys.TLS.mutualAuthnEnabled(properties); +properties.set(GrpcConfigKeys.TLS.PRIVATE_KEY_FILE_KEY, "/path/to/client-private-key.pem"); +properties.set(GrpcConfigKeys.TLS.TRUST_STORE_KEY, "/path/to/ca.crt"); +properties.set(GrpcConfigKeys.TLS.CERT_CHAIN_FILE_KEY, "/path/to/client.crt"); + +RaftClient.Builder builder = RaftClient.newBuilder(); +... +builder.setProperties(properties); + +RaftClient client = builder.build(); +</code></pre> + +<p>With Mutual TLS, there is no notion of a “client” or “server” only certificate. In the above example code, +as long as the certificate and private key are generated using the same certificate authority, any +should function.</p> + +<p>For the LogService, this client setup would be hidden behind the facade of the LogService client API.</p> + +<p>The HBase WALProvider implementation that uses the Ratis LogService would be providing the location of +this TLS material via the HBase configuration (hbase-site.xml), passing it down into the WALProvider +implementation. As the WALProvider is the broker that doles out readers and writers, and would also, presumably +manage the creation of the StateMachines, it can set up the proper Ratis configuration from the HBase configuration.</p> + +<p>[1] There are scenarios with shared trust across CA’s that enable other scenarios but these are ignored for the purpose +of this document.</p> + + + </ul> </div> diff --git a/logservice/index.html b/logservice/index.html index cb9a68b..6cba5b5 100644 --- a/logservice/index.html +++ b/logservice/index.html @@ -119,6 +119,7 @@ daemons provided for the LogService, but these are solely to be used for testing <ul> <li><a href="https://ratis.incubator.apache.org/logservice/testing/";>Testing</a></li> <li><a href="https://ratis.incubator.apache.org/logservice/lifecycle.html";>Log Lifecycle</a></li> +<li><a href="https://ratis.incubator.apache.org/logservice/security/";>Security</a></li> </ul> </div> diff --git a/logservice/index.xml b/logservice/index.xml index 14447cf..f40c922 100644 --- a/logservice/index.xml +++ b/logservice/index.xml @@ -30,5 +30,17 @@ OPEN This is the first state for a Log which is created in the LogService. A Log Like Ratis, the LogService is designed to be embedded into another application as a library, as opposed to a standalone daemon.</description> </item> + <item> + <title>LogService Security</title> + <link>https://ratis.incubator.apache.org/logservice/security/</link> + <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate> + + <guid>https://ratis.incubator.apache.org/logservice/security/</guid> + <description>This document aims to describe what the intended security deployment model of the Ratis LogService. +We will use integration into Apache HBase as an exemplar. +Background TLS is technology capable of giving us &ldquo;strong authentication&rdquo; over network communication. One-way TLS can provide encrypted communication while two-way or &ldquo;mutual&rdquo; TLS can provide encrypted communication and authentication. +One feature of Ratis is that it is decoupled from the RPC transport in use.</description> + </item> + </channel> </rss> \ No newline at end of file diff --git a/logservice/security/index.html b/logservice/security/index.html new file mode 100644 index 0000000..ea4ebef --- /dev/null +++ b/logservice/security/index.html @@ -0,0 +1,231 @@ + +<!DOCTYPE html> + +<html> + +<head> + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <meta name="description" content="Open source Java implementation for RAFT consensus protocol."> + <meta name="keywords" content="raft, java, ratis, library"/> + <meta name="robots" content="index,follow"/> + <meta name="language" content="en"/> + + <title>Apache Ratis</title> + + <base href="https://ratis.incubator.apache.org/";> + + <link rel="canonical" href="http://ratis.incubator.apache.org/";> + + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css"; + integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" crossorigin="anonymous"> + <link rel="stylesheet" href="style.css"> + +</head> + +<body> + + +<div class="topnav"> + <div class="container"> + <ul class="breadcrumb col-md-6"> + <li> + <img class="asf-logo" src="asf_feather.png" alt="ASF feather"/> + <a href="https://www.apache.org";>Apache Software Foundation</a> + </li> + <li> + <img class="asf-logo" src="logo-white.png" alt="Ratis logo"/> + <a href="https://ratis.incubator.apache.org/";>Apache Ratis™</a> + </li> + </ul> + <div class="col-md-6"> + <ul class="pull-right breadcrumb"> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + </div> + + <nav class="navbar navbar-default navbar-static-top" role="navigation"> + <div class="container"> + <div class="navbar-header"> + <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" + data-target="#ratis-menu" aria-expanded="false"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + </div> + + <div id="ratis-menu" class="collapse navbar-collapse"> + <ul class="nav navbar-nav navbar-right"> + <li><a href="#download">Download</a></li> + <li><a href="#gettingstarted">Getting started</a></li> + <li><a href="#source">Source</a></li> + <li><a href="#community">Community</a></li> + <li><a href="#resources">Resources</a></li> + </ul> + </div> + + + <div class="jumbotron"> + <h1> + Open source Java implementation for Raft consensus protocol. + </h1> + <p> + <a class="btn btn-default download" role="button" href="#download">Download Apache Ratis</a> + </p> + <p> + + <a class="changelog" + href="#gettingstarted">Getting started + </a> + </p> + </div> + </div> + + </nav> +</div> + +<div class="container"> +<h1>LogService Security</h1> + + +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<p>This document aims to describe what the intended security deployment model of the Ratis LogService.</p> + +<p>We will use integration into Apache HBase as an exemplar.</p> + +<h2 id="background">Background</h2> + +<p>TLS is technology capable of giving us “strong authentication” over network communication. One-way TLS can provide +encrypted communication while two-way or “mutual” TLS can provide encrypted communication and authentication.</p> + +<p>One feature of Ratis is that it is decoupled from the RPC transport in use. gRPC is the foremost transport, and +can be configured to use one-way or two-way/mutual TLS. gRPC is the only transport for Ratis which +supports TLS today.</p> + +<p>However, the majority of components under the “Hadoop Umbrella” rely on Kerberos to guarantee strong authentication. +In this respect, use of TLS is jarring. However, gRPC does not support SPNEGO (which allows Kerberos authentication) +which all but requires the use of two authentication mechanisms when combining Ratis with other projects (like HBase).</p> + +<p>We anticipate the use of the Ratis LogService as an “embedded WAL” inside of HBase RegionServers and Masters +will result in HBase services using Kerberos authentication to talk to HDFS as well as TLS for Ratis-internal +communication (intra-server Ratis communication and client-server Ratis communication).</p> + +<h2 id="mutual-tls">Mutual TLS</h2> + +<p>Mutual TLS relies on a common certificate authority (CA) to issue all certificates which forms a circle +of trust. Certificates generated by the same CA can be used to set up a mutual TLS connection. A certificate +generated by one CA cannot be used to set up a mutal TLS connection to a service using a certificate +generated by a different CA outside of the circle of trust. [1]</p> + +<p>To control the clients and servers with one instance of the LogService, we want to use a single CA to generate +certificates for clients and servers. We will consider this as an invariant going forward.</p> + +<h2 id="hbase-examplar">HBase Examplar</h2> + +<p>We expect the following material to be provided for every HBase service using Ratis:</p> + +<ul> +<li>File containing an X.509 certificate in PEM format</li> +<li>File containing the PKCS private key in PEM format</li> +<li>File containing the X.509 certificate for the CA</li> +</ul> + +<p>OpenSSL is capable of creating each of these; however, for this document, we will assume +that you already have these pre-made. The server certificate and private key are unique to every +host participating in the HBase cluster. The server certificate and truststore are not sensitive, +but the private key is sensitive and should be protected like a password.</p> + +<p>Every component in HBase using the Ratis LogService would need to ensure that each LogService StateMachine is +configured to use the server keystore and truststore. The LogService state machines would need to constructed +with the appropriate configuration options to specify this TLS material:</p> + +<pre><code class="language-java">RaftProperties properties = ...; + +GrpcConfigKeys.TLS.tlsEnabled(properties); +GrpcConfigKeys.TLS.mutualAuthnEnabled(properties); +properties.set(GrpcConfigKeys.TLS.PRIVATE_KEY_FILE_KEY, "/path/to/server-private-key.pem"); +properties.set(GrpcConfigKeys.TLS.TRUST_STORE_KEY, "/path/to/ca.crt"); +properties.set(GrpcConfigKeys.TLS.CERT_CHAIN_FILE_KEY, "/path/to/server.crt"); + +RaftServer.Builder builder = RaftServer.newBuilder(); +... +builder.setProperties(properties); + +RaftServer server = builder.build(); +</code></pre> + +<p>Clients to the StateMachine would construct a similar configuration:</p> + +<pre><code class="language-java">RaftProperties properties = ...; + +GrpcConfigKeys.TLS.tlsEnabled(properties); +GrpcConfigKeys.TLS.mutualAuthnEnabled(properties); +properties.set(GrpcConfigKeys.TLS.PRIVATE_KEY_FILE_KEY, "/path/to/client-private-key.pem"); +properties.set(GrpcConfigKeys.TLS.TRUST_STORE_KEY, "/path/to/ca.crt"); +properties.set(GrpcConfigKeys.TLS.CERT_CHAIN_FILE_KEY, "/path/to/client.crt"); + +RaftClient.Builder builder = RaftClient.newBuilder(); +... +builder.setProperties(properties); + +RaftClient client = builder.build(); +</code></pre> + +<p>With Mutual TLS, there is no notion of a “client” or “server” only certificate. In the above example code, +as long as the certificate and private key are generated using the same certificate authority, any +should function.</p> + +<p>For the LogService, this client setup would be hidden behind the facade of the LogService client API.</p> + +<p>The HBase WALProvider implementation that uses the Ratis LogService would be providing the location of +this TLS material via the HBase configuration (hbase-site.xml), passing it down into the WALProvider +implementation. As the WALProvider is the broker that doles out readers and writers, and would also, presumably +manage the creation of the StateMachines, it can set up the proper Ratis configuration from the HBase configuration.</p> + +<p>[1] There are scenarios with shared trust across CA’s that enable other scenarios but these are ignored for the purpose +of this document.</p> + +</div> + +<footer> + <div class="container"> + + <div class="col-md-12 trademark"> + <p>© 2019 <a href="http://apache.org";>The Apache Software Foundation</a>, + Apache, Apache Ratis, the Apache feather logo, Apache Ratis logo, Apache Incubator logo are trademarks of The Apache Software Foundation. + <p> + </div> + </div> +</footer> + + +<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js";></script> +<script src="./js/underscore-min.js"></script> +<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"; + integrity="sha384-0mSbJDEHialfmuBBQP6A4Qrprq5OVfW37PRR3j5ELqxss1yVqOtnepnHVP9aJ7xS" + crossorigin="anonymous"></script> + + +</body> +</html> + diff --git a/sitemap.xml b/sitemap.xml index 63ff498..b694125 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -57,6 +57,10 @@ </url> <url> + <loc>https://ratis.incubator.apache.org/logservice/security/</loc> + </url> + + <url> <loc>https://ratis.incubator.apache.org/logservice/testing/</loc> </url>
