Author: buildbot
Date: Sat Oct 11 02:12:09 2014
New Revision: 925331
Log:
Staging update by buildbot for slider
Modified:
websites/staging/slider/trunk/content/ (props changed)
websites/staging/slider/trunk/content/docs/security.html
Propchange: websites/staging/slider/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Sat Oct 11 02:12:09 2014
@@ -1 +1 @@
-1630615
+1631012
Modified: websites/staging/slider/trunk/content/docs/security.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/security.html (original)
+++ websites/staging/slider/trunk/content/docs/security.html Sat Oct 11
02:12:09 2014
@@ -183,11 +183,11 @@ listed at the bottom. </p>
<h2 id="concepts">Concepts</h2>
<p>Slider runs in secure clusters, but with restrictions</p>
<ol>
-<li>The keytabs to allow a worker to authenticate with the master must
- be distributed in advance: Slider does not attempt to pass these
around.</li>
+<li>The keytabs to allow the AM and components to authenticate with the master
are
+ either distributed in advance or can be distributed by Slider.</li>
<li>Until the location of Slider node instances can be strictly limited to
- a set of nodes (a future YARN feature), the keytabs must be passed to
- all the nodes in the cluster in advance, <em>and made available to the
+ a set of nodes (a future YARN feature), the keytabs are required by
+ all the nodes in the cluster, <em>and made available to the
user creating the cluster</em></li>
<li>due to the way that HBase and accumulo authenticate worker nodes to
the masters, any HBase node running on a server must authenticate as
@@ -242,10 +242,11 @@ kerberos identities.</li>
<ol>
<li>The user is expected to have their own Kerberos principal, and have used
<code>kinit</code>
or equivalent to authenticate with Kerberos and gain a (time-bounded)
TGT</li>
-<li>The user is expected to have their own principals for every host in the
cluster of the form
- username/hostname@REALM</li>
-<li>A keytab must be generated which contains all these principals -and
distributed
- to all the nodes in the cluster with read access permissions to the
user.</li>
+<li>The user is expected to have principals for every host in the cluster of
the form
+ username/hostname@REALM for component aunthentication. The AM
authentication requirements
+ can be satisfied with a non-host based principal (username@REALM).</li>
+<li>Separate keytabs should be generated for the AM, which contains the AM
login principal, and the service components, which contain all the service
principals. The keytabs can be manually distributed
+ to all the nodes in the cluster with read access permissions to the user, or
the user may elect to leverage the Slider keytab distribution mechanism.</li>
<li>When the user creates a secure cluster, they provide the standard HBase
kerberos options
to identify the principals to use and the keytab location.</li>
</ol>
@@ -263,6 +264,56 @@ user <code>r-x</code> for the group and
rights of the user that created the cluster.</p>
<p>The Application Master will read in the JSON cluster specification file,
and instantiate the
relevant number of componentss. </p>
+<h3 id="the-keytab-distributionaccess-options">The Keytab distribution/access
Options</h3>
+<p>The AM has been modified to leverage keytabs for authenticating rather than
relying on delegation-token based authentication mechanisms. In order to
perform this login the AM requires access to a keytab file that contains the
principal representing the user identity to be associated with the launched
application instance. There are two mechanisms supported for keytab access
and/or distribution:</p>
+<h4 id="local-keytab-file-access">Local Keytab file access:</h4>
+<p>An application deployer may choose to pre-distribute the keytab files
required to the node manager hosts in a yarn cluster. In that instance the
appConfig.json requires the following property:</p>
+<div class="codehilite"><pre><span class="p">.</span> <span class="p">.</span>
<span class="p">.</span>
+"<span class="n">components</span>"<span class="p">:</span> <span
class="p">{</span>
+ "<span class="n">slider</span><span class="o">-</span><span
class="n">appmaster</span>"<span class="p">:</span> <span
class="p">{</span>
+ "<span class="n">jvm</span><span class="p">.</span><span
class="n">heapsize</span>"<span class="p">:</span> "256<span
class="n">M</span>"<span class="p">,</span>
+ "<span class="n">slider</span><span class="p">.</span><span
class="n">am</span><span class="p">.</span><span class="n">keytab</span><span
class="p">.</span><span class="n">local</span><span class="p">.</span><span
class="n">path</span>"<span class="p">:</span> "<span
class="o">/</span><span class="n">etc</span><span class="o">/</span><span
class="n">security</span><span class="o">/</span><span
class="n">keytabs</span><span class="o">/</span><span
class="n">hbase</span><span class="p">.</span><span
class="n">headless</span><span class="p">.</span><span
class="n">keytab</span>"
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
+
+
+<p>The âslider.am.keytab.local.pathâ property provides the full path to
the keytab file location and is mandatory for the local lookup mechanism. In
this scenario the distribution of keytab files for the AM AND the application
itself is the purview of the application deployer. So, for example, for an
hbase deployment, the hbase site service keytab will have to be distributed as
well and indicated in the hbase-site properties:</p>
+<div class="codehilite"><pre> <span class="p">.</span> <span
class="p">.</span> <span class="p">.</span>
+ "<span class="n">site</span><span class="p">.</span><span
class="n">hbase</span><span class="o">-</span><span class="n">site</span><span
class="p">.</span><span class="n">hbase</span><span class="p">.</span><span
class="n">master</span><span class="p">.</span><span
class="n">kerberos</span><span class="p">.</span><span
class="n">principal</span>"<span class="p">:</span> "<span
class="n">hbase</span><span class="o">/</span><span class="n">_HOST</span><span
class="p">@</span><span class="n">EXAMPLE</span><span class="p">.</span><span
class="n">COM</span>"<span class="p">,</span>
+ "<span class="n">site</span><span class="p">.</span><span
class="n">hbase</span><span class="o">-</span><span class="n">site</span><span
class="p">.</span><span class="n">hbase</span><span class="p">.</span><span
class="n">master</span><span class="p">.</span><span
class="n">keytab</span><span class="p">.</span><span
class="n">file</span>"<span class="p">:</span> "<span
class="o">/</span><span class="n">etc</span><span class="o">/</span><span
class="n">security</span><span class="o">/</span><span
class="n">keytabs</span><span class="o">/</span><span
class="n">hbase</span><span class="p">.</span><span
class="n">service</span><span class="p">.</span><span
class="n">keytab</span>"<span class="p">,</span>
+ <span class="p">.</span> <span class="p">.</span> <span class="p">.</span>
+</pre></div>
+
+
+<h4 id="slider-keytab-distribution">Slider keytab distribution:</h4>
+<p>The deployer can select to upload the keytab files for the AM and the
application to an HDFS directory (with appropriate permissions set) and slider
will localize the keytab files to locations accessible by the AM or the
application containers:</p>
+<div class="codehilite"><pre><span class="p">.</span> <span class="p">.</span>
<span class="p">.</span>
+"<span class="n">components</span>"<span class="p">:</span> <span
class="p">{</span>
+ "<span class="n">slider</span><span class="o">-</span><span
class="n">appmaster</span>"<span class="p">:</span> <span
class="p">{</span>
+ "<span class="n">jvm</span><span class="p">.</span><span
class="n">heapsize</span>"<span class="p">:</span> "256<span
class="n">M</span>"<span class="p">,</span>
+ "<span class="n">slider</span><span class="p">.</span><span
class="n">hdfs</span><span class="p">.</span><span class="n">keytab</span><span
class="p">.</span><span class="n">dir</span>"<span class="p">:</span>
"<span class="p">.</span><span class="n">slider</span><span
class="o">/</span><span class="n">keytabs</span><span class="o">/</span><span
class="n">hbase</span>"<span class="p">,</span>
+ "<span class="n">slider</span><span class="p">.</span><span
class="n">am</span><span class="p">.</span><span class="n">login</span><span
class="p">.</span><span class="n">keytab</span><span class="p">.</span><span
class="n">name</span>"<span class="p">:</span> "<span
class="n">hbase</span><span class="p">.</span><span
class="n">headless</span><span class="p">.</span><span
class="n">keytab</span>"
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
+
+
+<p>The âslider.hdfs.keytab.dirâ points to an HDFS path, relative to the
userâs home directory (e.g. /users/hbase), in which slider can find all
keytab files required for both AM login as well as application services (e.g.
for hbase that would be the headless keytab for the AM and the service keytab
for the HBase application components). If no value is specified, a default
location of â.slider/keytabs/<cluster name>â is assumed.
+ The âslider.am.login.keytab.nameâ is the name of the keytab file
(mandatory property), found within the specified directory, that the AM will
use to lookup up the login principal and authenticate.</p>
+<p>If leveraging the slider-based distribution mechanism, the keytab files for
components will be accessible from a âkeytabsâ sub-directory of the
container work folder, e.g.:</p>
+<div class="codehilite"><pre> . . .
+ "site.hbase-site.hbase.master.kerberos.principal":
"hbase/[email protected]",
+ "site.hbase-site.hbase.master.keytab.file": "<span
class="cp">${</span><span class="n">AGENT_WORK_ROOT</span><span
class="cp">}</span>/keytabs/hbase.service.keytab",
+ . . .
+</pre></div>
+
+
+<p>For both mechanisms above, the principal name used for authentication is
either:</p>
+<ul>
+<li>The principal name established on the client side before invocation of the
Slider CLI (the principal used to âkinitâ) or</li>
+<li>The value specified for a âslider.keytab.principal.nameâ property.
</li>
+</ul>
<h2
id="securing-communications-between-the-slider-client-and-the-slider-am">Securing
communications between the Slider Client and the Slider AM.</h2>
<p>When the AM is deployed in a secure cluster,
it automatically uses Kerberos-authorized RPC channels. The client must
acquire a
