Author: buildbot
Date: Fri Feb 5 21:47:29 2016
New Revision: 979608
Log:
Staging update by buildbot for slider
Modified:
websites/staging/slider/trunk/content/ (props changed)
websites/staging/slider/trunk/content/docs/manpage.html
websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html
Propchange: websites/staging/slider/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Fri Feb 5 21:47:29 2016
@@ -1 +1 @@
-1726136
+1728735
Modified: websites/staging/slider/trunk/content/docs/manpage.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/manpage.html (original)
+++ websites/staging/slider/trunk/content/docs/manpage.html Fri Feb 5 21:47:29
2016
@@ -265,6 +265,24 @@ If not, the changes will be picked up wh
<p>Use the specific filesystem URI as an argument to the operation.</p>
<!--- =======================================================================
-->
+<h2 id="instance-naming">Instance Naming<a class="headerlink"
href="#instance-naming" title="Permanent link">¶</a></h2>
+<p>Application instance names must:</p>
+<ol>
+<li>be at least one character long</li>
+<li>begin with a lower case letter</li>
+<li>All other characters must be in the range [a-z,0-9,_]</li>
+<li>All upper case characters are converted to lower case</li>
+</ol>
+<p>Example valid names:</p>
+<div class="codehilite"><pre><span class="n">slider1</span>
+<span class="n">storm4</span>
+<span class="n">hbase_instance</span>
+<span class="n">accumulo_m1_tserve4</span>
+</pre></div>
+
+
+<!--- =======================================================================
-->
+
<!--- =======================================================================
-->
<h2 id="actions">Actions<a class="headerlink" href="#actions" title="Permanent
link">¶</a></h2>
@@ -525,7 +543,7 @@ and the principal to log in as. The file
If the JVM does not support this length, the command will fail.
The default value is to 256, as needed for the <code>AES256</code> encryption
scheme.
A JVM without the Java Cryptography Extensions installed does not support
-<code>--keylength 256</code>: Kerberos will not work Kerberos itself is
configured to use
+a key length of 256 bits: Kerberos will unless configured to use
an encryption scheme with a shorter key length.</p>
</li>
<li>
@@ -536,14 +554,22 @@ That is: if the hadoop authentication me
<p>Although there is a <code>--out outfile</code> option, much of the output
can come from the JRE
(to <code>stderr</code>) and via log4j (to <code>stdout</code>). To get all
the output, it is best
to redirect both these output streams to the same file, and omit the
<code>--out</code> option.</p>
-<div class="codehilite"><pre><span class="n">slider</span> <span
class="n">kdiag</span> <span class="o">--</span><span class="n">keytab</span>
<span class="n">zk</span><span class="p">.</span><span
class="n">service</span><span class="p">.</span><span class="n">keytab</span>
<span class="o">--</span><span class="n">principal</span> <span
class="n">zookeeper</span><span class="o">/</span><span
class="n">devix</span><span class="p">@</span><span class="n">REALM</span>
<span class="o">></span> <span class="n">out</span><span
class="p">.</span><span class="n">txt</span> 2<span class="o">>&</span>1
+<div class="codehilite"><pre><span class="n">slider</span> <span
class="n">kdiag</span> <span class="o">--</span><span class="n">keytab</span>
<span class="n">zk</span><span class="p">.</span><span
class="n">service</span><span class="p">.</span><span class="n">keytab</span>
<span class="o">--</span><span class="n">principal</span> <span
class="n">zookeeper</span><span class="o">/</span><span
class="n">devix</span><span class="p">.</span><span
class="n">example</span><span class="p">.</span><span class="n">org</span><span
class="p">@</span><span class="n">REALM</span> <span class="o">></span>
<span class="n">out</span><span class="p">.</span><span class="n">txt</span>
2<span class="o">>&</span>1
</pre></div>
<p>For extra logging during the operation</p>
<ol>
-<li>Set the environment variable <code>HADOOP_JAAS_DEBUG</code> to
<code>true</code>.</li>
-<li>Edit the <code>log4j.properties</code> file for the slider client:<div
class="codehilite"><pre><span class="n">log4j</span><span
class="p">.</span><span class="n">logger</span><span class="p">.</span><span
class="n">org</span><span class="p">.</span><span class="n">apache</span><span
class="p">.</span><span class="n">hadoop</span><span class="p">.</span><span
class="n">security</span><span class="p">=</span><span class="n">DEBUG</span>
+<li>
+<p>Set the environment variable <code>HADOOP_JAAS_DEBUG</code> to
<code>true</code>.</p>
+<div class="codehilite"><pre><span class="n">export</span> <span
class="n">HADOOP_JAAS_DEBUG</span><span class="p">=</span><span
class="n">true</span>
+</pre></div>
+
+
+</li>
+<li>
+<p>Edit the <code>log4j.properties</code> file for the slider client:</p>
+<div class="codehilite"><pre><span class="n">log4j</span><span
class="p">.</span><span class="n">logger</span><span class="p">.</span><span
class="n">org</span><span class="p">.</span><span class="n">apache</span><span
class="p">.</span><span class="n">hadoop</span><span class="p">.</span><span
class="n">security</span><span class="p">=</span><span class="n">DEBUG</span>
</pre></div>
@@ -741,8 +767,7 @@ the version of Hadoop that is currently
<p>Note that this is the client-side Hadoop version, not that running on the
server, though
that can be obtained in the status operation</p>
<h2 id="commands-for-testing">Commands for testing<a class="headerlink"
href="#commands-for-testing" title="Permanent link">¶</a></h2>
-<p>These are clearly abnormal operations; they are here primarily for testing
--and documented for completeness.</p>
+<p>These operations are here primarily for testing.</p>
<h3 id="kill-container-name-id-container-id"><code>kill-container <name>
--id container-id</code><a class="headerlink"
href="#kill-container-name-id-container-id" title="Permanent
link">¶</a></h3>
<p>Kill a YARN container belong to the application. This is useful primarily
for testing the
resilience to failures.</p>
@@ -758,22 +783,22 @@ operation that does not make any attempt
</pre></div>
-<!--- =======================================================================
-->
-
-<h2 id="instance-naming">Instance Naming<a class="headerlink"
href="#instance-naming" title="Permanent link">¶</a></h2>
-<p>Application instance names must:</p>
-<ol>
-<li>be at least one character long</li>
-<li>begin with a lower case letter</li>
-<li>All other characters must be in the range [a-z,0-9,_]</li>
-<li>All upper case characters are converted to lower case</li>
-</ol>
-<p>Example valid names:</p>
-<div class="codehilite"><pre><span class="n">slider1</span>
-<span class="n">storm4</span>
-<span class="n">hbase_instance</span>
-<span class="n">accumulo_m1_tserve4</span>
-</pre></div>
+<h3
id="tokens-source-file-out-file-keytab-keytab-principal-principal"><code>tokens
[--source <file>] [--out file] [--keytab <keytab> --principal
<principal>]</code><a class="headerlink"
href="#tokens-source-file-out-file-keytab-keytab-principal-principal"
title="Permanent link">¶</a></h3>
+<p>Lists current delegation tokens, or, on a secure cluster creates new
ones.</p>
+<p>This is useful for testing
+the delegation token mechanism offered by Oozie, in which Oozie collects
+the tokens needed by slider, saves them to a file, then starts slider in
+with the environment variable <code>HADOOP_TOKEN_FILE_LOCATION</code> set to
the location
+of this file. For ease of doing that, the bash command to set the property is
+printed.</p>
+<p>For reference the tokens needed are:</p>
+<ul>
+<li>An HDFS token.</li>
+<li>A YARN client token to interact with the RM.</li>
+<li>If the timeline server is enabled, a timeline server delegation token.</li>
+</ul>
+<p>If the <code>--keytab</code> and <code>--principal</code> arguments are
supplied, then the credentials
+will be generated with the named principal logged in from the specific
keytab.</p>
</div>
<div id="footer">
Modified:
websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html
==============================================================================
---
websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html
(original)
+++
websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html
Fri Feb 5 21:47:29 2016
@@ -203,12 +203,13 @@ h2:hover > .headerlink, h3:hover > .head
<ul>
-<li>SCRIPT is the top level script that implements the commands for the
component. </li>
-<li>COMMAND is one of the following default commands - START, STOP, INSTALL,
CONFIG, RESTART, STATUS or any custom commands. </li>
-<li>JSON_FILE includes all configuration parameters and the values. </li>
-<li>PACKAGE_ROOT is the root folder of the package. From this folder, its
possible to access files, scripts, templates, packages (e.g. tarballs), etc.
The Yarn-App author has complete control over the structure of the package as
long as the PACKAGE_ROOT and SCRIPT path is known to the management tool. </li>
-<li>STRUCTURED_OUT_FILE is the file where the script can output structured
data. </li>
-<li>LOG_LEVEL is the log level to use when executing the script - Slider
defaults to INFO, use other levels while debugging</li>
+<li><code>SCRIPT</code> is the top level script that implements the commands
for the component. </li>
+<li><code>COMMAND</code> is one of the following default commands -
<code>START</code>, <code>STOP</code>, <code>INSTALL</code>,
<code>CONFIG</code>, <code>RESTART</code>, <code>STATUS</code> or any custom
commands. </li>
+<li><code>JSON_FILE</code> includes all configuration parameters and the
values. </li>
+<li><code>PACKAGE_ROOT</code> is the root folder of the package. From this
folder, its possible to access files, scripts, templates, packages (e.g.
tarballs), etc.
+The Yarn-App author has complete control over the structure of the package as
long as the <code>PACKAGE_ROOT</code> and <code>SCRIPT</code> path is known to
the management tool. </li>
+<li><code>STRUCTURED_OUT_FILE</code> is the file where the script can output
structured data. </li>
+<li><code>LOG_LEVEL</code> is the log level to use when executing the script -
Slider defaults to INFO, use other levels while debugging</li>
</ul>
<p>The management infrastructure is expected to automatically reports back
STD_OUT and STD_ERR.</p>
<p>Sample:</p>
@@ -250,7 +251,7 @@ h2:hover > .headerlink, h3:hover > .head
<h3 id="sample-configuration-parameters">Sample configuration parameters<a
class="headerlink" href="#sample-configuration-parameters" title="Permanent
link">¶</a></h3>
-<p>While a script can access all the config parameters the key property bags
are <strong>hostLevelParams</strong> and <strong>configurations</strong>.</p>
+<p>While a script can access all the config parameters the key property bags
are <strong><code>hostLevelParams</code></strong> and
<strong><code>configurations</code></strong>.</p>
<div class="codehilite"><pre>{
"roleCommand": "START",
"commandParams": {
@@ -351,10 +352,10 @@ h2:hover > .headerlink, h3:hover > .head
<p>At this point, you are free to implement as you wish. Slider provides a
rich library for basic operations needed for application commands. So the
structure we suggest is:</p>
<ul>
-<li>Create a params.py file to read the json config file into variables. This
will allow you to provide initial validation, provide default values, or create
complex values</li>
+<li>Create a <code>params.py</code> file to read the json config file into
variables. This will allow you to provide initial validation, provide default
values, or create complex values</li>
<li>Use the above parameters in your scripts</li>
</ul>
-<p>A params.py file may be as follows:</p>
+<p>A <code>params.py</code> file may be as follows:</p>
<div class="codehilite"><pre><span class="n">from</span> <span
class="n">resource_management</span> <span class="n">import</span> <span
class="o">*</span>
# <span class="n">server</span> <span class="n">configurations</span>
