Modified:
websites/staging/slider/trunk/content/developing/submitting_patches.html
==============================================================================
--- websites/staging/slider/trunk/content/developing/submitting_patches.html
(original)
+++ websites/staging/slider/trunk/content/developing/submitting_patches.html
Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block"
href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink,
h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink,
dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,8 +193,8 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="title">title</h1>
-<h2 id="recommended-patch-development-process">Recommended Patch development
process</h2>
+<h1 id="title">title<a class="headerlink" href="#title" title="Permanent
link">¶</a></h1>
+<h2 id="recommended-patch-development-process">Recommended Patch development
process<a class="headerlink" href="#recommended-patch-development-process"
title="Permanent link">¶</a></h2>
<ol>
<li>Git clone the repository, checking out <code>develop</code></li>
<li>Make sure it builds and tests, following the instructions.</li>
@@ -192,7 +203,7 @@ Latest release: <strong>0.80.0-incubatin
<li>Check in your code as you go along, including the relevant JIRA value in
the commit message, e.g. "SLIDER-149 patch pom"</li>
<li>Rebase before the final patch submission, and ideally regularly</li>
</ol>
-<h3 id="rebasing">Rebasing</h3>
+<h3 id="rebasing">Rebasing<a class="headerlink" href="#rebasing"
title="Permanent link">¶</a></h3>
<p>Patches will be applied to the head of the develop branch, so need to be
submitted as diffs against that. This is best done during development by
rebasing your commit history <em>provided you are not collaborating with others
on that branch</em></p>
<ol>
<li>Tag the current version of the branch, something like
<code>tag_<JIRA-NAME>_<DATE>_rebasing</code>. Example
<code>tag_SLIDER-149_2014-08-14_rebasing</code>. This ensures the pre-rebase
commit tree is still retained somewhere.</li>
@@ -200,7 +211,7 @@ Latest release: <strong>0.80.0-incubatin
<li>check out your feature branch, then rebase it with <code>git rebase
develop</code>.</li>
<li>Fix any conflicts.</li>
</ol>
-<h2 id="generating-patches">Generating Patches</h2>
+<h2 id="generating-patches">Generating Patches<a class="headerlink"
href="#generating-patches" title="Permanent link">¶</a></h2>
<p>There's two ways you can create patches. </p>
<ol>
<li>
@@ -216,7 +227,7 @@ Latest release: <strong>0.80.0-incubatin
<li>Every time you create a new patch, update the patch revision number.</li>
<li>Consider tagging your branch whenever you create a patch, including the
patch filename in the tag, e.g. <code>tag_patch_SLIDER-149-002</code>. This
makes the work history easier to trace.</li>
</ol>
-<h3 id="simple-git-diff">Simple git diff</h3>
+<h3 id="simple-git-diff">Simple git diff<a class="headerlink"
href="#simple-git-diff" title="Permanent link">¶</a></h3>
<p>Create a diff between the <code>develop</code> branch and your latest
commit</p>
<div class="codehilite"><pre><span class="n">git</span> <span
class="n">diff</span> <span class="n">develop</span><span
class="p">...</span><span class="n">HEAD</span> <span class="o">></span>
<span class="o"><</span><span class="n">PATCH_FILENAME</span><span
class="o">></span>
</pre></div>
@@ -227,7 +238,7 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h3 id="formatted-patches">formatted patches</h3>
+<h3 id="formatted-patches">formatted patches<a class="headerlink"
href="#formatted-patches" title="Permanent link">¶</a></h3>
<p>A git formatted patch includes all the commits applied during the
development process. This must be called against a branch which has either been
rebased, or had its patch history squashed to a single patch. The process of
doing that is <a
href="http://stackoverflow.com/questions/616556/how-do-you-squash-commits-into-one-patch-with-git-format-patch";>documented
on stackoverflow</a></p>
<div class="codehilite"><pre><span class="n">git</span> <span
class="n">format</span><span class="o">-</span><span class="n">patch</span>
<span class="o">--</span><span class="n">stdout</span> <span
class="n">develop</span> <span class="o">></span> <span
class="o"><</span><span class="n">PATCH_FILENAME</span><span
class="o">></span>
</pre></div>
@@ -238,11 +249,11 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h3 id="submitting-patches">Submitting Patches</h3>
+<h3 id="submitting-patches">Submitting Patches<a class="headerlink"
href="#submitting-patches" title="Permanent link">¶</a></h3>
<p>Add the patch to the JIRA to which it is related.</p>
-<h2 id="applying-patches">Applying patches</h2>
+<h2 id="applying-patches">Applying patches<a class="headerlink"
href="#applying-patches" title="Permanent link">¶</a></h2>
<p>Patches MUST be applied to a new feature branch created off the latest
commit of <code>develop</code>; if a long-lived branch is being developed, it
MUST be rebased first.</p>
-<h3 id="simple-patches">simple patches</h3>
+<h3 id="simple-patches">simple patches<a class="headerlink"
href="#simple-patches" title="Permanent link">¶</a></h3>
<p>Before applying a patch, check it actually works:</p>
<div class="codehilite"><pre><span class="n">git</span> <span
class="n">apply</span> <span class="o">-</span><span class="n">p0</span> <span
class="o">--</span><span class="n">check</span> <span class="o">--</span><span
class="n">verbose</span> <span class="o">--</span><span
class="n">whitespace</span><span class="p">=</span><span class="nb">fix</span>
<span class="o"><</span><span class="n">PATH</span><span
class="o">-</span><span class="n">TO</span><span class="o">-</span><span
class="n">PATCH</span><span class="o">></span>
</pre></div>
@@ -254,7 +265,7 @@ Latest release: <strong>0.80.0-incubatin
<p>Then: clean build, test. </p>
-<h3 id="formatted-patches_1">Formatted patches</h3>
+<h3 id="formatted-patches_1">Formatted patches<a class="headerlink"
href="#formatted-patches_1" title="Permanent link">¶</a></h3>
<p>Formatted patches, with the commit history, are applied with <code>git
am</code></p>
<div class="codehilite"><pre><span class="n">git</span> <span
class="n">am</span> <span class="o">--</span><span
class="n">whitespace</span><span class="p">=</span><span class="nb">fix</span>
<span class="o"><</span> <span class="o"><</span><span
class="n">PATH</span><span class="o">-</span><span class="n">TO</span><span
class="o">-</span><span class="n">PATCH</span><span class="o">></span>
</pre></div>
@@ -265,7 +276,7 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h2 id="reviewing-patches">Reviewing patches</h2>
+<h2 id="reviewing-patches">Reviewing patches<a class="headerlink"
href="#reviewing-patches" title="Permanent link">¶</a></h2>
<p>After being applied, the feature branch must be committed. This ensures
that there is a history of the submitted patch. The commit message must include
the patch filename.</p>
<p>As well as reviewing the code and its style, all tests that it patch must
be run before attempting to merge the code back into the develop branch.</p>
<p>Ideally, the full test run, minicluster and functional should be performed.
A less-reliable shortcut is to run all the tests that are changed in the patch,
and all tests which are known to depend on the expected feature. As an example,
if a client command is changed, search for uses of that command and execute
those tests.</p>
Modified: websites/staging/slider/trunk/content/developing/testing.html
==============================================================================
--- websites/staging/slider/trunk/content/developing/testing.html (original)
+++ websites/staging/slider/trunk/content/developing/testing.html Mon Nov 2
17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block"
href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink,
h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink,
dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,7 +196,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="testing-apache-slider">Testing Apache Slider</h1>
+<h1 id="testing-apache-slider">Testing Apache Slider<a class="headerlink"
href="#testing-apache-slider" title="Permanent link">¶</a></h1>
<div class="codehilite"><pre> <span class="n">The</span> <span
class="n">key</span> <span class="n">words</span> "<span
class="n">MUST</span>"<span class="p">,</span> "<span
class="n">MUST</span> <span class="n">NOT</span>"<span class="p">,</span>
"<span class="n">REQUIRED</span>"<span class="p">,</span> "<span
class="n">SHALL</span>"<span class="p">,</span> "<span
class="n">SHALL</span>
<span class="n">NOT</span>"<span class="p">,</span> "<span
class="n">SHOULD</span>"<span class="p">,</span> "<span
class="n">SHOULD</span> <span class="n">NOT</span>"<span
class="p">,</span> "<span class="n">RECOMMENDED</span>"<span
class="p">,</span> "<span class="n">MAY</span>"<span
class="p">,</span> <span class="n">and</span>
"<span class="n">OPTIONAL</span>" <span class="n">in</span> <span
class="n">this</span> <span class="n">document</span> <span
class="n">are</span> <span class="n">to</span> <span class="n">be</span> <span
class="n">interpreted</span> <span class="n">as</span> <span
class="n">described</span> <span class="n">in</span>
@@ -193,7 +204,7 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h2 id="standalone-tests">Standalone Tests</h2>
+<h2 id="standalone-tests">Standalone Tests<a class="headerlink"
href="#standalone-tests" title="Permanent link">¶</a></h2>
<p>Slider core contains a suite of tests that are designed to run on the local
machine,
using Hadoop's <code>MiniDFSCluster</code> and <code>MiniYARNCluster</code>
classes to create small,
one-node test clusters. All the YARN/HDFS code runs in the JUnit process; the
Modified: websites/staging/slider/trunk/content/developing/windows.html
==============================================================================
--- websites/staging/slider/trunk/content/developing/windows.html (original)
+++ websites/staging/slider/trunk/content/developing/windows.html Mon Nov 2
17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block"
href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink,
h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink,
dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,37 +193,37 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="windows-development-and-testing">Windows Development and Testing</h1>
+<h1 id="windows-development-and-testing">Windows Development and Testing<a
class="headerlink" href="#windows-development-and-testing" title="Permanent
link">¶</a></h1>
<p>This documents what you need to build and test on Windows</p>
<ol>
<li>Windows server 2012</li>
<li>Create an admin account for yourself</li>
<li>On a virtual machine: take snapshots as you go along</li>
</ol>
-<h1 id="core-installs">Core installs</h1>
+<h1 id="core-installs">Core installs<a class="headerlink"
href="#core-installs" title="Permanent link">¶</a></h1>
<ul>
<li>Install <a
href="http://slproweb.com/products/Win32OpenSSL.html";>openssl 1.0.1i 64-bit</a>
into a dir with no spaces, <code>C:\bin\openssl</code>. Add its bin dir to
your path.
(this will also need the Visual C++ 2008 Redistributables (DLL libraries)
installed.</li>
</ul>
-<h2 id="virtual-box">Virtual Box</h2>
+<h2 id="virtual-box">Virtual Box<a class="headerlink" href="#virtual-box"
title="Permanent link">¶</a></h2>
<ol>
<li>Ask for lots of RAM, CPU. You can vary CPU and limits as you go, and
increase them for test runs/full screen work.</li>
<li>2D acceleration of video</li>
<li>Critical: Enable PAE under "processor". this avoids random hangs.</li>
</ol>
-<h3 id="os-features">OS features</h3>
+<h3 id="os-features">OS features<a class="headerlink" href="#os-features"
title="Permanent link">¶</a></h3>
<p>In the server management app, go to "manage features", where you can add</p>
<ul>
<li>Telnet server: lets you telnet in.</li>
<li>Maybe: desktop experience</li>
</ul>
-<h3 id="os-settings">OS settings</h3>
+<h3 id="os-settings">OS settings<a class="headerlink" href="#os-settings"
title="Permanent link">¶</a></h3>
<p>Firewall setup is a mystery. You need to convince the OS that your Ether
network
is "private", then edit the firewall to open it up.</p>
-<h2 id="dev-tools">Dev tools</h2>
+<h2 id="dev-tools">Dev tools<a class="headerlink" href="#dev-tools"
title="Permanent link">¶</a></h2>
<ol>
<li>Bitvise SSH server; add your private key to the account you create.
(even with this installed, you will still be mostly working from the windows
@@ -230,7 +241,7 @@ view all web pages in IE to verify they
<li><a
href="http://technet.microsoft.com/en-us/sysinternals/bb545021.aspx";>Sysinternals</a>.
Their <code>tcpview</code> tool is ideal for seeing what ports Hadoop services
are listening on.</li>
</ol>
-<h3 id="knowledge">Knowledge</h3>
+<h3 id="knowledge">Knowledge<a class="headerlink" href="#knowledge"
title="Permanent link">¶</a></h3>
<ol>
<li>Knowledge of the <a
href="http://technet.microsoft.com/en-us/library/bb490954.aspx";>CMD.EXE command
line</a></li>
<li>
@@ -246,7 +257,7 @@ Their <code>tcpview</code> tool is ideal
</ol>
</li>
</ol>
-<h2 id="windows">Windows</h2>
+<h2 id="windows">Windows<a class="headerlink" href="#windows" title="Permanent
link">¶</a></h2>
<ol>
<li>Follow the Hadoop <code>BUILDING.TXT</code> instructions. </li>
<li>install cygwin64 with it's <code>sh</code> command.</li>
@@ -265,11 +276,11 @@ not any cygwin one.</li>
<em> <code>awk</code>
</em> <code>grep</code>
* <code>xargs</code></p>
-<h2 id="avoid">Avoid</h2>
+<h2 id="avoid">Avoid<a class="headerlink" href="#avoid" title="Permanent
link">¶</a></h2>
<ul>
<li>having cygwin <code>cmake</code> on your path; it must be the native
cmake</li>
</ul>
-<h3 id="env-variables">env variables</h3>
+<h3 id="env-variables">env variables<a class="headerlink"
href="#env-variables" title="Permanent link">¶</a></h3>
<div class="codehilite"><pre><span class="n">CYGWIN</span><span
class="p">=</span><span class="n">nodosfilewarning</span>
<span class="n">PATH</span> <span class="p">=</span> <span
class="n">cmake</span> <span class="n">bin</span><span class="p">,</span> <span
class="n">cygwin</span> <span class="n">bin</span><span class="p">,</span>
<span class="n">python</span> <span class="n">bin</span><span
class="p">,</span> <span class="n">hadoop_home</span> <span
class="n">bin</span> <span class="p">(</span><span class="n">see</span> <span
class="n">below</span><span class="p">)</span>
</pre></div>
@@ -286,7 +297,7 @@ not any cygwin one.</li>
</pre></div>
-<h3 id="commands-to-test-cli-setup">Commands to test CLI setup</h3>
+<h3 id="commands-to-test-cli-setup">Commands to test CLI setup<a
class="headerlink" href="#commands-to-test-cli-setup" title="Permanent
link">¶</a></h3>
<div class="codehilite"><pre><span class="n">cmake</span> <span
class="o">--</span><span class="n">version</span>
<span class="n">openssl</span> <span class="n">version</span>
<span class="n">python</span> <span class="o">--</span><span
class="n">version</span>
@@ -308,14 +319,14 @@ not any cygwin one.</li>
</pre></div>
-<h3 id="sliders-test-for-windows-support">Slider's test for windows
support</h3>
+<h3 id="sliders-test-for-windows-support">Slider's test for windows support<a
class="headerlink" href="#sliders-test-for-windows-support" title="Permanent
link">¶</a></h3>
<p>You can test for windows support with the slider-core test</p>
<div class="codehilite"><pre><span class="n">mvn</span> <span
class="n">test</span> <span class="o">-</span><span
class="n">Dtest</span><span class="p">=</span><span
class="n">TestWindowsSupport</span>
</pre></div>
<p>This looks for the arguments and other valid settings</p>
-<h2 id="building-hadoop">Building Hadoop</h2>
+<h2 id="building-hadoop">Building Hadoop<a class="headerlink"
href="#building-hadoop" title="Permanent link">¶</a></h2>
<ol>
<li>Find the "Windows SDK 7.1 command prompt" using the search field in the
windows 8 screen</li>
Modified: websites/staging/slider/trunk/content/docs/api/index.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/api/index.html (original)
+++ websites/staging/slider/trunk/content/docs/api/index.html Mon Nov 2
17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block"
href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink,
h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink,
dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,7 +193,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="slider-rest-apis">Slider REST APIs</h1>
+<h1 id="slider-rest-apis">Slider REST APIs<a class="headerlink"
href="#slider-rest-apis" title="Permanent link">¶</a></h1>
<ul>
<li><a href="slider_REST_v1.html">REST API v1</a>: Implemented in Slider
0.60</li>
<li><a href="slider_REST_api_v2.html">REST API v2</a>: V2 API</li>
Modified: websites/staging/slider/trunk/content/docs/api/slider_REST_api_v2.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/api/slider_REST_api_v2.html
(original)
+++ websites/staging/slider/trunk/content/docs/api/slider_REST_api_v2.html Mon
Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block"
href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink,
h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink,
dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,7 +193,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="slider-am-rest-api-v2">Slider AM REST API: v2</h1>
+<h1 id="slider-am-rest-api-v2">Slider AM REST API: v2<a class="headerlink"
href="#slider-am-rest-api-v2" title="Permanent link">¶</a></h1>
<p>This is a successor to the <a href="slider_REST_v1.html">Slider v1 REST
API</a></p>
<p>This document represents the third iteration of designing a REST API to be
implemented by the Slider Application Master.</p>
<div class="codehilite"><pre> <span class="n">The</span> <span
class="n">key</span> <span class="n">words</span> "<span
class="n">MUST</span>"<span class="p">,</span> "<span
class="n">MUST</span> <span class="n">NOT</span>"<span class="p">,</span>
"<span class="n">REQUIRED</span>"<span class="p">,</span> "<span
class="n">SHALL</span>"<span class="p">,</span> "<span
class="n">SHALL</span>
@@ -192,11 +203,11 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h2 id="history">History</h2>
+<h2 id="history">History<a class="headerlink" href="#history" title="Permanent
link">¶</a></h2>
<ol>
<li>Created: January 2014</li>
</ol>
-<h1 id="introduction-and-background">Introduction and Background</h1>
+<h1 id="introduction-and-background">Introduction and Background<a
class="headerlink" href="#introduction-and-background" title="Permanent
link">¶</a></h1>
<p>Slider 0.60 uses Hadoop IPC for communications between the Slider client
and the per-instance application master,
with a READ-only JSON view of the cluster, as documented in the <a
href="slider_REST_v1.html">Slider v1 REST API</a></p>
<p>Were Slider to support a read/write REST API, it would be possible to:</p>
@@ -211,7 +222,7 @@ with a READ-only JSON view of the cluste
<p>Offer alternative methods of constructing an application</p>
</li>
</ol>
-<h2 id="slider-configuration-model-and-rest">Slider Configuration Model and
REST</h2>
+<h2 id="slider-configuration-model-and-rest">Slider Configuration Model and
REST<a class="headerlink" href="#slider-configuration-model-and-rest"
title="Permanent link">¶</a></h2>
<p>Slider's declarative view of the application to deploy fits in well with a
REST world view: one does not make calls to operations such as "increase region
server count by two", instead the JSON specification of YARN resources is
altered such that the region server count is implemented, then the new JSON
document submitted. currently this is done via IPC.</p>
<p>Where Slider does not integrate well with REST is</p>
<ol>
@@ -228,7 +239,7 @@ with a READ-only JSON view of the cluste
<p>No support of configuration update of a running application. The content
may be read, but a write is not supported (or if it is, they are not addressed
until the application is next restarted).</p>
</li>
</ol>
-<h2 id="existing-ipc-api">Existing IPC API</h2>
+<h2 id="existing-ipc-api">Existing IPC API<a class="headerlink"
href="#existing-ipc-api" title="Permanent link">¶</a></h2>
<p>The slider IPC layer uses protobuf-formatted payloads, with the Hadoop IPC
layer handling security: authorization, authentication and encryption.</p>
<div class="codehilite"><pre><span class="n">service</span> <span
class="n">SliderClusterProtocolPB</span> <span class="p">{</span>
@@ -271,10 +282,10 @@ with a READ-only JSON view of the cluste
</li>
</ol>
<p>Note that for a "version robust" marshalling format, Protobuf is (a) hard
to work with in Java and (b) very hard to examine at the payload layer in Java
(e.g. to enumerate elements which were not known at compile time) and (c) due
to Google's lack of backwards compatibility in libraries and generated code,
very brittle in the Java source. </p>
-<h1 id="use-cases-of-an-am-rest-api">Use Cases of an AM REST API</h1>
+<h1 id="use-cases-of-an-am-rest-api">Use Cases of an AM REST API<a
class="headerlink" href="#use-cases-of-an-am-rest-api" title="Permanent
link">¶</a></h1>
<p>Here are the possible different cases of a Slider REST API. </p>
<p>Each one has different requirements âso the priority of supporting
different use cases will scope and direct effort</p>
-<h2 id="command-line-tooling">Command Line tooling</h2>
+<h2 id="command-line-tooling">Command Line tooling<a class="headerlink"
href="#command-line-tooling" title="Permanent link">¶</a></h2>
<p>Direct communication with Slider via <code>curl</code>, <code>wget</code>,
Python, and other lightweight tooling, rather than exclusively via the Java
Slider JAR</p>
<ul>
<li>
@@ -295,7 +306,7 @@ with a READ-only JSON view of the cluste
<p>Scripts may need to have the ability to block on an operation until the
application reaches a desired state (e.g containers match requested count)</p>
</li>
</ul>
-<h2 id="web-ui">Web UI</h2>
+<h2 id="web-ui">Web UI<a class="headerlink" href="#web-ui" title="Permanent
link">¶</a></h2>
<p>The (currently minimal) Slider Web UI could forward operations the REST API
via HTML/HTML5 forms.</p>
<p>The state of the application could be presented better than it is today.</p>
<p>It would also be possible to build a more complex web application that that
offered by slider today.</p>
@@ -305,7 +316,7 @@ with a READ-only JSON view of the cluste
<ul>
<li>Support for HTML form submission required.</li>
</ul>
-<h2 id="management-tooling">Management tooling</h2>
+<h2 id="management-tooling">Management tooling<a class="headerlink"
href="#management-tooling" title="Permanent link">¶</a></h2>
<p>The example of this is Apache Ambari, but it is not restricted to this
program; Ambari is merely a representative example of "<em>a web application
launching and controlling an application via slider, on behalf of
users"</em>.</p>
<p>We know today that such applications built in Java do not need a REST API;
the slider client itself can be used for this. </p>
<p>What a REST API could do is</p>
@@ -342,7 +353,7 @@ with a READ-only JSON view of the cluste
<p>May want more slider application metrics.</p>
</li>
</ul>
-<h2 id="long-haul-client">Long-haul Client</h2>
+<h2 id="long-haul-client">Long-haul Client<a class="headerlink"
href="#long-haul-client" title="Permanent link">¶</a></h2>
<p>A long-haul clients is probably the most complex client application. It can
be probably be done within the slider client codebase, so allowing remote
application creation and manipulation.</p>
<p>Remote cloud deployments are a key target here âso we cannot expect the
cluster's HDFS storage to be persistent over time. Instead we must keep
persistent data (packages, JSON configurations) in the platform's persistent
store (amazon S3, Azure AVS, etc). YARN node managers do already "localize"
resources served up this way; persisting application state may be more complex
if the consistency model of the object store does not match that of HDFS.</p>
<p><strong>Requirements</strong></p>
@@ -381,7 +392,7 @@ with a READ-only JSON view of the cluste
<p>Persistent application instance configuration to be in the persistent data
store rather than transient HDFS storage.</p>
</li>
</ul>
-<h2 id="functional-testing">Functional Testing</h2>
+<h2 id="functional-testing">Functional Testing<a class="headerlink"
href="#functional-testing" title="Permanent link">¶</a></h2>
<p>Slider uses the slider-client as an in-VM library during its minicluster
unit tests.</p>
<p>For functional testing, it uses the slider CLI as an external application.
This guarantees full testing of the CLI, including the shell/python scripts
themselves. It was this testing which picked up some problems with the python
script on windows, and a later regression related to accumulo. It also forces
us to ensure that the return codes of operations differentiate between
different failure causes, rather than providing a simple "-1" error indicating
that an operation failed. Our <a href="/docs/exitcodes.html">exit codes</a> are
now something which may be used for support and debugging.</p>
<p>A REST API could also be used for testing, though not, for the reasons
above, by slider itself, except in the special case of functional tests of the
REST API itself.</p>
@@ -413,7 +424,7 @@ with a READ-only JSON view of the cluste
<p>API calls to probe for state (ideally blocking)</p>
</li>
</ul>
-<h2 id="deployed-application">Deployed Application</h2>
+<h2 id="deployed-application">Deployed Application<a class="headerlink"
href="#deployed-application" title="Permanent link">¶</a></h2>
<p>This is a use case which came from the Storm team: give the application the
ability to talk to Slider and so query and manipulate its own deployment.</p>
<p>This allows the application to expand and contract itself based on
perceived need, and to explicitly release specific components which are no
longer required. It can also expose the YARN cluster details to the
application, so allowing the deployed application to build a model of the YARN
cluster without talking directly to it.</p>
<p>In this design, the Slider AM's REST API is no longer for clients of the
application, or even management tools âit becomes the API by which deployed
applications integrate with YARN. To use the current fashionable terminology,
it becomes a "microservice" rather than a library.</p>
@@ -444,8 +455,8 @@ with a READ-only JSON view of the cluste
<p>Ability to query recent event history.</p>
</li>
</ul>
-<h1 id="api-principles-high-rest-with-asynchronous-state-changes">API
Principles: High Rest with Asynchronous state changes</h1>
-<h2 id="uris-for-overall-and-detailed-access">URIs for overall and detailed
access</h2>
+<h1 id="api-principles-high-rest-with-asynchronous-state-changes">API
Principles: High Rest with Asynchronous state changes<a class="headerlink"
href="#api-principles-high-rest-with-asynchronous-state-changes"
title="Permanent link">¶</a></h1>
+<h2 id="uris-for-overall-and-detailed-access">URIs for overall and detailed
access<a class="headerlink" href="#uris-for-overall-and-detailed-access"
title="Permanent link">¶</a></h2>
<p>Resources SHOULD use URIs over ? parameters or arguments within the body.
</p>
<ul>
<li>
@@ -466,10 +477,10 @@ with a READ-only JSON view of the cluste
</ol>
</li>
</ul>
-<h3 id="use-and-generate-standard-http-headers-when-possible">Use and generate
standard HTTP Headers when possible</h3>
+<h3 id="use-and-generate-standard-http-headers-when-possible">Use and generate
standard HTTP Headers when possible<a class="headerlink"
href="#use-and-generate-standard-http-headers-when-possible" title="Permanent
link">¶</a></h3>
<p>If there is a standard HTTP header for an option (such as a
<code>range:</code> header), it MUST be used. This boosts compatibility with
browsers and existing applications.</p>
<p>The services MUST return information that defines cache duration of
retrieved data, possibly 0 seconds. Proxy caching MUST be disabled. (this comes
for free with the <code>NoCacheFilter</code> âtests are needed to verify the
filter is adding the values)</p>
-<h2 id="get-for-queries-and-only-queries">GET for queries âand only
queries</h2>
+<h2 id="get-for-queries-and-only-queries">GET for queries âand only
queries<a class="headerlink" href="#get-for-queries-and-only-queries"
title="Permanent link">¶</a></h2>
<ol>
<li>
<p>All side-effect free queries MUST be implemented via GET operations.</p>
@@ -479,23 +490,23 @@ with a READ-only JSON view of the cluste
</li>
</ol>
<p>Rule #1 is for a coherent REST API. Rule #2 is mandated in the HTTP
specification, and assumed to hold by those browsers which perform pre-emptive
fetching.</p>
-<h3
id="put-for-overwrites-to-existing-resources-or-explicit-creation-of-new-ones">PUT
for overwrites to existing resources, or explicit creation of new ones.</h3>
+<h3
id="put-for-overwrites-to-existing-resources-or-explicit-creation-of-new-ones">PUT
for overwrites to existing resources, or explicit creation of new ones.<a
class="headerlink"
href="#put-for-overwrites-to-existing-resources-or-explicit-creation-of-new-ones"
title="Permanent link">¶</a></h3>
<p>If a URL references a valid resource, and an update to it makes sense (e.g.
overwriting an existing resource topology with a new declaration), then the PUT
verb SHOULD be preferred to POST.</p>
<p>It MAY also be used for resource creation operations âbut only if the
result of the PUT is a new resource at the final URL specified.</p>
-<h2 id="put-operations-must-be-idempotent">PUT operations MUST be
idempotent</h2>
+<h2 id="put-operations-must-be-idempotent">PUT operations MUST be idempotent<a
class="headerlink" href="#put-operations-must-be-idempotent" title="Permanent
link">¶</a></h2>
<p>If a PUT operation is repeated, the final state of the model MUST be the
same.</p>
<p>Processing of the initial PUT may result in external/visible actions. These
actions SHOULD NOT be repeated when the second PUT is received. As an example,
a PUT, twice, of a new resources.json model should eventually result in the
final resource counts matching the desired state, without more container
creation and deletion than if a single PUT had occurred. </p>
<p>Operations which aren't idempotent MUST NOT be implemented as PUT; instead
they </p>
-<h2 id="delete-for-resource-deletion-operations">DELETE for resource deletion
operations</h2>
+<h2 id="delete-for-resource-deletion-operations">DELETE for resource deletion
operations<a class="headerlink" href="#delete-for-resource-deletion-operations"
title="Permanent link">¶</a></h2>
<p>If resources are to be deleted, then DELETE is the operation of choice.</p>
-<h3
id="post-operations-for-system-state-changes-that-do-not-match-a-resource-model">POST
operations for system state changes that do not match a resource model.</h3>
+<h3
id="post-operations-for-system-state-changes-that-do-not-match-a-resource-model">POST
operations for system state changes that do not match a resource model.<a
class="headerlink"
href="#post-operations-for-system-state-changes-that-do-not-match-a-resource-model"
title="Permanent link">¶</a></h3>
<p>Operations which do not map directly the resource model SHOULD be
implemented as POST operations. </p>
-<h3 id="post-operations-may-be-non-idempotent">POST operations MAY be
non-idempotent</h3>
+<h3 id="post-operations-may-be-non-idempotent">POST operations MAY be
non-idempotent<a class="headerlink"
href="#post-operations-may-be-non-idempotent" title="Permanent
link">¶</a></h3>
<p>There is no requirement for POST operations to be idempotent.</p>
-<h2 id="payloads-should-be-json-payloads">Payloads SHOULD be JSON payloads</h2>
+<h2 id="payloads-should-be-json-payloads">Payloads SHOULD be JSON payloads<a
class="headerlink" href="#payloads-should-be-json-payloads" title="Permanent
link">¶</a></h2>
<p>The bodies of operations SHOULD be JSON.</p>
<p>POST operations MAY ALSO support application/x-www-form-urlencoded, so as
to handle data directly from an HTML form.</p>
-<h2 id="errors-must-be-meaningful">Errors MUST be meaningful</h2>
+<h2 id="errors-must-be-meaningful">Errors MUST be meaningful<a
class="headerlink" href="#errors-must-be-meaningful" title="Permanent
link">¶</a></h2>
<p>If an operation fails enough information should be provided to aid
diagnosis of the problem.</p>
<ol>
<li>
@@ -509,17 +520,17 @@ with a READ-only JSON view of the cluste
</li>
</ol>
<p>Jersey is going to interfere here with its own exception logic; methods
must catch all exceptions and convert them to <code>WebAppException</code>
instances to avoid them being mishandled.</p>
-<h2 id="the-api-must-be-secure">The API MUST be Secure</h2>
+<h2 id="the-api-must-be-secure">The API MUST be Secure<a class="headerlink"
href="#the-api-must-be-secure" title="Permanent link">¶</a></h2>
<p>The REST API must be secure. In the context of a YARN application, this
means all communications in a secure cluster must be via the
Kerberos/SPNEGO-authenticated ResourceManager proxy.</p>
<p><strong>Development time exception</strong> : disable the proxy on the /ws/
path of the web application, so that the full set of HTTP verbs can be used,
without depending on Hadoop-2.7 proxy improvements.</p>
-<h2 id="minimal">Minimal</h2>
+<h2 id="minimal">Minimal<a class="headerlink" href="#minimal" title="Permanent
link">¶</a></h2>
<p>Features implemented via HDFS and YARN SHOULD NOT be re-implemented in the
Slider AM REST API. </p>
-<h2 id="asynchronous-actions-and-state-changes">Asynchronous Actions and
state-changes</h2>
+<h2 id="asynchronous-actions-and-state-changes">Asynchronous Actions and
state-changes<a class="headerlink"
href="#asynchronous-actions-and-state-changes" title="Permanent
link">¶</a></h2>
<p>All state changes are asynchronous, serialized and queued within the AM.</p>
<p>This is what happens today; there is some optimisation for handling
multiple cluster-size changing events in the queue such that a "review and
request containers" operation is postponed until all pending size-changing
events (flex, container-loss, ...) have been processed. </p>
<p>This means that while REST operations (and YARN events) are queued in the
order of receipt, some operations âsuch as a flex operationâ may not have
any work performed while later events arrive in the queue.</p>
<p>The response code to an asynchronous operation SHOULD be 201, ACCEPTED</p>
-<h1 id="yarn-dependencies">YARN dependencies</h1>
+<h1 id="yarn-dependencies">YARN dependencies<a class="headerlink"
href="#yarn-dependencies" title="Permanent link">¶</a></h1>
<p>What do we need from YARN?</p>
<ol>
<li>
@@ -541,7 +552,7 @@ with a READ-only JSON view of the cluste
<p>For a pure-REST client, a RESTy registry API in both YARN <em>and
KNOX.</em></p>
</li>
</ol>
-<h1 id="resource-structure">Resource Structure</h1>
+<h1 id="resource-structure">Resource Structure<a class="headerlink"
href="#resource-structure" title="Permanent link">¶</a></h1>
<p><img alt="Slider REST model" src="slider_REST_model_v2.jpg" title="Slider
REST Resource model" /></p>
<p>Core concepts:</p>
<ol>
@@ -552,12 +563,12 @@ with a READ-only JSON view of the cluste
<p>The live view of what is going on in the application under
<code>/application/model</code>.</p>
</li>
</ol>
-<h2 id="application">/application</h2>
-<h3 id="all-application-resources">All Application resources</h3>
+<h2 id="application">/application<a class="headerlink" href="#application"
title="Permanent link">¶</a></h2>
+<h3 id="all-application-resources">All Application resources<a
class="headerlink" href="#all-application-resources" title="Permanent
link">¶</a></h3>
<p>All entries will be under the service path <code>/application</code>, which
itself is under the <code>/ws/v1/</code> path of the Slider web interface.</p>
-<h2 id="applicationmodel">/application/model/ :</h2>
-<h3 id="get-and-for-some-urls-put-view-of-the-specification">GET/ and, for
some URLs, PUT view of the specification</h3>
-<h3 id="applicationmodeldesired">/application/model/desired/</h3>
+<h2 id="applicationmodel">/application/model/ :<a class="headerlink"
href="#applicationmodel" title="Permanent link">¶</a></h2>
+<h3 id="get-and-for-some-urls-put-view-of-the-specification">GET/ and, for
some URLs, PUT view of the specification<a class="headerlink"
href="#get-and-for-some-urls-put-view-of-the-specification" title="Permanent
link">¶</a></h3>
+<h3 id="applicationmodeldesired">/application/model/desired/<a
class="headerlink" href="#applicationmodeldesired" title="Permanent
link">¶</a></h3>
<p>This is where the specification of the application: resources and
configuration, can be read and written. </p>
<ol>
<li>
@@ -567,12 +578,12 @@ with a READ-only JSON view of the cluste
<p>Write accesses to <code>configuration</code> will only take effect on a
cluster upgrade or restart</p>
</li>
</ol>
-<h3 id="applicationmodelresolved">/application/model/resolved/</h3>
+<h3 id="applicationmodelresolved">/application/model/resolved/<a
class="headerlink" href="#applicationmodelresolved" title="Permanent
link">¶</a></h3>
<p>The resolved specification, the one where we implement the inheritance,
and, when we eventually do x-refs, all non-LAZY references. This lets the
caller see the final configuration model.</p>
-<h3 id="applicationmodelinternal">/application/model/internal/</h3>
+<h3 id="applicationmodelinternal">/application/model/internal/<a
class="headerlink" href="#applicationmodelinternal" title="Permanent
link">¶</a></h3>
<p>Read-only view of <code>internal.json</code>. Exported for diagnostics and
completeness.</p>
-<h2 id="applicationlive">/application/live/ :</h2>
-<h3 id="get-and-delete-view-of-the-live-application">GET and DELETE view of
the live application</h3>
+<h2 id="applicationlive">/application/live/ :<a class="headerlink"
href="#applicationlive" title="Permanent link">¶</a></h2>
+<h3 id="get-and-delete-view-of-the-live-application">GET and DELETE view of
the live application<a class="headerlink"
href="#get-and-delete-view-of-the-live-application" title="Permanent
link">¶</a></h3>
<p>This provides different views of the system, something which we can delve
into</p>
<ol>
<li>
@@ -601,10 +612,10 @@ DELETE node_id will decommission all con
<p>"system" state: AM state, outstanding requests, upgrade in progress</p>
</li>
</ol>
-<h2 id="applicationactions">/application/actions</h2>
-<h3 id="post-state-changing-operations">POST state changing operations</h3>
+<h2 id="applicationactions">/application/actions<a class="headerlink"
href="#applicationactions" title="Permanent link">¶</a></h2>
+<h3 id="post-state-changing-operations">POST state changing operations<a
class="headerlink" href="#post-state-changing-operations" title="Permanent
link">¶</a></h3>
<p>These are for operations which are hard to represent in a simple REST view
within the AM itself.</p>
-<h1 id="proposed-state-query-operations">Proposed State Query Operations</h1>
+<h1 id="proposed-state-query-operations">Proposed State Query Operations<a
class="headerlink" href="#proposed-state-query-operations" title="Permanent
link">¶</a></h1>
<p>All of these are GET operations on data that is not directly mutable</p>
<table>
<tr>
@@ -679,7 +690,7 @@ DELETE node_id will decommission all con
<p>All live values will be cached and refreshed regularly; the caching ensures
that a heavy load of GET operations does not overload the application master.
It does mean that there may be a delay of under a second
before an updated value is visible.</p>
-<h1 id="actions">Actions</h1>
+<h1 id="actions">Actions<a class="headerlink" href="#actions" title="Permanent
link">¶</a></h1>
<p>Actions are POST operations.</p>
<table>
<tr>
@@ -713,8 +724,8 @@ before an updated value is visible.</p>
<p>It MAY be possible to overwrite an existing operation with a new one,
though that will depend on the active operation. Specifically, "upgrade" would
only support STOP; "stop" would only support "stop". the empty operation, ""
will support anything</p>
</li>
</ol>
-<h1 id="non-normative-example-data-structures">Non-normative Example Data
structures</h1>
-<h2 id="applicationliveresources">application/live/resources</h2>
+<h1 id="non-normative-example-data-structures">Non-normative Example Data
structures<a class="headerlink" href="#non-normative-example-data-structures"
title="Permanent link">¶</a></h1>
+<h2 id="applicationliveresources">application/live/resources<a
class="headerlink" href="#applicationliveresources" title="Permanent
link">¶</a></h2>
<p>The contents of application/live/resources on an application which only has
an application master deployed. The entries in italic are the statistics
related to the live state; the remainder the original values.</p>
<div class="codehilite"><pre><span class="p">{</span>
"<span class="n">schema</span>" <span class="p">:</span>
"<span class="n">http</span><span class="p">:</span><span
class="o">//</span><span class="n">example</span><span class="p">.</span><span
class="n">org</span><span class="o">/</span><span
class="n">specification</span><span class="o">/</span><span
class="n">v2</span><span class="p">.</span>0<span
class="p">.</span>0"<span class="p">,</span>
@@ -741,7 +752,7 @@ before an updated value is visible.</p>
</pre></div>
-<h2 id="liveliveness"><code>live/liveness</code></h2>
+<h2 id="liveliveness"><code>live/liveness</code><a class="headerlink"
href="#liveliveness" title="Permanent link">¶</a></h2>
<p>The liveness URL returns a JSON structure on the liveness of the
application as perceived by Slider itself.</p>
<p>See
<code>org.apache.slider.api.types.ApplicationLivenessInformation</code></p>
@@ -757,7 +768,7 @@ application as perceived by Slider itsel
percentage of desired containers of each component type have been allocated
âor even how many were actually running.</p>
<p>Any new liveness probes will supplement rather than replace the current
values.</p>
-<h2 id="to-be-defined-livestatisticts">To be defined:
<code>live/statisticts</code></h2>
+<h2 id="to-be-defined-livestatisticts">To be defined:
<code>live/statisticts</code><a class="headerlink"
href="#to-be-defined-livestatisticts" title="Permanent link">¶</a></h2>
<p>The statistics will cover the collected statistics on a component type, as
well
as aggregate statistics on an application instance.</p>
<p>When an Application Master is restarted, all statistics will be lost.</p>
Modified: websites/staging/slider/trunk/content/docs/api/slider_REST_v1.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/api/slider_REST_v1.html
(original)
+++ websites/staging/slider/trunk/content/docs/api/slider_REST_v1.html Mon Nov
2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block"
href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink,
h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink,
dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,8 +193,8 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="slider-rest-apis-v1">Slider REST APIs v1</h1>
-<h2 id="december-2014">December 2014</h2>
+<h1 id="slider-rest-apis-v1">Slider REST APIs v1<a class="headerlink"
href="#slider-rest-apis-v1" title="Permanent link">¶</a></h1>
+<h2 id="december-2014">December 2014<a class="headerlink"
href="#december-2014" title="Permanent link">¶</a></h2>
<p>This document covers the external REST APIs of Slider</p>
<div class="codehilite"><pre> <span class="n">The</span> <span
class="n">key</span> <span class="n">words</span> "<span
class="n">MUST</span>"<span class="p">,</span> "<span
class="n">MUST</span> <span class="n">NOT</span>"<span class="p">,</span>
"<span class="n">REQUIRED</span>"<span class="p">,</span> "<span
class="n">SHALL</span>"<span class="p">,</span> "<span
class="n">SHALL</span>
<span class="n">NOT</span>"<span class="p">,</span> "<span
class="n">SHOULD</span>"<span class="p">,</span> "<span
class="n">SHOULD</span> <span class="n">NOT</span>"<span
class="p">,</span> "<span class="n">RECOMMENDED</span>"<span
class="p">,</span> "<span class="n">MAY</span>"<span
class="p">,</span> <span class="n">and</span>
@@ -192,11 +203,11 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h1 id="introduction-and-background">Introduction and Background</h1>
+<h1 id="introduction-and-background">Introduction and Background<a
class="headerlink" href="#introduction-and-background" title="Permanent
link">¶</a></h1>
<p>Slider 0.60 Uses Hadoop IPC for communications between the Slider client
and the per-instance application master. This uses protobuf-formatted payloads,
with the Hadoop IPC layer handling security: authorization, authentication and
encryption.</p>
<p>It provides read-only views of generated content and state through a set of
web service URLs. All these services are implemented by the Jersey JAX-RS
engine, operating in an embedded Jetty Web engine, with the YARN
<code>AmWebFilter</code> class redirecting all requests not coming from the RM
Proxy IP address to that RM Proxy via a 302 (redirect-as-GET)</p>
<p>Excluding the internal-use-only the AM<->Agent channel operating on a
private HTTP port, there are a number of current services</p>
-<h1 id="read-only-access-to-the-yarn-registry-wsv1sliderregistry">Read only
access to the YARN registry: /ws/v1/slider/registry</h1>
+<h1 id="read-only-access-to-the-yarn-registry-wsv1sliderregistry">Read only
access to the YARN registry: /ws/v1/slider/registry<a class="headerlink"
href="#read-only-access-to-the-yarn-registry-wsv1sliderregistry"
title="Permanent link">¶</a></h1>
<p>This was done primarily to get a browseable view of what the YARN-913
registry was publishing in the absence of any web view built in to that
registry. The unit tests against this service actually discovered some JSON
marshalling problems with the registry design and Jersey integration. </p>
<p>The root path <code>/v1/slider/registry</code> lists the root path of the
registry; all entries below it follow the path in the registry. </p>
<div class="codehilite"><pre><span class="o">/</span><span
class="n">ws</span><span class="o">/</span><span class="n">v1</span><span
class="o">/</span><span class="n">slider</span><span class="o">/</span><span
class="n">registry</span><span class="p">:</span> <span
class="p">{</span>"<span class="n">nodes</span>"<span
class="p">:[</span>"<span class="n">services</span>"<span
class="p">,</span>"<span class="n">users</span>"<span
class="p">]}</span>
@@ -272,7 +283,7 @@ Latest release: <strong>0.80.0-incubatin
<p>The slider code makes no direct use of this registry service except for a
pair of tests. The Agents MUST use the core (ZK-based) registry service to
locate a restarted AM. The slider client operations to enumerate and retrieve
configuration files could use it, but they currently also use the YARN registry
directly.</p>
<p>As and when the YARN registry itself implements a REST view (possibly an
R/W one), then this view will become obsolete âand possibly disabled.</p>
-<h1 id="generated-configurations-wsv1sliderpublisher">Generated
configurations: <code>/ws/v1/slider/publisher</code></h1>
+<h1 id="generated-configurations-wsv1sliderpublisher">Generated
configurations: <code>/ws/v1/slider/publisher</code><a class="headerlink"
href="#generated-configurations-wsv1sliderpublisher" title="Permanent
link">¶</a></h1>
<p>This service contains a one or more "published configuration sets"</p>
<div class="codehilite"><pre><span class="n">http</span><span
class="p">:</span><span class="o">//</span><span class="n">devix</span><span
class="p">.</span><span class="n">cotham</span><span class="p">.</span><span
class="n">uk</span><span class="p">:</span>8088<span class="o">/</span><span
class="n">proxy</span><span class="o">/</span><span
class="n">application_1418130391694_0002</span><span class="o">/</span><span
class="n">ws</span><span class="o">/</span><span class="n">v1</span><span
class="o">/</span><span class="n">slider</span><span class="o">/</span><span
class="n">publisher</span>
@@ -328,7 +339,7 @@ Latest release: <strong>0.80.0-incubatin
<p>This configuration publishing service is listed in the YARN Registry, and
could be implemented by other applications to publish their configurations
âincluding the standard YARN services.</p>
-<h1 id="application-state-in-slider-v1-format">Application state in slider v1
format</h1>
+<h1 id="application-state-in-slider-v1-format">Application state in slider v1
format<a class="headerlink" href="#application-state-in-slider-v1-format"
title="Permanent link">¶</a></h1>
<p>The original "v1" formatted slider specification/state is provided, wrapped
in HTML, at:</p>
<div class="codehilite"><pre><span class="n">http</span><span
class="p">:</span><span class="o">//</span><span class="n">devix</span><span
class="p">.</span><span class="n">cotham</span><span class="p">.</span><span
class="n">uk</span><span class="p">:</span>8088<span class="o">/</span><span
class="n">proxy</span><span class="o">/</span><span
class="n">application_1418130391694_0002</span><span class="o">/</span><span
class="n">slideram</span><span class="o">/</span><span class="n">spec</span>
</pre></div>
@@ -336,7 +347,7 @@ Latest release: <strong>0.80.0-incubatin
<p>This combines the declaration of what is desired with information about
what has actually been provided</p>
<p>(this shows the merits of a single file model which can describe both the
desired state and the actual state; the same architecture of <a
href="http://www.hpl.hp.com/techreports/2012/HPL-2012-158R1.pdf";>the HPL Cells
cloud infrastructure</a>)</p>
-<h1 id="slider-070-codahale-metrics-servlets">Slider 0.70: codahale metrics
servlets</h1>
+<h1 id="slider-070-codahale-metrics-servlets">Slider 0.70: codahale metrics
servlets<a class="headerlink" href="#slider-070-codahale-metrics-servlets"
title="Permanent link">¶</a></h1>
<p>Slider 0.70 is adding metrics via the Codahale metrics library; as well as
(optionally) publishing to Ganglia, the metrics service export metrics,health
and liveness information, as well as a thread dump</p>
<table>
<tr>
Modified: websites/staging/slider/trunk/content/docs/client-configuration.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/client-configuration.html
(original)
+++ websites/staging/slider/trunk/content/docs/client-configuration.html Mon
Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block"
href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink,
h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink,
dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,9 +196,9 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-client-configuration">Apache Slider Client
Configuration</h1>
+<h1 id="apache-slider-client-configuration">Apache Slider Client
Configuration<a class="headerlink" href="#apache-slider-client-configuration"
title="Permanent link">¶</a></h1>
<p>This document covers how the client application is itself configured.</p>
-<h2 id="summary">Summary</h2>
+<h2 id="summary">Summary<a class="headerlink" href="#summary" title="Permanent
link">¶</a></h2>
<p>The client application can be configured</p>
<ol>
<li>On the command line, which can set client options and JVM system
properties.</li>
@@ -201,7 +212,7 @@ directory.</li>
</ol>
<p>The options defined in a Slider cluster configuration are only used by the
client
when creating a cluster -not for the actual client itself.</p>
-<h2 id="introduction">Introduction</h2>
+<h2 id="introduction">Introduction<a class="headerlink" href="#introduction"
title="Permanent link">¶</a></h2>
<p>The Slider client needs to be configured to talk to a Hadoop filesystem and
a
YARN resource manager ("the RM"). In a secure cluster it needs to be told the
Kerberos
identity, the <em>principal</em> of both the HDFS namenode and the YARN RM
-and it may
@@ -213,7 +224,7 @@ work on client machines that may not hav
<p>If these files are present, and the <code>HADOOP_CONF_DIR</code>
environment variable is
set to the directory that contains them âslider will pick up the settings in
these files. If not, all settings must be placed in the file
<code>slider-client.xml</code></p>
-<h2 id="setting-slider-jvm-options">Setting Slider JVM options</h2>
+<h2 id="setting-slider-jvm-options">Setting Slider JVM options<a
class="headerlink" href="#setting-slider-jvm-options" title="Permanent
link">¶</a></h2>
<p>Core JVM options can be set in the environment variable
<code>SLIDER_JVM_OPTS</code>;
if unset the <code>bin/slider</code> script will use the default values that
were
current when that version of Slider was released. These values may change
@@ -244,7 +255,7 @@ duplicate declarations. In such a case t
<p>For any system property that the user expects to have to issue on every
command
-including any kerberos-related properties, adding them to the JVM options
environment variable guarantees that they are always set.</p>
-<h2
id="setting-slider-client-options-on-the-command-line-with-the-d-parameter">Setting
Slider client options on the command line with the <code>-D</code>
parameter</h2>
+<h2
id="setting-slider-client-options-on-the-command-line-with-the-d-parameter">Setting
Slider client options on the command line with the <code>-D</code> parameter<a
class="headerlink"
href="#setting-slider-client-options-on-the-command-line-with-the-d-parameter"
title="Permanent link">¶</a></h2>
<p>The slider client is configured via Hadoop-style configuration options.
To be precise, all standard Hadoop-common, hadoop-hdfs client and hadoop-yar
client-side options control how Slider communicates with the Hadoop YARN
cluster.</p>
@@ -268,7 +279,7 @@ parameter followed by the appropriate <c
<p>If duplicate declarations are made the order of assignment is undefined.</p>
-<h1
id="setting-common-options-through-specific-command-line-arguments">Setting
common options through specific command-line arguments</h1>
+<h1
id="setting-common-options-through-specific-command-line-arguments">Setting
common options through specific command-line arguments<a class="headerlink"
href="#setting-common-options-through-specific-command-line-arguments"
title="Permanent link">¶</a></h1>
<p>Some Hadoop and Slider options are so common that they have specific
shortcut commands to aid their use</p>
<p><code>-m</code>, <code>--manager</code> : sets the YARN resource manager.
Equivalent to setting the
@@ -277,7 +288,7 @@ shortcut commands to aid their use</p>
<code>fs.defaultFS</code> option</p>
<p>If these shortcuts are used and the options are also defined via
<code>-D</code>
declarations, the order of assignment is undefined.</p>
-<h1
id="defining-hadoop-and-slider-options-in-the-slider-clientxml-file">Defining
Hadoop and Slider Options in the <code>slider-client.xml</code> file.</h1>
+<h1
id="defining-hadoop-and-slider-options-in-the-slider-clientxml-file">Defining
Hadoop and Slider Options in the <code>slider-client.xml</code> file.<a
class="headerlink"
href="#defining-hadoop-and-slider-options-in-the-slider-clientxml-file"
title="Permanent link">¶</a></h1>
<p>In the Slider installation, alongside the <code>bin/slider</code> script is
a configuration directory <code>conf</code>. This contains the files:</p>
<ol>
@@ -312,11 +323,11 @@ to declare either on the command line.</
<p>If an option is defined in the <code>slider-client.xml</code> file and on
the command line
-be it by a <code>-D key=value</code> declaration or a <code>--manager</code>
or <code>--filesystem</code>
definition. (this holds even if the value is declared with
<code><final>true</final></code>).</p>
-<h2 id="selecting-an-alternate-slider-configuration-directory">Selecting an
alternate Slider configuration directory</h2>
+<h2 id="selecting-an-alternate-slider-configuration-directory">Selecting an
alternate Slider configuration directory<a class="headerlink"
href="#selecting-an-alternate-slider-configuration-directory" title="Permanent
link">¶</a></h2>
<p>The environment variable <code>SLIDER_CONF_DIR</code> can be used to
declare an alternate
configuration directory. If set, the directory it identifies will be used
as the source of the <code>log4j.properties</code> and
<code>slider-client.xml</code> files.</p>
-<h2 id="slider-client-configuration-options">Slider Client Configuration
options</h2>
+<h2 id="slider-client-configuration-options">Slider Client Configuration
options<a class="headerlink" href="#slider-client-configuration-options"
title="Permanent link">¶</a></h2>
<p>As well as standard YARN and Hadoop configuration options, Slider supports
a limited number of slider-specific configuration parameters.</p>
<div class="codehilite"><pre><span class="nt"><property></span>
@@ -348,7 +359,7 @@ a limited number of slider-specific conf
</pre></div>
-<h3
id="slideryarnrestartlimit-set-limit-on-application-master-restarts"><code>slider.yarn.restart.limit</code>
- set limit on Application Master Restarts</h3>
+<h3
id="slideryarnrestartlimit-set-limit-on-application-master-restarts"><code>slider.yarn.restart.limit</code>
- set limit on Application Master Restarts<a class="headerlink"
href="#slideryarnrestartlimit-set-limit-on-application-master-restarts"
title="Permanent link">¶</a></h3>
<p>This limits how many times YARN should start a failed application
master.</p>
<p>A short restart limit is useful when initially creating a cluster, as it
ensures that YARN does not repeatedly try to restart a failing application.</p>
@@ -361,12 +372,12 @@ permitted.</p>
places an upper limit on the number of retries that any application can
request.
If the application fails after less restarts than requested, check this cluster
setting.</p>
-<h3
id="slideryarnqueue-the-name-of-the-yarn-queue-for-the-cluster"><code>slider.yarn.queue</code>
- the name of the YARN queue for the cluster.</h3>
+<h3
id="slideryarnqueue-the-name-of-the-yarn-queue-for-the-cluster"><code>slider.yarn.queue</code>
- the name of the YARN queue for the cluster.<a class="headerlink"
href="#slideryarnqueue-the-name-of-the-yarn-queue-for-the-cluster"
title="Permanent link">¶</a></h3>
<p>This identifies the queue submit the application creation request to, which
can
define the priority, resource limits and other values of an application. All
containers created in the Slider cluster will share this same queue.</p>
<p>Default value: <code>default</code>.</p>
-<h3
id="slideryarnqueuepriority-the-name-of-the-yarn-queue-for-the-cluster"><code>slider.yarn.queue.priority</code>
- the name of the YARN queue for the cluster.</h3>
+<h3
id="slideryarnqueuepriority-the-name-of-the-yarn-queue-for-the-cluster"><code>slider.yarn.queue.priority</code>
- the name of the YARN queue for the cluster.<a class="headerlink"
href="#slideryarnqueuepriority-the-name-of-the-yarn-queue-for-the-cluster"
title="Permanent link">¶</a></h3>
<p>This identifies the priority within the queue. The lower the value, the
higher the
priority</p>
<p>Default value: <code>1</code>.</p>
@@ -374,7 +385,7 @@ priority</p>
</pre></div>
-<h4
id="sliderclusterdirectorypermissions"><code>slider.cluster.directory.permissions</code></h4>
+<h4
id="sliderclusterdirectorypermissions"><code>slider.cluster.directory.permissions</code><a
class="headerlink" href="#sliderclusterdirectorypermissions" title="Permanent
link">¶</a></h4>
<p>An octal-format (<code>chmod</code>-style) permissions mask for the
directory
that contains the cluster specification
<code>${user.home}/.slider/clusters/${clustername}</code></p>
<div class="codehilite"><pre><span class="nt"><property></span>
@@ -384,7 +395,7 @@ that contains the cluster specification
</pre></div>
-<h4
id="sliderdatadirectorypermissions"><code>slider.data.directory.permissions</code></h4>
+<h4
id="sliderdatadirectorypermissions"><code>slider.data.directory.permissions</code><a
class="headerlink" href="#sliderdatadirectorypermissions" title="Permanent
link">¶</a></h4>
<p>An octal-format (<code>chmod</code>-style) permissions mask for the
directory
that contains the application data
<code>${user.home}/.slider/clusters/${clustername}/database</code></p>
<div class="codehilite"><pre><span class="nt"><property></span>
@@ -394,7 +405,7 @@ that contains the application data <code
</pre></div>
-<h2 id="yarn-registry-binding">YARN Registry binding</h2>
+<h2 id="yarn-registry-binding">YARN Registry binding<a class="headerlink"
href="#yarn-registry-binding" title="Permanent link">¶</a></h2>
<p>Slider uses the (evolving) YARN service registry to register application
instances, to locate them, and as a means to publish the configuration
information
needed to connect to the slider-deployed applications.</p>
@@ -403,7 +414,7 @@ for a cluster, for use by all applicatio
Resource Manager is set to automatically create the paths in the registry for
users âa required feature in secure clustersâ then it must share the same
settings.</p>
<p>Consult the YARN registry documentation for more information.</p>
-<h3
id="hadoopregistryzkquorum-the-zookeeper-quorum"><code>hadoop.registry.zk.quorum</code>
- the zookeeper quorum.</h3>
+<h3
id="hadoopregistryzkquorum-the-zookeeper-quorum"><code>hadoop.registry.zk.quorum</code>
- the zookeeper quorum.<a class="headerlink"
href="#hadoopregistryzkquorum-the-zookeeper-quorum" title="Permanent
link">¶</a></h3>
<p>This defines the zookeeper quorum for this YARN cluster. </p>
<p>It is used to locate the service registry, enable running instances to
publish
information about their application, and for clients to query this. </p>
@@ -418,7 +429,7 @@ defined will be copied into the instance
</pre></div>
-<h3
id="hadoopregistryzkroot-the-zookeeper-path-for-the-service-registry"><code>hadoop.registry.zk.root</code>
- the zookeeper path for the service registry</h3>
+<h3
id="hadoopregistryzkroot-the-zookeeper-path-for-the-service-registry"><code>hadoop.registry.zk.root</code>
- the zookeeper path for the service registry<a class="headerlink"
href="#hadoopregistryzkroot-the-zookeeper-path-for-the-service-registry"
title="Permanent link">¶</a></h3>
<p>This declares the the zookeeper path for the service registry. </p>
<div class="codehilite"><pre><span class="nt"><property></span>
<span class="nt"><name></span>hadoop.registry.zk.root<span
class="nt"></name></span>
@@ -427,10 +438,10 @@ defined will be copied into the instance
</pre></div>
-<h2 id="debugging-configuration-issues">Debugging configuration issues</h2>
+<h2 id="debugging-configuration-issues">Debugging configuration issues<a
class="headerlink" href="#debugging-configuration-issues" title="Permanent
link">¶</a></h2>
<p>If the slider packages are set to log at debug level in the log4j
configuration
file, details on properties will be part of the copious output.</p>
-<h2 id="how-client-options-are-passed-down-to-created-clusters">How client
options are passed down to created clusters.</h2>
+<h2 id="how-client-options-are-passed-down-to-created-clusters">How client
options are passed down to created clusters.<a class="headerlink"
href="#how-client-options-are-passed-down-to-created-clusters" title="Permanent
link">¶</a></h2>
<p>Most cient configuration options are
not passed down to the XML site specification of the created cluster.</p>
<p>Exceptions:</p>
Modified:
websites/staging/slider/trunk/content/docs/configuration/appconfig.html.html
==============================================================================
---
websites/staging/slider/trunk/content/docs/configuration/appconfig.html.html
(original)
+++
websites/staging/slider/trunk/content/docs/configuration/appconfig.html.html
Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block"
href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink,
h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink,
dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,7 +193,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="title">title</h1>
+<h1 id="title">title<a class="headerlink" href="#title" title="Permanent
link">¶</a></h1>
</div>
<div id="footer">
Modified: websites/staging/slider/trunk/content/docs/configuration/core.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/configuration/core.html
(original)
+++ websites/staging/slider/trunk/content/docs/configuration/core.html Mon Nov
2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block"
href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink,
h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink,
dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,8 +196,8 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-core-configuration-specification-version-20">Apache
Slider Core Configuration Specification, version 2.0</h1>
-<h2 id="terminology">Terminology</h2>
+<h1 id="apache-slider-core-configuration-specification-version-20">Apache
Slider Core Configuration Specification, version 2.0<a class="headerlink"
href="#apache-slider-core-configuration-specification-version-20"
title="Permanent link">¶</a></h1>
+<h2 id="terminology">Terminology<a class="headerlink" href="#terminology"
title="Permanent link">¶</a></h2>
<p><em>Application</em> A single application, such as an HBase cluster. An
application
is distribed across the YARN cluster.</p>
<p><em>Component</em> A single executable part of the larger application. An
application
@@ -205,7 +216,7 @@ size of the application in terms of its
and what their resource requirements are. </p>
<p><em><code>internal.json</code></em>: A file which contains Slider's
internal configuration
parameters.</p>
-<h2 id="structure">Structure</h2>
+<h2 id="structure">Structure<a class="headerlink" href="#structure"
title="Permanent link">¶</a></h2>
<p>Configurations are stored in well-formed JSON files.
1. Text MUST be saved in the UTF-8 format.
1. Duplicate entries MUST NOT occur in any section.
@@ -241,7 +252,7 @@ author, or any other information that is
</pre></div>
-<h2 id="property-inheritance-model-and-resolution">Property inheritance model
and <em>resolution</em></h2>
+<h2 id="property-inheritance-model-and-resolution">Property inheritance model
and <em>resolution</em><a class="headerlink"
href="#property-inheritance-model-and-resolution" title="Permanent
link">¶</a></h2>
<p>There is a simple global to component inheritance model.</p>
<ol>
<li>Properties defined in <code>/global</code> define parameters across the
entire application.</li>
@@ -258,7 +269,7 @@ MAY be added in future.</li>
<li>There is NO sharing of information from the different <code>.json</code>
files in a
an application configuration.</li>
</ol>
-<h3 id="example">Example</h3>
+<h3 id="example">Example<a class="headerlink" href="#example" title="Permanent
link">¶</a></h3>
<p>Here is an example configuration</p>
<div class="codehilite"><pre><span class="p">{</span>
"<span class="n">schema</span>"<span class="p">:</span>
"<span class="n">http</span><span class="p">:</span><span
class="o">//</span><span class="n">example</span><span class="p">.</span><span
class="n">org</span><span class="o">/</span><span
class="n">specification</span><span class="o">/</span><span
class="n">v2</span><span class="p">.</span>0<span
class="p">.</span>0"<span class="p">,</span>
@@ -294,7 +305,7 @@ an application configuration.</li>
<p>These are the values visible to any part of the application which is
not itself one of the components. </p>
<p>There are three components defined, <code>simple</code>,
<code>master</code> and <code>worker</code>.</p>
-<h4 id="component-simple">component <code>simple</code>:</h4>
+<h4 id="component-simple">component <code>simple</code>:<a class="headerlink"
href="#component-simple" title="Permanent link">¶</a></h4>
<div class="codehilite"><pre><span class="n">g1</span><span
class="p">=</span>"<span class="n">a</span>"
<span class="n">g2</span><span class="p">=</span>"<span
class="n">b</span>"
</pre></div>
@@ -302,7 +313,7 @@ not itself one of the components. </p>
<p>No settings have been defined specifically for the component; the global
settings are applied.</p>
-<h4 id="component-master">component <code>master</code>:</h4>
+<h4 id="component-master">component <code>master</code>:<a class="headerlink"
href="#component-master" title="Permanent link">¶</a></h4>
<div class="codehilite"><pre><span class="n">name</span><span
class="p">=</span>"<span class="n">m</span>"<span class="p">,</span>
<span class="n">g1</span><span class="p">=</span>"<span
class="n">overridden</span>"
<span class="n">g2</span><span class="p">=</span>"<span
class="n">b</span>"
@@ -312,7 +323,7 @@ settings are applied.</p>
<p>A new attribute, <code>name</code>, has been defined with the value
<code>"m"</code>, and the
global property <code>g1</code> has been overridden with the new value,
<code>"overridden"</code>.
The global property <code>g2</code> is passed down unchanged.</p>
-<h4 id="component-worker">component <code>worker</code>:</h4>
+<h4 id="component-worker">component <code>worker</code>:<a class="headerlink"
href="#component-worker" title="Permanent link">¶</a></h4>
<div class="codehilite"><pre><span class="n">name</span><span
class="p">=</span>"<span class="n">w</span>"<span class="p">,</span>
<span class="n">g1</span><span class="p">=</span>"<span
class="n">overridden</span><span class="o">-</span><span
class="n">by</span><span class="o">-</span><span class="n">worker</span>"
<span class="n">g2</span><span class="p">=</span>"<span
class="n">b</span>"
@@ -366,12 +377,12 @@ in the example, without any propagation
to individual components. </p>
<p>Note that a resolved specification can still have the resolution operation
applied
to it -it just does not have any effect.</p>
-<h2 id="metadata">Metadata</h2>
+<h2 id="metadata">Metadata<a class="headerlink" href="#metadata"
title="Permanent link">¶</a></h2>
<p>The metadata section can contain arbitrary string values for use in
diagnostics
and by other applications.</p>
<p>To avoid conflict with other applications, please use a unique name in
strings,
such as java-style package names.</p>
-<h1 id="resource-requirements-resourcesjson">Resource Requirements:
<code>resources.json</code></h1>
+<h1 id="resource-requirements-resourcesjson">Resource Requirements:
<code>resources.json</code><a class="headerlink"
href="#resource-requirements-resourcesjson" title="Permanent
link">¶</a></h1>
<p>This file declares the resource requirements for YARN for the components
of an application.</p>
<p><code>instances</code>: the number of instances of a role desired.
@@ -434,7 +445,7 @@ of an application.</p>
<p>This declares this deployment of the application to consist of one instance
of
the master component, using 1 vcore and 1024MB of RAM, and five worker
components
each using one vcore and 512 MB of RAM.</p>
-<h2 id="deployment-specification-app_configurationjson">Deployment
specification: <code>app_configuration.json</code></h2>
+<h2 id="deployment-specification-app_configurationjson">Deployment
specification: <code>app_configuration.json</code><a class="headerlink"
href="#deployment-specification-app_configurationjson" title="Permanent
link">¶</a></h2>
<p>This defines parameters that are to be used when creating the instance of
the
application, and instances of the individual components.</p>
<div class="codehilite"><pre><span class="p">{</span>
@@ -522,10 +533,10 @@ does not prevent any component from over
to create local configurations incompatible with the global state. (i.e.,
there is no way to declare an attribute as final). It is the responsibility
of the author of the configuration file (and their tools) to detect such
issues.</p>
-<h3 id="key-application-configuration-items">Key Application Configuration
Items</h3>
+<h3 id="key-application-configuration-items">Key Application Configuration
Items<a class="headerlink" href="#key-application-configuration-items"
title="Permanent link">¶</a></h3>
<p>The following sections provides details about certain application
configuration
properties that can be utilized to tailor the deployment of a given
application:</p>
-<h4 id="controlling-assigned-port-ranges">Controlling assigned port ranges</h4>
+<h4 id="controlling-assigned-port-ranges">Controlling assigned port ranges<a
class="headerlink" href="#controlling-assigned-port-ranges" title="Permanent
link">¶</a></h4>
<p>For certain deployments, the ports available for communication with clients
(Web UI ports, RPC ports, etc) are restricted to a specific set (e.g when
using a firewall).
In those situations you can designate the set of allowed ports with the
@@ -537,7 +548,7 @@ In those situations you can designate th
<p>The AM exposed ports (Web UI, RPC), as well as the ports allocated to
launched
application containers, will be limited to the ranges specified by the
property value.</p>
-<h4 id="delaying-container-launch">Delaying container launch</h4>
+<h4 id="delaying-container-launch">Delaying container launch<a
class="headerlink" href="#delaying-container-launch" title="Permanent
link">¶</a></h4>
<p>In situations where container restarts may need to be delayed to allow for
platform resources to be released (e.g. a port assigned to a previous container
may be slow to release), a delay can be designated by setting the
<code>container.launch.delay.sec</code> property.</p>
@@ -548,7 +559,7 @@ may be slow to release), a delay can be
</pre></div>
-<h4 id="specifying-the-python-executable-path">Specifying the Python
Executable Path</h4>
+<h4 id="specifying-the-python-executable-path">Specifying the Python
Executable Path<a class="headerlink"
href="#specifying-the-python-executable-path" title="Permanent
link">¶</a></h4>
<p>Slider containers use python for component scripts in the containers.
When deploying applications on certain variations of linux or other operating
systems (e.g. Centos 5),
the version of python on the system PATH may be incompatible with the
component script
Modified: websites/staging/slider/trunk/content/docs/configuration/index.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/configuration/index.html
(original)
+++ websites/staging/slider/trunk/content/docs/configuration/index.html Mon Nov
2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block"
href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink,
h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink,
dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,7 +196,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1
id="apache-slider-specification-of-an-application-instance-revision-20">Apache
Slider: Specification of an application instance, revision 2.0</h1>
+<h1
id="apache-slider-specification-of-an-application-instance-revision-20">Apache
Slider: Specification of an application instance, revision 2.0<a
class="headerlink"
href="#apache-slider-specification-of-an-application-instance-revision-20"
title="Permanent link">¶</a></h1>
<p>The specification of an application comprises</p>
<ol>
<li>The persistent description of an application's configuration</li>
