Author: buildbot
Date: Thu Oct 9 01:03:09 2014
New Revision: 925107
Log:
Staging update by buildbot for slider
Added:
websites/staging/slider/trunk/content/docs/getting_started_40.html
Modified:
websites/staging/slider/trunk/content/ (props changed)
websites/staging/slider/trunk/content/docs/getting_started.html
websites/staging/slider/trunk/content/docs/manpage.html
websites/staging/slider/trunk/content/docs/slider_specs/resource_specification.html
Propchange: websites/staging/slider/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Thu Oct 9 01:03:09 2014
@@ -1 +1 @@
-1630262
+1630264
Modified: websites/staging/slider/trunk/content/docs/getting_started.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/getting_started.html (original)
+++ websites/staging/slider/trunk/content/docs/getting_started.html Thu Oct 9
01:03:09 2014
@@ -173,6 +173,8 @@ Latest release: <strong>0.40-incubating<
-->
<h1 id="apache-slider-getting-started">Apache Slider: Getting Started</h1>
+<p>This page is updated to reflect the latest code in "develop".</p>
+<p>For the version applicable to the last release go to <a
href="/docs/getting_started_40.html">Slider-40</a>.</p>
<h2 id="introduction">Introduction</h2>
<p>The following provides the steps required for setting up a cluster and
deploying a YARN hosted application using Slider.</p>
<ul>
@@ -205,7 +207,7 @@ Latest release: <strong>0.40-incubating<
<p>The Slider deployment has the following minimum system requirements:</p>
<ul>
<li>
-<p>Hadoop 2.4+</p>
+<p>Hadoop 2.6+</p>
</li>
<li>
<p>Required Services: HDFS, YARN and ZooKeeper</p>
@@ -221,8 +223,9 @@ Latest release: <strong>0.40-incubating<
</li>
</ul>
<h2 id="wzxhzdk22wzxhzdk23setup-the-cluster"><a name="setup"></a>Setup the
Cluster</h2>
-<p>After setting up your Hadoop cluster (using Ambari or other means) with the
-services listed above, modify your YARN configuration to allow for multiple
+<p>Setting up your Hadoop cluster with the services listed above.</p>
+<p>If you are using a one-box or a sandbox then you may need to modify your
YARN
+configuration to allow for multiple, modify your YARN configuration to allow
for multiple
containers on a single host. In <code>yarn-site.xml</code> make the following
modifications:</p>
<table>
<tr>
@@ -241,13 +244,11 @@ containers on a single host. In <code>ya
<p>There are other options detailed in the Troubleshooting file available <a
href="/docs/troubleshooting.html">here</a>.</p>
<h2 id="wzxhzdk24wzxhzdk25download-slider-packages"><a
name="download"></a>Download Slider Packages</h2>
-<p>Slider releases are available at
-<a
href="https://www.apache.org/dyn/closer.cgi/incubator/slider";>https://www.apache.org/dyn/closer.cgi/incubator/slider</a>.</p>
-<p><em>Or, you can build it as described below.</em></p>
+<p><em>You can build it as described below.</em></p>
<h2 id="wzxhzdk26wzxhzdk27build-slider"><a name="build"></a>Build Slider</h2>
<ul>
<li>From the top level directory, execute <code>mvn clean site:site site:stage
package -DskipTests</code></li>
-<li>Use the generated compressed tar file in slider-assembly/target directory
(e.g. slider-0.40.0-all.tar.gz or slider-0.40.0-all.zip) for the subsequent
steps</li>
+<li>Use the generated compressed tar file in slider-assembly/target directory
(e.g. slider-0.50.0-all.tar.gz or slider-0.50.0-all.zip) for the subsequent
steps</li>
<li>If you are cloning the Slider git repo, go to <code>master</code> branch
for the latest released or <code>develop</code> for the latest under
development</li>
</ul>
<h2 id="wzxhzdk28wzxhzdk29install-slider"><a name="install"></a>Install
Slider</h2>
@@ -258,39 +259,40 @@ cd <span class="cp">${</span><span class
</pre></div>
+<p>You can run Slider app as any user. Only requirement is that the user
should have a home directory on HDFS. For the reminder of the doc, it is
assumed that "yarn" user is being used.</p>
<p>Login as the "yarn" user (assuming this is a host associated with the
installed cluster). E.g., <code>su yarn</code>
-<em>This assumes that all apps are being run as âyarnâ user. Any other
user can be used to run the apps - ensure that file permission is granted as
required.</em></p>
-<p>Expand the tar file: <code>tar -xvf
slider-0.40.0-incubating-all.tar.gz</code> or <code>unzip
slider-0.40.0-incubating-all.zip</code></p>
-<p>Browse to the Slider directory: <code>cd slider-0.40.0/bin</code></p>
-<p><em>Add java to the PATH.</em></p>
-<div class="codehilite"><pre> <span class="n">export</span> <span
class="n">PATH</span><span class="p">=</span>$<span class="n">PATH</span><span
class="p">:</span><span class="o">/</span><span class="n">usr</span><span
class="o">/</span><span class="n">jdk64</span><span class="o">/</span><span
class="n">jdk1</span><span class="p">.</span>7<span class="p">.</span>0<span
class="n">_45</span><span class="o">/</span><span class="n">bin</span>
+<em>This assumes that all apps are being run as âyarnâ user</em></p>
+<p>Expand the tar file: <code>tar -xvf
slider-0.50.0-incubating-all.tar.gz</code> or <code>unzip
slider-0.50.0-incubating-all.zip</code></p>
+<h2 id="configure-slider">Configure Slider</h2>
+<p>Browse to the Slider directory: <code>cd slider-0.50.0/conf</code></p>
+<p>Edit slider-env.sh and specify correct values.</p>
+<div class="codehilite"><pre><span class="n">export</span> <span
class="n">JAVA_HOME</span><span class="p">=</span><span class="o">/</span><span
class="n">usr</span><span class="o">/</span><span class="n">jdk64</span><span
class="o">/</span><span class="n">jdk1</span><span class="p">.</span>7<span
class="p">.</span>0<span class="n">_67</span>
+<span class="n">export</span> <span class="n">HADOOP_CONF_DIR</span><span
class="p">=</span><span class="o">/</span><span class="n">etc</span><span
class="o">/</span><span class="n">hadoop</span><span class="o">/</span><span
class="n">conf</span>
</pre></div>
-<p>(or the path to the JDK bin directory)</p>
-<p>Modify Slider configuration file
<code>${slider-install-dir}/slider-0.40.0/conf/slider-client.xml</code> to add
the following properties:</p>
+<p>If you are on a node that does not have the hadoop conf folder then you can
add the relevant configurations into slider-client.xml.</p>
+<p>For example, if you are targeting a non-secure cluster with no HA for
NameNode or ResourceManager, modify Slider configuration file
<code>${slider-install-dir}/slider-0.50.0/conf/slider-client.xml</code> to add
the following properties:</p>
<div class="codehilite"><pre> <span class="nt"><property></span>
<span class="nt"><name></span>yarn.application.classpath<span
class="nt"></name></span>
<span
class="nt"><value></span>/etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*<span
class="nt"></value></span>
<span class="nt"></property></span>
-
<span class="nt"><property></span>
<span class="nt"><name></span>slider.zookeeper.quorum<span
class="nt"></name></span>
<span class="nt"><value></span>yourZooKeeperHost:port<span
class="nt"></value></span>
<span class="nt"></property></span>
-
-<span class="nt"><property></span>
+ <span class="nt"><property></span>
<span class="nt"><name></span>yarn.resourcemanager.address<span
class="nt"></name></span>
<span class="nt"><value></span>yourResourceManagerHost:8050<span
class="nt"></value></span>
-<span class="nt"></property></span>
-<span class="nt"><property></span>
+ <span class="nt"></property></span>
+ <span class="nt"><property></span>
<span
class="nt"><name></span>yarn.resourcemanager.scheduler.address<span
class="nt"></name></span>
<span class="nt"><value></span>yourResourceManagerHost:8030<span
class="nt"></value></span>
-<span class="nt"></property></span>
-<span class="nt"><property></span>
+ <span class="nt"></property></span>
+ <span class="nt"><property></span>
<span class="nt"><name></span>fs.defaultFS<span
class="nt"></name></span>
<span class="nt"><value></span>hdfs://yourNameNodeHost:8020<span
class="nt"></value></span>
-<span class="nt"></property></span>
+ <span class="nt"></property></span>
</pre></div>
@@ -305,13 +307,13 @@ cd <span class="cp">${</span><span class
<p>Execute:</p>
-<div class="codehilite"><pre><span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span class="cp">}</span>/slider-0.40.0/bin/slider version
+<div class="codehilite"><pre><span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span class="cp">}</span>/slider-0.50.0/bin/slider version
OR
-python %slider-install-dir%/slider-0.40.0/bin/slider.py version
+python %slider-install-dir%/slider-0.50.0/bin/slider.py version
</pre></div>
-<p>Ensure there are no errors and you can see "Compiled against Hadoop 2.4.0".
<strong>This ensures that Slider is installed correctly.</strong></p>
+<p>Ensure there are no errors and you can see "Compiled against Hadoop 2.6.0".
<strong>This ensures that Slider is installed correctly.</strong></p>
<h2 id="wzxhzdk30wzxhzdk31deploy-slider-resources"><a name="deploy"></a>Deploy
Slider Resources</h2>
<p>Ensure that all file folders are accessible to the user creating the
application instance. The example assumes "yarn" to be that user. The default
yarn user may be different e.g. hadoop.</p>
<h3 id="ensure-hdfs-home-folder-for-the-user">Ensure HDFS home folder for the
User</h3>
@@ -324,15 +326,6 @@ python %slider-install-dir%/slider-0.40.
</pre></div>
-<h3 id="load-slider-agent">Load Slider Agent</h3>
-<div class="codehilite"><pre>su yarn
-
-hdfs dfs -mkdir /user/yarn/agent
-
-hdfs dfs -copyFromLocal <span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span
class="cp">}</span>/slider-0.40.0/agent/slider-agent.tar.gz /user/yarn/agent
-</pre></div>
-
-
<h2 id="wzxhzdk32wzxhzdk33create-application-packages"><a
name="downsample"></a>Create Application Packages</h2>
<p>Slider "master" branch has several sample application packages available
for use with Slider:</p>
<ul>
@@ -346,14 +339,14 @@ hdfs dfs -copyFromLocal <span class="cp"
<p>app-packages/accumulo: The README.txt file describes how to create a Slider
Application Package for Accumulo.</p>
</li>
<li>
-<p>app-packages/storm: The README.txt describes how to create the Slider
Application package for Storm.</p>
+<p>app-packages/storm or storm-win: The README.txt describes how to create the
Slider Application package for Storm.</p>
</li>
</ul>
<p>Create one or more Slider application packages and follow the steps below
to install them.</p>
<h2
id="wzxhzdk34wzxhzdk35install-configure-start-and-verify-sample-application"><a
name="installapp"></a>Install, Configure, Start and Verify Sample
Application</h2>
<ul>
<li>
-<p><a href="#load">Load Sample Application Package</a></p>
+<p><a href="#load">Install Sample Application Package</a></p>
</li>
<li>
<p><a href="#create">Create Application Specifications</a></p>
@@ -368,18 +361,23 @@ hdfs dfs -copyFromLocal <span class="cp"
<p><a href="#manage">Manage the Application Lifecycle</a></p>
</li>
</ul>
-<h3 id="wzxhzdk36wzxhzdk37load-sample-application-package"><a
name="load"></a>Load Sample Application Package</h3>
-<div class="codehilite"><pre><span class="n">hdfs</span> <span
class="n">dfs</span> <span class="o">-</span><span
class="n">copyFromLocal</span> <span class="o">*</span><span
class="n">sample</span><span class="o">-</span><span
class="n">application</span><span class="o">-</span><span
class="n">package</span><span class="o">*</span> <span class="o">/</span><span
class="n">user</span><span class="o">/</span><span class="n">yarn</span>
+<h3 id="wzxhzdk36wzxhzdk37install-sample-application-package"><a
name="load"></a>Install Sample Application Package</h3>
+<div class="codehilite"><pre><span class="n">slider</span> <span
class="n">install</span><span class="o">-</span><span class="n">package</span>
<span class="o">--</span><span class="n">name</span> <span
class="o">*</span><span class="n">package</span> <span
class="n">name</span><span class="o">*</span> <span class="o">--</span><span
class="n">package</span> <span class="o">*</span><span
class="n">sample</span><span class="o">-</span><span
class="n">application</span><span class="o">-</span><span
class="n">package</span><span class="o">*</span>
+</pre></div>
+
+
+<p>For example, you can use the following command for HBase application
package.</p>
+<div class="codehilite"><pre><span class="n">slider</span> <span
class="n">install</span><span class="o">-</span><span class="n">package</span>
<span class="o">--</span><span class="n">name</span> <span
class="n">HBASE</span> <span class="o">--</span><span class="n">package</span>
<span class="n">path</span><span class="o">-</span><span
class="n">to</span><span class="o">-</span><span class="n">hbase</span><span
class="o">-</span><span class="n">package</span><span class="p">.</span><span
class="n">zip</span>
</pre></div>
<h3 id="wzxhzdk38wzxhzdk39create-application-specifications"><a
name="create"></a>Create Application Specifications</h3>
<p>Configuring a Slider application consists of two parts: the <a
href="#resspec">Resource Specification</a>,
and the *<a href="#appconfig">Application Configuration</a>.</p>
-<p><strong>Note</strong>: There are sample Resource Specifications
(<strong>resources.json</strong>) and Application Configuration
-(<strong>appConfig.json</strong>) files in the application packages zip files
(e.g. /<strong>resources.json</strong> and
/<strong>appConfig.json</strong>).</p>
+<p><strong>Note</strong>: There are sample Resource Specifications
(<strong>resources-default.json</strong>) and Application Configuration
+(<strong>appConfig-default.json</strong>) files in the application packages
zip files.</p>
<h4 id="wzxhzdk40wzxhzdk41resource-specification"><a
name="resspec"></a>Resource Specification</h4>
-<p>Slider needs to know what components (and how many components) are in an
application package to deploy.</p>
+<p>Slider needs to know what components (and how many components) to
deploy.</p>
<p>As Slider creates each instance of a component in its own YARN container,
it also needs to know what to ask YARN for in terms of <strong>memory</strong>
and <strong>CPU</strong> for those containers. </p>
<p>All this information goes into the <strong>Resources Specification</strong>
file ("Resource Spec") named <code>resources.json</code>. The Resource Spec
tells Slider how many instances of each component in the application (such as
an HBase RegionServer) to deploy and the parameters for YARN.</p>
<p>An application package should contain the default resources.json and you
can start from there. Or you can create one based on <a
href="slider_specs/resource_specification.html">Resource Specification</a>.</p>
@@ -394,12 +392,12 @@ hdfs dfs -copyFromLocal <span class="cp"
<h3 id="wzxhzdk44wzxhzdk45start-the-application"><a name="start"></a>Start the
Application</h3>
<p>Once the steps above are completed, the application can be started through
the <strong>Slider Command Line Interface (CLI)</strong>.</p>
<p>Change directory to the "bin" directory under the slider installation</p>
-<div class="codehilite"><pre>cd <span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span class="cp">}</span>/slider-0.40.0/bin
+<div class="codehilite"><pre>cd <span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span class="cp">}</span>/slider-0.50.0/bin
</pre></div>
<p>Execute the following command:</p>
-<div class="codehilite"><pre><span class="o">./</span><span
class="n">slider</span> <span class="n">create</span> <span
class="n">cl1</span> <span class="o">--</span><span class="n">image</span>
<span class="n">hdfs</span><span class="p">:</span><span
class="o">//</span><span class="n">yourNameNodeHost</span><span
class="p">:</span>8020<span class="o">/</span><span class="n">user</span><span
class="o">/</span><span class="n">yarn</span><span class="o">/</span><span
class="n">agent</span><span class="o">/</span><span
class="n">slider</span><span class="o">-</span><span
class="n">agent</span><span class="p">.</span><span class="n">tar</span><span
class="p">.</span><span class="n">gz</span> <span class="o">--</span><span
class="n">template</span> <span class="n">appConfig</span><span
class="p">.</span><span class="n">json</span> <span class="o">--</span><span
class="n">resources</span> <span class="n">resources</span><span
class="p">.</span><span class="n">json</span>
+<div class="codehilite"><pre><span class="o">./</span><span
class="n">slider</span> <span class="n">create</span> <span
class="n">cl1</span> <span class="o">--</span><span class="n">template</span>
<span class="n">appConfig</span><span class="p">.</span><span
class="n">json</span> <span class="o">--</span><span class="n">resources</span>
<span class="n">resources</span><span class="p">.</span><span
class="n">json</span>
</pre></div>
@@ -410,6 +408,7 @@ hdfs dfs -copyFromLocal <span class="cp"
<p><img alt="image alt text" src="../images/image_0_app_master_ui_lrg.png"
/></p>
<h3 id="wzxhzdk48wzxhzdk49obtain-client-config"><a
name="client-config"></a>Obtain client config</h3>
<p>An application publishes several useful details that can be used to manage
and communicate with the application instance.</p>
+<p>You can use <code>registry</code> commands to get all published data.</p>
<p><strong>All published data</strong></p>
<p>All published data are available at the
<code>/ws/v1/slider/publisher</code> end point of the AppMaster (e.g.
http://c6403.ambari.apache.org:34701/ws/v1/slider/publisher). Its obtainable
through <code>slider-client status <app name></code> where field
<code>info.am.web.url</code> specifies the base address. The URL is also
advertised in the AppMaster tracking UI.</p>
<p><strong>Client configuration</strong>
@@ -439,15 +438,13 @@ Client configurations are at <code>/ws/v
<p>An application may publish some quick links for the monitoring UI, JMX
endpoint, etc. as the listening ports are dynamically allocated and thus need
to be published. These information is available at
<code>/ws/v1/slider/publisher/slider/quicklinks</code>.</p>
<h3 id="wzxhzdk50wzxhzdk51manage-the-application-lifecycle"><a
name="manage"></a>Manage the Application Lifecycle</h3>
<p>Once started, applications can be frozen/stopped, thawed/restarted, and
destroyed/removed as follows:</p>
-<h4 id="frozen">Frozen:</h4>
-<p>(<em>on develop branch freeze has been renamed to stop</em>)</p>
-<div class="codehilite"><pre><span class="o">./</span><span
class="n">slider</span> <span class="n">freeze</span> <span class="n">cl1</span>
+<h4 id="stopped">Stopped:</h4>
+<div class="codehilite"><pre><span class="o">./</span><span
class="n">slider</span> <span class="n">stop</span> <span class="n">cl1</span>
</pre></div>
-<h4 id="thawed">Thawed:</h4>
-<p>(<em>on develop branch thaw has been renamed to start</em>)</p>
-<div class="codehilite"><pre><span class="o">./</span><span
class="n">slider</span> <span class="n">thaw</span> <span class="n">cl1</span>
+<h4 id="started">Started:</h4>
+<div class="codehilite"><pre><span class="o">./</span><span
class="n">slider</span> <span class="n">start</span> <span class="n">cl1</span>
</pre></div>
@@ -464,7 +461,7 @@ Client configurations are at <code>/ws/v
<h1 id="wzxhzdk52wzxhzdk53appendix-a-debugging-slider-agent"><a
name="appendixa"></a>Appendix A: Debugging Slider-Agent</h1>
<h2 id="create-and-deploy-slider-agent-configuration">Create and deploy Slider
Agent configuration</h2>
<p>Create an agent config file (agent.ini) based on the sample available
at:</p>
-<div class="codehilite"><pre><span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span
class="cp">}</span>/slider-0.40.0/agent/conf/agent.ini
+<div class="codehilite"><pre><span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span
class="cp">}</span>/slider-0.50.0/agent/conf/agent.ini
</pre></div>
Added: websites/staging/slider/trunk/content/docs/getting_started_40.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/getting_started_40.html (added)
+++ websites/staging/slider/trunk/content/docs/getting_started_40.html Thu Oct
9 01:03:09 2014
@@ -0,0 +1,512 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href="/css/bootstrap.min.css" rel="stylesheet">
+ <link href="/css/bootstrap-theme.min.css" rel="stylesheet">
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script
src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js";></script>
+ <script
src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js";></script>
+ <![endif]-->
+ <script
src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js";></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <link href="/css/slider.css" rel="stylesheet" type="text/css">
+ <link
href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css"
rel="stylesheet">
+ <title></title>
+
+</head>
+
+<body>
+<nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+<div class="container-fluid">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle" data-toggle="collapse"
data-target="#navbar-items">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/index.html">Slider</a>
+ </div>
+ <div class="collapse navbar-collapse" id="navbar-items">
+ <ul class="nav navbar-nav">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Project <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_index"><a href="/index.html">Home</a></li>
+<li id="nav_downloads"><a href="/downloads">Downloads</a></li>
+<li id="nav_source"><a
href="https://git-wip-us.apache.org/repos/asf/incubator-slider.git";><i
class="fa fa-external-link"></i> Source</a></li>
+<li><a href="http://www.apache.org/licenses/LICENSE-2.0";><i class="fa
fa-external-link"></i> License</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Community</li>
+<li id="nav_team"><a href="/team.html">Team</a></li>
+<li id="nav_mailinglists"><a href="/mailing_lists.html">Mailing Lists</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Tools</li>
+<li id="nav_jira"><a href="https://issues.apache.org/jira/browse/SLIDER";><i
class="fa fa-external-link"></i> Issues</a></li>
+<li id="nav_builds"><a href="https://builds.apache.org/job/Slider-develop";><i
class="fa fa-external-link"></i> Builds</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/design">
+ Design <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_design_overview"><a
href="/design/architecture.html">Overview</a></li>
+<li id="nav_app_needs"><a
href="/docs/slider_specs/application_needs.html">Application Needs</a></li>
+<li id="nav_spec"><a href="/design/specification/index.html">Expected
Behavior</a></li>
+<li id="nav_registry"><a href="/design/registry/index.html">Service
Registry</a></li>
+<li id="nav_ssl_impl"><a href="/design/ssl_implementation.html">SSL
Implementation</a></li>
+<li id="nav_role_history"><a href="/design/rolehistory.html">Role
History</a></li>
+</ul>
+</li>
+
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/developing">
+ Developing <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_dev_building"><a href="/developing/building.html">Building</a></li>
+<li id="nav_dev_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_dev_testing"><a href="/developing/testing.html">Testing</a></li>
+<li id="nav_dev_funtesting"><a
href="/developing/functional_tests.html">Functional Testing</a></li>
+<li id="nav_dev_manual_testing"><a
href="/developing/manual_testing.html">Manual Testing</a></li>
+<li id="nav_releasing"><a href="/developing/releasing.html">Releasing</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Using <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_use_getting_started"><a href="/docs/getting_started.html">Getting
Started</a></li>
+<li id="nav_use_manpage"><a href="/docs/manpage.html">Man Page</a></li>
+<li id="nav_use_examples"><a href="/docs/examples.html">Examples</a></li>
+<li id="nav_use_client_config"><a
href="/docs/client-configuration.html">Client Configuration</a></li>
+<li id="nav_use_exitcodes"><a href="/docs/exitcodes.html">Client Exit
Codes</a></li>
+<li id="nav_use_security"><a href="/docs/security.html">Security</a></li>
+<li id="nav_am_agent_ssl"><a href="/docs/ssl.html">Agent to AM SSL</a></li>
+<li id="nav_use_app_package"><a href="/docs/slider_specs/index.html">Creating
App Packages</a></li>
+<li id="nav_use_app_configuration"><a
href="/docs/configuration/index.html">App Configuration Model</a></li>
+<li id="nav_use_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_use_troubleshooting"><a
href="/docs/troubleshooting.html">Troubleshooting</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ ASF Links <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li><a href="http://www.apache.org";><i class="fa fa-external-link"></i> Apache
Software Foundation</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html";><i class="fa
fa-external-link"></i> Sponsorship</a></li>
+<li><a href="http://www.apache.org/security/";><i class="fa
fa-external-link"></i> Security</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html";><i class="fa
fa-external-link"></i> Thanks</a></li>
+</ul>
+</li>
+</ul>
+
+ </div>
+
+</div>
+</nav>
+
+
+<div class="container-fluid">
+<div class="row">
+ <div class="col-md-2" id="sidebar">
+ <div style="text-align: center">
+ <h1><a href="/index.html">Apache Slider (incubating)</a></h1>
+ <hr>
+Latest release: <strong>0.40-incubating</strong><br>
+ <br>
+ <a id="download-button-sidebar" class="btn btn-success btn-block"
href="/downloads/" role="button">Download</a>
+ </div>
+ <hr>
+ <a href="http://incubator.apache.org";><img id="logo" alt="Apache Incubator
™" class="img-responsive"
src="http://incubator.apache.org/images/apache-incubator-logo.png"/></a>
+ </div>
+ <div class="col-md-8 col-md-offset-1">
+
+ <div id="content">
+
+ <h1 class="title"></h1>
+
+ <!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<h1 id="apache-slider-getting-started">Apache Slider: Getting Started</h1>
+<p>This page is updated to reflect the latest code in "develop".</p>
+<p>For the version applicable to the last release go to <a
href="#slider40">Slider-40</a>.</p>
+<h2 id="introduction">Introduction</h2>
+<p>The following provides the steps required for setting up a cluster and
deploying a YARN hosted application using Slider.</p>
+<ul>
+<li>
+<p><a href="#sysreqs">System Requirements</a></p>
+</li>
+<li>
+<p><a href="#setup">Setup the Cluster</a></p>
+</li>
+<li>
+<p><a href="#download">Download Slider Packages</a></p>
+</li>
+<li>
+<p><a href="#build">Build Slider</a></p>
+</li>
+<li>
+<p><a href="#install">Install Slider</a></p>
+</li>
+<li>
+<p><a href="#deploy">Deploy Slider Resources</a></p>
+</li>
+<li>
+<p><a href="#downsample">Create Application Packages</a></p>
+</li>
+<li>
+<p><a href="#installapp">Install, Configure, Start and Verify Sample
Application</a></p>
+</li>
+</ul>
+<h2 id="wzxhzdk20wzxhzdk21system-requirements"><a name="sysreqs"></a>System
Requirements</h2>
+<p>The Slider deployment has the following minimum system requirements:</p>
+<ul>
+<li>
+<p>Hadoop 2.4+</p>
+</li>
+<li>
+<p>Required Services: HDFS, YARN and ZooKeeper</p>
+</li>
+<li>
+<p>Oracle JDK 1.6 (64-bit)</p>
+</li>
+<li>
+<p>Python 2.6</p>
+</li>
+<li>
+<p>openssl</p>
+</li>
+</ul>
+<h2 id="wzxhzdk22wzxhzdk23setup-the-cluster"><a name="setup"></a>Setup the
Cluster</h2>
+<p>After setting up your Hadoop cluster (using Ambari or other means) with the
+services listed above, modify your YARN configuration to allow for multiple
+containers on a single host. In <code>yarn-site.xml</code> make the following
modifications:</p>
+<table>
+ <tr>
+ <td>Property</td>
+ <td>Value</td>
+ </tr>
+ <tr>
+ <td>yarn.scheduler.minimum-allocation-mb</td>
+ <td>>= 256 (ensure that YARN can allocate sufficient number of
containers)</td>
+ </tr>
+ <tr>
+ <td>yarn.nodemanager.delete.debug-delay-sec</td>
+ <td>>= 3600 (to retain for an hour)</td>
+ </tr>
+</table>
+
+<p>There are other options detailed in the Troubleshooting file available <a
href="/docs/troubleshooting.html">here</a>.</p>
+<h2 id="wzxhzdk24wzxhzdk25download-slider-packages"><a
name="download"></a>Download Slider Packages</h2>
+<p>Slider releases are available at
+<a
href="https://www.apache.org/dyn/closer.cgi/incubator/slider";>https://www.apache.org/dyn/closer.cgi/incubator/slider</a>.</p>
+<p><em>Or, you can build it as described below.</em></p>
+<h2 id="wzxhzdk26wzxhzdk27build-slider"><a name="build"></a>Build Slider</h2>
+<ul>
+<li>From the top level directory, execute <code>mvn clean site:site site:stage
package -DskipTests</code></li>
+<li>Use the generated compressed tar file in slider-assembly/target directory
(e.g. slider-0.40.0-all.tar.gz or slider-0.40.0-all.zip) for the subsequent
steps</li>
+<li>If you are cloning the Slider git repo, go to <code>master</code> branch
for the latest released or <code>develop</code> for the latest under
development</li>
+</ul>
+<h2 id="wzxhzdk28wzxhzdk29install-slider"><a name="install"></a>Install
Slider</h2>
+<p>Slider is installed on a client machine that can access the hadoop cluster.
Follow the following steps to expand/install Slider:</p>
+<div class="codehilite"><pre>mkdir <span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span class="cp">}</span>
+
+cd <span class="cp">${</span><span class="n">slider</span><span
class="o">-</span><span class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span class="cp">}</span>
+</pre></div>
+
+
+<p>Login as the "yarn" user (assuming this is a host associated with the
installed cluster). E.g., <code>su yarn</code>
+<em>This assumes that all apps are being run as âyarnâ user. Any other
user can be used to run the apps - ensure that file permission is granted as
required.</em></p>
+<p>Expand the tar file: <code>tar -xvf
slider-0.40.0-incubating-all.tar.gz</code> or <code>unzip
slider-0.40.0-incubating-all.zip</code></p>
+<p>Browse to the Slider directory: <code>cd slider-0.40.0/bin</code></p>
+<p><em>Add java to the PATH.</em></p>
+<div class="codehilite"><pre> <span class="n">export</span> <span
class="n">PATH</span><span class="p">=</span>$<span class="n">PATH</span><span
class="p">:</span><span class="o">/</span><span class="n">usr</span><span
class="o">/</span><span class="n">jdk64</span><span class="o">/</span><span
class="n">jdk1</span><span class="p">.</span>7<span class="p">.</span>0<span
class="n">_45</span><span class="o">/</span><span class="n">bin</span>
+</pre></div>
+
+
+<p>(or the path to the JDK bin directory)</p>
+<p>Modify Slider configuration file
<code>${slider-install-dir}/slider-0.40.0/conf/slider-client.xml</code> to add
the following properties:</p>
+<div class="codehilite"><pre> <span class="nt"><property></span>
+ <span class="nt"><name></span>yarn.application.classpath<span
class="nt"></name></span>
+ <span
class="nt"><value></span>/etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*<span
class="nt"></value></span>
+ <span class="nt"></property></span>
+
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.zookeeper.quorum<span
class="nt"></name></span>
+ <span class="nt"><value></span>yourZooKeeperHost:port<span
class="nt"></value></span>
+ <span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>yarn.resourcemanager.address<span
class="nt"></name></span>
+ <span class="nt"><value></span>yourResourceManagerHost:8050<span
class="nt"></value></span>
+<span class="nt"></property></span>
+<span class="nt"><property></span>
+ <span
class="nt"><name></span>yarn.resourcemanager.scheduler.address<span
class="nt"></name></span>
+ <span class="nt"><value></span>yourResourceManagerHost:8030<span
class="nt"></value></span>
+<span class="nt"></property></span>
+<span class="nt"><property></span>
+ <span class="nt"><name></span>fs.defaultFS<span
class="nt"></name></span>
+ <span class="nt"><value></span>hdfs://yourNameNodeHost:8020<span
class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
+
+
+<p>If the target cluster is on Windows then make sure that the
<code>yarn.application.classpath</code> is set accordingly.</p>
+<div class="codehilite"><pre> <span class="nt"><property></span>
+ <span class="nt"><name></span>yarn.application.classpath<span
class="nt"></name></span>
+ <span class="nt"><value></span>
+
%HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,...
+ <span class="nt"></value></span>
+ <span class="nt"></property></span>
+</pre></div>
+
+
+<p>Execute:</p>
+<div class="codehilite"><pre><span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span class="cp">}</span>/slider-0.40.0/bin/slider version
+OR
+python %slider-install-dir%/slider-0.40.0/bin/slider.py version
+</pre></div>
+
+
+<p>Ensure there are no errors and you can see "Compiled against Hadoop 2.4.0".
<strong>This ensures that Slider is installed correctly.</strong></p>
+<h2 id="wzxhzdk30wzxhzdk31deploy-slider-resources"><a name="deploy"></a>Deploy
Slider Resources</h2>
+<p>Ensure that all file folders are accessible to the user creating the
application instance. The example assumes "yarn" to be that user. The default
yarn user may be different e.g. hadoop.</p>
+<h3 id="ensure-hdfs-home-folder-for-the-user">Ensure HDFS home folder for the
User</h3>
+<p>Perform the following steps to create the user home folder with the
appropriate permissions:</p>
+<div class="codehilite"><pre><span class="n">su</span> <span
class="n">hdfs</span>
+
+<span class="n">hdfs</span> <span class="n">dfs</span> <span
class="o">-</span><span class="n">mkdir</span> <span class="o">/</span><span
class="n">user</span><span class="o">/</span><span class="n">yarn</span>
+
+<span class="n">hdfs</span> <span class="n">dfs</span> <span
class="o">-</span><span class="n">chown</span> <span class="n">yarn</span><span
class="p">:</span><span class="n">hdfs</span> <span class="o">/</span><span
class="n">user</span><span class="o">/</span><span class="n">yarn</span>
+</pre></div>
+
+
+<h3 id="load-slider-agent">Load Slider Agent</h3>
+<div class="codehilite"><pre>su yarn
+
+hdfs dfs -mkdir /user/yarn/agent
+
+hdfs dfs -copyFromLocal <span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span
class="cp">}</span>/slider-0.40.0/agent/slider-agent.tar.gz /user/yarn/agent
+</pre></div>
+
+
+<h2 id="wzxhzdk32wzxhzdk33create-application-packages"><a
name="downsample"></a>Create Application Packages</h2>
+<p>Slider "master" branch has several sample application packages available
for use with Slider:</p>
+<ul>
+<li>
+<p>app-packages/memcached-win or memcached: The README.txt describes how to
create the Slider Application package for memcached for linux or windows.
<strong>memcached</strong> app packages are a good place to <a
href="slider_specs/hello_world_slider_app.html">start</a></p>
+</li>
+<li>
+<p>app-packages/hbase or hbase-win: The README.txt file describes how to
create a Slider Application Package for HBase. By default it will create a
package for HBase 0.98.3 but you can create the same for other versions.</p>
+</li>
+<li>
+<p>app-packages/accumulo: The README.txt file describes how to create a Slider
Application Package for Accumulo.</p>
+</li>
+<li>
+<p>app-packages/storm: The README.txt describes how to create the Slider
Application package for Storm.</p>
+</li>
+</ul>
+<p>Create one or more Slider application packages and follow the steps below
to install them.</p>
+<h2
id="wzxhzdk34wzxhzdk35install-configure-start-and-verify-sample-application"><a
name="installapp"></a>Install, Configure, Start and Verify Sample
Application</h2>
+<ul>
+<li>
+<p><a href="#load">Load Sample Application Package</a></p>
+</li>
+<li>
+<p><a href="#create">Create Application Specifications</a></p>
+</li>
+<li>
+<p><a href="#start">Start the Application</a></p>
+</li>
+<li>
+<p><a href="#verify">Verify the Application</a></p>
+</li>
+<li>
+<p><a href="#manage">Manage the Application Lifecycle</a></p>
+</li>
+</ul>
+<h3 id="wzxhzdk36wzxhzdk37load-sample-application-package"><a
name="load"></a>Load Sample Application Package</h3>
+<div class="codehilite"><pre><span class="n">hdfs</span> <span
class="n">dfs</span> <span class="o">-</span><span
class="n">copyFromLocal</span> <span class="o">*</span><span
class="n">sample</span><span class="o">-</span><span
class="n">application</span><span class="o">-</span><span
class="n">package</span><span class="o">*</span> <span class="o">/</span><span
class="n">user</span><span class="o">/</span><span class="n">yarn</span>
+</pre></div>
+
+
+<h3 id="wzxhzdk38wzxhzdk39create-application-specifications"><a
name="create"></a>Create Application Specifications</h3>
+<p>Configuring a Slider application consists of two parts: the <a
href="#resspec">Resource Specification</a>,
+ and the *<a href="#appconfig">Application Configuration</a>.</p>
+<p><strong>Note</strong>: There are sample Resource Specifications
(<strong>resources.json</strong>) and Application Configuration
+(<strong>appConfig.json</strong>) files in the application packages zip files
(e.g. /<strong>resources.json</strong> and
/<strong>appConfig.json</strong>).</p>
+<h4 id="wzxhzdk40wzxhzdk41resource-specification"><a
name="resspec"></a>Resource Specification</h4>
+<p>Slider needs to know what components (and how many components) are in an
application package to deploy.</p>
+<p>As Slider creates each instance of a component in its own YARN container,
it also needs to know what to ask YARN for in terms of <strong>memory</strong>
and <strong>CPU</strong> for those containers. </p>
+<p>All this information goes into the <strong>Resources Specification</strong>
file ("Resource Spec") named <code>resources.json</code>. The Resource Spec
tells Slider how many instances of each component in the application (such as
an HBase RegionServer) to deploy and the parameters for YARN.</p>
+<p>An application package should contain the default resources.json and you
can start from there. Or you can create one based on <a
href="slider_specs/resource_specification.html">Resource Specification</a>.</p>
+<p>Store the Resource Spec file on your local disk (e.g.
<code>/tmp/resources.json</code>).</p>
+<h4 id="wzxhzdk42wzxhzdk43application-configuration"><a
name="appconfig"></a>Application Configuration</h4>
+<p>Alongside the Resource Spec there is the <strong>Application
Configuration</strong> file ("App Config") which includes parameters that are
specific to the application, rather than YARN. The App Config is a file that
contains all application configuration. This configuration is applied to the
default configuration provided by the application definition and then handed
off to the associated component agent.</p>
+<p>For example, the heap sizes of the JVMs, The App Config defines the
configuration details <strong>specific to the application and
component</strong> instances. An application package should contain the default
resources.json and you can start from there.</p>
+<p>Ensure the following variables are accurate:</p>
+<ul>
+<li>"application.def": "application_package.zip" (path to uploaded application
package)</li>
+</ul>
+<h3 id="wzxhzdk44wzxhzdk45start-the-application"><a name="start"></a>Start the
Application</h3>
+<p>Once the steps above are completed, the application can be started through
the <strong>Slider Command Line Interface (CLI)</strong>.</p>
+<p>Change directory to the "bin" directory under the slider installation</p>
+<div class="codehilite"><pre>cd <span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span class="cp">}</span>/slider-0.40.0/bin
+</pre></div>
+
+
+<p>Execute the following command:</p>
+<div class="codehilite"><pre><span class="o">./</span><span
class="n">slider</span> <span class="n">create</span> <span
class="n">cl1</span> <span class="o">--</span><span class="n">image</span>
<span class="n">hdfs</span><span class="p">:</span><span
class="o">//</span><span class="n">yourNameNodeHost</span><span
class="p">:</span>8020<span class="o">/</span><span class="n">user</span><span
class="o">/</span><span class="n">yarn</span><span class="o">/</span><span
class="n">agent</span><span class="o">/</span><span
class="n">slider</span><span class="o">-</span><span
class="n">agent</span><span class="p">.</span><span class="n">tar</span><span
class="p">.</span><span class="n">gz</span> <span class="o">--</span><span
class="n">template</span> <span class="n">appConfig</span><span
class="p">.</span><span class="n">json</span> <span class="o">--</span><span
class="n">resources</span> <span class="n">resources</span><span
class="p">.</span><span class="n">json</span>
+</pre></div>
+
+
+<h3 id="wzxhzdk46wzxhzdk47verify-the-application"><a name="verify"></a>Verify
the Application</h3>
+<p>The successful launch of the application can be verified via the YARN
Resource Manager Web UI. In most instances, this UI is accessible via a web
browser at port 8088 of the Resource Manager Host:</p>
+<p><img alt="image alt text" src="../images/image_1_apps_rm_ui-lrg.png" /></p>
+<p>The specific information for the running application is accessible via the
"ApplicationMaster" link that can be seen in the far right column of the row
associated with the running application (probably the top row):</p>
+<p><img alt="image alt text" src="../images/image_0_app_master_ui_lrg.png"
/></p>
+<h3 id="wzxhzdk48wzxhzdk49obtain-client-config"><a
name="client-config"></a>Obtain client config</h3>
+<p>An application publishes several useful details that can be used to manage
and communicate with the application instance.</p>
+<p><strong>All published data</strong></p>
+<p>All published data are available at the
<code>/ws/v1/slider/publisher</code> end point of the AppMaster (e.g.
http://c6403.ambari.apache.org:34701/ws/v1/slider/publisher). Its obtainable
through <code>slider-client status <app name></code> where field
<code>info.am.web.url</code> specifies the base address. The URL is also
advertised in the AppMaster tracking UI.</p>
+<p><strong>Client configuration</strong>
+Client configurations are at <code>/ws/v1/slider/publisher/slider/<config
name></code> where <em>config name</em> can be </p>
+<ul>
+<li>site config file name without extension, such as <code>hbase-site</code>
(e.g.
http://c6403.ambari.apache.org:34701/ws/v1/slider/publisher/slider/hbase-site)
where the output is json formatted name-value pairs</li>
+<li>or, with extension such as <code>hbase-site.xml</code>
(http://c6403.ambari.apache.org:34701/ws/v1/slider/publisher/slider/hbase-site.xml)
to get an XML formatted output that can be consumed by the clients</li>
+</ul>
+<p><strong>Log locations</strong></p>
+<p>The log locations for various containers in the application instance are at
<code>ws/v1/slider/publisher/slider/logfolders</code></p>
+<div class="codehilite"><pre><span class="p">{</span>
+
+ "<span class="n">description</span>"<span class="p">:</span>
"<span class="n">LogFolders</span>"<span class="p">,</span>
+ "<span class="n">entries</span>"<span class="p">:</span> <span
class="p">{</span>
+ "<span class="n">c6403</span><span class="p">.</span><span
class="n">ambari</span><span class="p">.</span><span
class="n">apache</span><span class="p">.</span><span class="n">org</span><span
class="o">-</span><span
class="n">container_1404783501621_0002_01_000002</span><span
class="o">-</span><span class="n">AGENT_LOG_ROOT</span>"<span
class="p">:</span> "<span class="o">/</span><span
class="n">hadoop</span><span class="o">/</span><span class="n">yarn</span><span
class="o">/</span><span class="nb">log</span><span class="o">/</span><span
class="n">application_1404783501621_0002</span><span class="o">/</span><span
class="n">container_1404783501621_0002_01_000002</span>"<span
class="p">,</span>
+ "<span class="n">c6403</span><span class="p">.</span><span
class="n">ambari</span><span class="p">.</span><span
class="n">apache</span><span class="p">.</span><span class="n">org</span><span
class="o">-</span><span
class="n">container_1404783501621_0002_01_000003</span><span
class="o">-</span><span class="n">AGENT_LOG_ROOT</span>"<span
class="p">:</span> "<span class="o">/</span><span
class="n">hadoop</span><span class="o">/</span><span class="n">yarn</span><span
class="o">/</span><span class="nb">log</span><span class="o">/</span><span
class="n">application_1404783501621_0002</span><span class="o">/</span><span
class="n">container_1404783501621_0002_01_000003</span>"<span
class="p">,</span>
+ "<span class="n">c6403</span><span class="p">.</span><span
class="n">ambari</span><span class="p">.</span><span
class="n">apache</span><span class="p">.</span><span class="n">org</span><span
class="o">-</span><span
class="n">container_1404783501621_0002_01_000003</span><span
class="o">-</span><span class="n">AGENT_WORK_ROOT</span>"<span
class="p">:</span> "<span class="o">/</span><span
class="n">hadoop</span><span class="o">/</span><span class="n">yarn</span><span
class="o">/</span><span class="n">local</span><span class="o">/</span><span
class="n">usercache</span><span class="o">/</span><span
class="n">yarn</span><span class="o">/</span><span
class="n">appcache</span><span class="o">/</span><span
class="n">application_1404783501621_0002</span><span class="o">/</span><span
class="n">container_1404783501621_0002_01_000003</span>"<span
class="p">,</span>
+ "<span class="n">c6403</span><span class="p">.</span><span
class="n">ambari</span><span class="p">.</span><span
class="n">apache</span><span class="p">.</span><span class="n">org</span><span
class="o">-</span><span
class="n">container_1404783501621_0002_01_000002</span><span
class="o">-</span><span class="n">AGENT_WORK_ROOT</span>"<span
class="p">:</span> "<span class="o">/</span><span
class="n">hadoop</span><span class="o">/</span><span class="n">yarn</span><span
class="o">/</span><span class="n">local</span><span class="o">/</span><span
class="n">usercache</span><span class="o">/</span><span
class="n">yarn</span><span class="o">/</span><span
class="n">appcache</span><span class="o">/</span><span
class="n">application_1404783501621_0002</span><span class="o">/</span><span
class="n">container_1404783501621_0002_01_000002</span>"
+ <span class="p">},</span>
+ "<span class="n">updated</span>"<span class="p">:</span> 0<span
class="p">,</span>
+ "<span class="n">empty</span>"<span class="p">:</span> <span
class="n">false</span>
+<span class="p">}</span>
+</pre></div>
+
+
+<p><strong>Quick links</strong></p>
+<p>An application may publish some quick links for the monitoring UI, JMX
endpoint, etc. as the listening ports are dynamically allocated and thus need
to be published. These information is available at
<code>/ws/v1/slider/publisher/slider/quicklinks</code>.</p>
+<h3 id="wzxhzdk50wzxhzdk51manage-the-application-lifecycle"><a
name="manage"></a>Manage the Application Lifecycle</h3>
+<p>Once started, applications can be frozen/stopped, thawed/restarted, and
destroyed/removed as follows:</p>
+<h4 id="frozen">Frozen:</h4>
+<p>(<em>on develop branch freeze has been renamed to stop</em>)</p>
+<div class="codehilite"><pre><span class="o">./</span><span
class="n">slider</span> <span class="n">freeze</span> <span class="n">cl1</span>
+</pre></div>
+
+
+<h4 id="thawed">Thawed:</h4>
+<p>(<em>on develop branch thaw has been renamed to start</em>)</p>
+<div class="codehilite"><pre><span class="o">./</span><span
class="n">slider</span> <span class="n">thaw</span> <span class="n">cl1</span>
+</pre></div>
+
+
+<h4 id="destroyed">Destroyed:</h4>
+<div class="codehilite"><pre><span class="o">./</span><span
class="n">slider</span> <span class="n">destroy</span>
+</pre></div>
+
+
+<h4 id="flexed">Flexed:</h4>
+<div class="codehilite"><pre><span class="o">./</span><span
class="n">slider</span> <span class="n">flex</span> <span class="n">cl1</span>
<span class="o">--</span><span class="n">component</span> <span
class="n">worker</span> 5
+</pre></div>
+
+
+<h1 id="wzxhzdk52wzxhzdk53appendix-a-debugging-slider-agent"><a
name="appendixa"></a>Appendix A: Debugging Slider-Agent</h1>
+<h2 id="create-and-deploy-slider-agent-configuration">Create and deploy Slider
Agent configuration</h2>
+<p>Create an agent config file (agent.ini) based on the sample available
at:</p>
+<div class="codehilite"><pre><span class="cp">${</span><span
class="n">slider</span><span class="o">-</span><span
class="n">install</span><span class="o">-</span><span
class="nb">dir</span><span
class="cp">}</span>/slider-0.40.0/agent/conf/agent.ini
+</pre></div>
+
+
+<p>The sample agent.ini file can be used as is (see below). Some of the
parameters of interest are:</p>
+<p><code>log_level</code> = INFO or DEBUG, to control the verbosity of log</p>
+<p><code>app_log_dir</code> = the relative location of the application log
file</p>
+<p><code>log_dir</code> = the relative location of the agent and command log
file</p>
+<p>Once created, deploy the agent.ini file to HDFS (it can be deployed in any
location accessible to application instance):</p>
+<div class="codehilite"><pre><span class="n">su</span> <span
class="n">yarn</span>
+
+<span class="n">hdfs</span> <span class="n">dfs</span> <span
class="o">-</span><span class="n">copyFromLocal</span> <span
class="n">agent</span><span class="p">.</span><span class="n">ini</span> <span
class="o">/</span><span class="n">user</span><span class="o">/</span><span
class="n">yarn</span><span class="o">/</span><span class="n">agent</span><span
class="o">/</span><span class="n">conf</span>
+</pre></div>
+
+
+<p>Modify the --template json file (appConfig.json) and add the location of
the agent.ini file.</p>
+<div class="codehilite"><pre>"<span class="n">agent</span><span
class="p">.</span><span class="n">conf</span>"<span class="p">:</span>
"<span class="o">/</span><span class="n">user</span><span
class="o">/</span><span class="n">yarn</span><span class="o">/</span><span
class="n">agent</span><span class="o">/</span><span class="n">conf</span><span
class="o">/</span><span class="n">agent</span><span class="p">.</span><span
class="n">ini</span>"
+</pre></div>
+ </div>
+
+ <div id="footer">
+ <a alt="Apache Software Foundation" href="http://www.apache.org";>
+ <img id="asf-logo" alt="Apache Software Foundation"
src="/images/feather-small.gif" width="100">
+ </a>
+ <div class="copyright">
+ <p>
+ Copyright © 2011-2014 The Apache Software Foundation, Licensed
under
+ the <a href="http://www.apache.org/licenses/LICENSE-2.0";>Apache
License, Version 2.0</a>.
+ Apache Slider, Slider, Apache, the Apache feather logo, and the Apache
Incubator
+ project logo are trademarks of the <a
href="http://www.apache.org";>Apache Software Foundation</a>.<br />
+ Site created with <a href="http://getbootstrap.com/";>Bootstrap</a>
including icons from <a href="http://glyphicons.com/";>GLYPHICONS</a> and <a
href="http://fontawesome.io/";>Font Awesome</a>.
+ </p>
+ </div>
+ </div>
+ </div>
+
+</div>
+</div>
+
+<script type="text/javascript">
+
+</script>
+</body>
+</html>
Modified: websites/staging/slider/trunk/content/docs/manpage.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/manpage.html (original)
+++ websites/staging/slider/trunk/content/docs/manpage.html Thu Oct 9 01:03:09
2014
@@ -266,7 +266,7 @@ If the application instance is live, the
<p>Build an instance of the given name, with the specific options.</p>
<p>It is not started; this can be done later with a <code>thaw</code>
command.</p>
<h3 id="create-name"><code>create <name></code></h3>
-<p>Build and run an applicaton instance of the given name </p>
+<p>Build and run an application instance of the given name </p>
<p>The <code>--wait</code> parameter, if provided, specifies the time to wait
until the YARN application is actually running. Even after the YARN application
has started, there may be some delay for the instance to start up.</p>
<h3 id="arguments-for-build-and-create">Arguments for <code>build</code> and
<code>create</code></h3>
<h5 id="-package-uri-to-package"><code>--package
<uri-to-package></code></h5>
@@ -386,7 +386,7 @@ is used</p>
<h3 id="destroy-name"><code>destroy <name></code></h3>
-<p>Destroy a (stopped) applicaton instance .</p>
+<p>Destroy a (stopped) application instance .</p>
<p>Important: This deletes all persistent data</p>
<p>Example</p>
<div class="codehilite"><pre><span class="n">slider</span> <span
class="n">destroy</span> <span class="n">instance1</span>
Modified:
websites/staging/slider/trunk/content/docs/slider_specs/resource_specification.html
==============================================================================
---
websites/staging/slider/trunk/content/docs/slider_specs/resource_specification.html
(original)
+++
websites/staging/slider/trunk/content/docs/slider_specs/resource_specification.html
Thu Oct 9 01:03:09 2014
@@ -182,7 +182,7 @@ Latest release: <strong>0.40-incubating<
<p>An example resource requirement for an application that has two components
"master" and "worker" is as follows. Slider will automatically add the
requirements for the AppMaster for the application. This component is named
"slider-appmaster".</p>
<p>Some parameters that can be specified for a component instance include:</p>
<ul>
-<li><code>yarn.memory</code>: amount of memory requried for the component
instance</li>
+<li><code>yarn.memory</code>: amount of memory required for the component
instance</li>
<li><code>yarn.vcores</code>: number of vcores requested</li>
<li><code>yarn.role.priority</code>: each component must be assigned unique
priority. Component with higher priority come up earlier than components with
lower priority</li>
<li><code>yarn.component.instances</code>: number of instances for this
component type</li>
@@ -324,8 +324,8 @@ YARN cluster and hardware.</p>
</em> Capacity Queues can be defined with access to one or more labels
<em> Ensure application components are associated with appropriate label
expressions
</em> Create the application using specific queue</p>
-<p>This way, you can gurantee that a certain set of nodes are reserved for an
application or for a component within an application.</p>
-<p>Label expression is specified through property "yarn.label.expression".
When no label expression is specified then it is assummed that only non-labeled
nodes are used when allocating containers for component instances.</p>
+<p>This way, you can guarantee that a certain set of nodes are reserved for an
application or for a component within an application.</p>
+<p>Label expression is specified through property "yarn.label.expression".
When no label expression is specified then it is assumed that only non-labeled
nodes are used when allocating containers for component instances.</p>
<p>If label expression is specified for slider-appmaster then it also becomes
the default label expression for all component. To take advantage of default
label expression leave out the property (see HBASE_REGIONSERVER in the
example). Label expression with empty string ("yarn.label.expression":"") means
nodes without labels. </p>
<h3 id="example_1">Example</h3>
<p>Here is a <code>resource.json</code> file for an HBase cluster which uses
labels. The label for the application instance is "hbase1" and the label
expression for the HBASE_MASTER components is "hbase1_master".
HBASE_REGIONSERVER instances will automatically use label "hbase1".
Alternatively, if you specify ("yarn.label.expression":"") for
HBASE_REGIONSERVER then the containers will only be allocated on nodes with no
labels.</p>
![http://incubator.apache.org/images/apache-incubator-logo.png"/](http://incubator.apache.org/images/apache-incubator-logo.png"/){kind=link}