Author: smohanty
Date: Tue Sep 2 14:26:15 2014
New Revision: 1622019
URL: http://svn.apache.org/r1622019
Log:
Changes to the getting started document
Removed:
incubator/slider/site/trunk/content/docs/slider_specs/why_slider.md
Modified:
incubator/slider/site/trunk/content/docs/getting_started.md
Modified: incubator/slider/site/trunk/content/docs/getting_started.md
URL:
http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/getting_started.md?rev=1622019&r1=1622018&r2=1622019&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/docs/getting_started.md (original)
+++ incubator/slider/site/trunk/content/docs/getting_started.md Tue Sep 2
14:26:15 2014
@@ -45,11 +45,13 @@ The Slider deployment has the following
* Hadoop 2.4+
-* Required Services: HDFS, YARN, MapReduce2 and ZooKeeper
+* Required Services: HDFS, YARN and ZooKeeper
-* Oracle JDK 1.7 (64-bit)
+* Oracle JDK 1.6 (64-bit)
-* Python 2.7
+* Python 2.6
+
+* openssl
## <a name="setup"></a>Setup the Cluster
@@ -86,22 +88,23 @@ Slider releases are available at
## <a name="build"></a>Build Slider
* From the top level directory, execute `mvn clean site:site site:stage
package -DskipTests`
-* Use the generated compressed tar file in slider-assembly/target directory
(e.g. slider-0.30.0-all.tar.gz) for the subsequent steps
+* Use the generated compressed tar file in slider-assembly/target directory
(e.g. slider-0.40.0-all.tar.gz or slider-0.40.0-all.zip) for the subsequent
steps
+* If you are cloning the Slider git repo, go to `master` branch for the latest
released or `develop` for the latest under development
## <a name="install"></a>Install Slider
Slider is installed on a client machine that can access the hadoop cluster.
Follow the following steps to expand/install Slider:
- mkdir ${slider-install-dir*;
+ mkdir ${slider-install-dir}
cd ${slider-install-dir}
Login as the "yarn" user (assuming this is a host associated with the
installed cluster). E.g., `su yarn`
*This assumes that all apps are being run as âyarnâ user. Any other user
can be used to run the apps - ensure that file permission is granted as
required.*
-Expand the tar file: `tar -xvf slider-0.30.0-all.tar.gz`
+Expand the tar file: `tar -xvf slider-0.40.0-incubating-all.tar.gz` or `unzip
slider-0.40.0-incubating-all.zip`
-Browse to the Slider directory: `cd slider-0.30.0/bin`
+Browse to the Slider directory: `cd slider-0.40.0/bin`
*Add java to the PATH.*
@@ -109,7 +112,7 @@ Browse to the Slider directory: `cd slid
(or the path to the JDK bin directory)
-Modify Slider configuration file
`${slider-install-dir}/slider-0.30.0/conf/slider-client.xml` to add the
following properties:
+Modify Slider configuration file
`${slider-install-dir}/slider-0.40.0/conf/slider-client.xml` to add the
following properties:
<property>
<name>yarn.application.classpath</name>
@@ -134,20 +137,30 @@ Modify Slider configuration file `${slid
<value>hdfs://yourNameNodeHost:8020</value>
</property>
+If the target cluster is on Windows then make sure that the
`yarn.application.classpath` is set accordingly.
+
+ <property>
+ <name>yarn.application.classpath</name>
+ <value>
+
%HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,...
+ </value>
+ </property>
Execute:
- ${slider-install-dir}/slider-0.30.0/bin/slider version
+ ${slider-install-dir}/slider-0.40.0/bin/slider version
+ OR
+ python %slider-install-dir%/slider-0.40.0/bin/slider.py version
Ensure there are no errors and you can see "Compiled against Hadoop 2.4.0".
**This ensures that Slider is installed correctly.**
## <a name="deploy"></a>Deploy Slider Resources
-Ensure that all file folders are accessible to the user creating the
application instance. The example assumes "yarn" to be that user. The HDFS
folder "/slider" is a suggestion and the default application packages and
associated configurations assume this path while accessing the slider-agent
tarball.
+Ensure that all file folders are accessible to the user creating the
application instance. The example assumes "yarn" to be that user. The default
yarn user may be different e.g. hadoop.
-### Create HDFS root folder for Slider
+### Ensure HDFS home folder for the User
-Perform the following steps to create the Slider root folder with the
appropriate permissions:
+Perform the following steps to create the user home folder with the
appropriate permissions:
su hdfs
@@ -161,16 +174,16 @@ Perform the following steps to create th
hdfs dfs -mkdir /user/yarn/agent
- hdfs dfs -copyFromLocal
${slider-install-dir}/slider-0.30.0/agent/slider-agent.tar.gz /user/yarn/agent
+ hdfs dfs -copyFromLocal
${slider-install-dir}/slider-0.40.0/agent/slider-agent.tar.gz /user/yarn/agent
## <a name="downsample"></a>Create Application Packages
-Few pre-created application packages are available at: TBD
+Slider "master" branch has several sample application packages available for
use with Slider:
-Slider "master" branch has three sample application packages available for use
with Slider:
+* app-packages/memcached-win or memcached: The README.txt describes how to
create the Slider Application package for memcached for linux or windows.
**memcached** app packages are a good place to
[start](slider_specs/hello_world_slider_app.html)
-* app-packages/hbase: The README.txt file describes how to create a Slider
Application Package for HBase. By default it will create a package for HBase
0.98.3 but you can create the same for other versions.
+* app-packages/hbase or hbase-win: The README.txt file describes how to create
a Slider Application Package for HBase. By default it will create a package for
HBase 0.98.3 but you can create the same for other versions.
* app-packages/accumulo: The README.txt file describes how to create a Slider
Application Package for Accumulo.
@@ -200,7 +213,7 @@ Configuring a Slider application consist
and the *[Application Configuration](#appconfig).
**Note**: There are sample Resource Specifications (**resources.json**) and
Application Configuration
-(**appConfig.json**) files in application packages (e.g. /**resources.json**
and /**appConfig.json**).
+(**appConfig.json**) files in the application packages zip files (e.g.
/**resources.json** and /**appConfig.json**).
#### <a name="resspec"></a>Resource Specification
@@ -221,6 +234,10 @@ Alongside the Resource Spec there is the
For example, the heap sizes of the JVMs, The App Config defines the
configuration details **specific to the application and component** instances.
An application package should contain the default resources.json and you can
start from there.
+Ensure the following variables are accurate:
+
+* "application.def": "application_package.zip" (path to uploaded application
package)
+
### <a name="start"></a>Start the Application
@@ -228,7 +245,7 @@ Once the steps above are completed, the
Change directory to the "bin" directory under the slider installation
- cd ${slider-install-dir}/slider-0.30.0/bin
+ cd ${slider-install-dir}/slider-0.40.0/bin
Execute the following command:
@@ -253,7 +270,7 @@ An application publishes several useful
All published data are available at the `/ws/v1/slider/publisher` end point of
the AppMaster (e.g.
http://c6403.ambari.apache.org:34701/ws/v1/slider/publisher). Its obtainable
through `slider-client status <app name>` where field `info.am.web.url`
specifies the base address. The URL is also advertised in the AppMaster
tracking UI.
**Client configuration**
-Client configurations are at `/ws/v1/slider/publisher/slider\<config name>`
where *config name* can be
+Client configurations are at `/ws/v1/slider/publisher/slider/<config name>`
where *config name* can be
* site config file name without extension, such as `hbase-site` (e.g.
http://c6403.ambari.apache.org:34701/ws/v1/slider/publisher/slider/hbase-site)
where the output is json formatted name-value pairs
* or, with extension such as `hbase-site.xml`
(http://c6403.ambari.apache.org:34701/ws/v1/slider/publisher/slider/hbase-site.xml)
to get an XML formatted output that can be consumed by the clients
@@ -284,10 +301,12 @@ An application may publish some quick li
Once started, applications can be frozen/stopped, thawed/restarted, and
destroyed/removed as follows:
#### Frozen:
+(*on develop branch freeze has been renamed to stop*)
./slider freeze cl1
#### Thawed:
+(*on develop branch thaw has been renamed to start*)
./slider thaw cl1
@@ -306,7 +325,7 @@ Once started, applications can be frozen
Create an agent config file (agent.ini) based on the sample available at:
- ${slider-install-dir}/slider-0.30.0/agent/conf/agent.ini
+ ${slider-install-dir}/slider-0.40.0/agent/conf/agent.ini
The sample agent.ini file can be used as is (see below). Some of the
parameters of interest are:
