Author: buildbot
Date: Mon Jul 27 21:02:04 2015
New Revision: 959809
Log:
Staging update by buildbot for slider
Modified:
websites/staging/slider/trunk/content/ (props changed)
websites/staging/slider/trunk/content/docs/slider_specs/application_pkg_docker.html
Propchange: websites/staging/slider/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Mon Jul 27 21:02:04 2015
@@ -1 +1 @@
-1688925
+1692951
Modified:
websites/staging/slider/trunk/content/docs/slider_specs/application_pkg_docker.html
==============================================================================
---
websites/staging/slider/trunk/content/docs/slider_specs/application_pkg_docker.html
(original)
+++
websites/staging/slider/trunk/content/docs/slider_specs/application_pkg_docker.html
Mon Jul 27 21:02:04 2015
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink,
h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink,
dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,8 +196,8 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1
id="apache-slider-docker-based-application-packaging-support-tech-preview">Apache
Slider Docker Based Application Packaging Support (Tech Preview)</h1>
-<h2 id="introduction">Introduction</h2>
+<h1
id="apache-slider-docker-based-application-packaging-support-tech-preview">Apache
Slider Docker Based Application Packaging Support (Tech Preview)<a
class="headerlink"
href="#apache-slider-docker-based-application-packaging-support-tech-preview"
title="Permanent link">¶</a></h1>
+<h2 id="introduction">Introduction<a class="headerlink" href="#introduction"
title="Permanent link">¶</a></h2>
<p>Docker is an open platform for developers and sysadmins to build, ship, and
run distributed applications.</p>
<ul>
<li>Users can use their existing Docker images without any modifications.</li>
@@ -194,20 +205,20 @@ Latest release: <strong>0.80.0-incubatin
<li>Slider can deploy their Docker containers within YARN cluster, with
resource management capability from YARN</li>
<li>They can use Slider to monitor and manage their Docker
containers/applications</li>
</ul>
-<h2 id="terminology">Terminology</h2>
-<h3 id="docker-image">Docker Image</h3>
+<h2 id="terminology">Terminology<a class="headerlink" href="#terminology"
title="Permanent link">¶</a></h2>
+<h3 id="docker-image">Docker Image<a class="headerlink" href="#docker-image"
title="Permanent link">¶</a></h3>
<ul>
<li>Applications definition file</li>
<li>Is composed of one or multiple layers</li>
<li>Users create Docker containers out of Docker images</li>
</ul>
-<h3 id="docker-container">Docker Container</h3>
+<h3 id="docker-container">Docker Container<a class="headerlink"
href="#docker-container" title="Permanent link">¶</a></h3>
<ul>
<li>A running instance of a Docker image</li>
<li>Comprises the application and its dependencies</li>
<li>Enjoys the resource isolation and allocation benefits of VMs but is much
more portable and efficient.</li>
</ul>
-<h2 id="specifications">Specifications</h2>
+<h2 id="specifications">Specifications<a class="headerlink"
href="#specifications" title="Permanent link">¶</a></h2>
<p>Refer to <a href="https://www.docker.com/whatisdocker/";><em>What is
Docker</em></a> for a quick overview of Docker and why it excels VM.</p>
<p><strong>Goal</strong></p>
<ul>
@@ -235,90 +246,89 @@ Latest release: <strong>0.80.0-incubatin
<p>Below is an example of how we can use Slider to deploy a multi-component
Dockerized application</p>
-<p><strong>appConfig.json</strong></p>
+<p><strong>metainfo.json</strong></p>
<div class="codehilite"><pre><span class="p">{</span>
- "<span class="n">schema</span>"<span class="p">:</span>
"<span class="n">http</span><span class="p">:</span><span
class="o">//</span><span class="n">example</span><span class="p">.</span><span
class="n">org</span><span class="o">/</span><span
class="n">specification</span><span class="o">/</span><span
class="n">v2</span><span class="p">.</span>0<span
class="p">.</span>0"<span class="p">,</span>
- "<span class="n">metadata</span>"<span class="p">:</span> <span
class="p">{</span> <span class="p">},</span>
- "<span class="k">global</span>"<span class="p">:</span> <span
class="p">{</span> <span class="p">},</span>
- "<span class="n">components</span>"<span class="p">:</span>
<span class="p">{</span>
- "<span class="n">REDIS</span>"<span class="p">:</span> <span
class="p">{</span>
- "<span class="n">memcached</span><span
class="p">.</span><span class="n">commandPath</span>"<span
class="p">:</span> "<span class="o">/</span><span
class="n">user</span><span class="o">/</span><span class="n">local</span><span
class="o">/</span><span class="n">bin</span><span class="o">/</span><span
class="n">docker</span>"<span class="p">,</span>
- "<span class="n">memcached</span><span
class="p">.</span><span class="n">options</span>"<span class="p">:</span>
"<span class="o">-</span><span class="n">d</span> <span
class="o">-</span><span class="n">e</span> <span
class="n">REDIS_PASS</span><span class="p">=</span><span
class="o">\</span>"<span class="o">**</span><span
class="n">None</span><span class="o">**\</span>""<span
class="p">,</span>
- "<span class="n">memcached</span><span
class="p">.</span><span class="n">statusCommand</span>"<span
class="p">:</span> "<span class="o">/</span><span
class="n">usr</span><span class="o">/</span><span class="n">bin</span><span
class="o">/</span><span class="n">docker</span> <span
class="n">ps</span>"<span class="p">,</span>
- "<span class="n">memcached</span><span
class="p">.</span><span class="n">inputFiles</span>"<span
class="p">:</span> <span class="p">[</span>
+"<span class="n">schemaVersion</span>"<span class="p">:</span>
"2<span class="p">.</span>1"<span class="p">,</span>
+"<span class="n">application</span>"<span class="p">:</span> <span
class="p">{</span>
+ "<span class="n">name</span>"<span class="p">:</span>
"<span class="n">NODEJS</span><span class="o">-</span><span
class="n">REDIS</span>"<span class="p">,</span>
+ "<span class="n">components</span>"<span class="p">:</span>
<span class="p">[</span>
+ <span class="p">{</span>
+ "<span class="n">name</span>"<span
class="p">:</span> "<span class="n">NODEJS</span>"<span
class="p">,</span>
+ "<span class="n">type</span>"<span
class="p">:</span> "<span class="n">docker</span>"<span
class="p">,</span>
+ "<span class="n">dockerContainers</span>"<span
class="p">:</span> <span class="p">[</span>
+ <span class="p">{</span>
+ "<span class="n">name</span>"<span
class="p">:</span> "<span class="n">nodejs</span>"<span
class="p">,</span>
+ "<span class="n">commandPath</span>"<span
class="p">:</span> "<span class="o">/</span><span
class="n">usr</span><span class="o">/</span><span class="n">bin</span><span
class="o">/</span><span class="n">docker</span>"<span class="p">,</span>
+ "<span class="n">image</span>"<span
class="p">:</span> "<span class="n">rsahahw</span><span
class="o">/</span><span class="n">centos</span><span class="o">-</span><span
class="n">node</span><span class="o">-</span><span
class="n">redis</span>"<span class="p">,</span>
+ "<span class="n">ports</span>"<span
class="p">:</span> <span class="p">[{</span>
+ "<span
class="n">containerPort</span>" <span class="p">:</span> "8000"
+
+ <span class="p">}]</span>
+ <span class="p">}</span>
+ <span class="p">]</span>
+ <span class="p">},</span>
+ <span class="p">{</span>
+ "<span class="n">name</span>"<span
class="p">:</span> "<span class="n">REDIS</span>"<span
class="p">,</span>
+ "<span class="n">type</span>"<span
class="p">:</span> "<span class="n">docker</span>"<span
class="p">,</span>
+ "<span class="n">dockerContainers</span>"<span
class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
- "<span class="n">containerPath</span>"<span
class="p">:</span> "<span class="o">/</span><span
class="n">tmp</span><span class="o">/</span><span
class="n">input</span>"<span class="p">,</span>
- "<span class="n">fileLocalPath</span>"<span
class="p">:</span> "<span class="o">/</span><span
class="n">Users</span><span class="o">/</span><span class="n">peter</span><span
class="o">/</span><span class="n">config</span><span class="p">.</span><span
class="n">json</span>"
- <span class="p">}</span>
- <span class="p">]</span>
- <span class="p">}</span>
+ "<span class="n">name</span>"<span
class="p">:</span> "<span class="n">redis</span>"<span
class="p">,</span>
+ "<span class="n">commandPath</span>"<span
class="p">:</span> "<span class="o">/</span><span
class="n">usr</span><span class="o">/</span><span class="n">bin</span><span
class="o">/</span><span class="n">docker</span>"<span class="p">,</span>
+ "<span class="n">image</span>"<span
class="p">:</span> "<span class="n">tutum</span><span
class="o">/</span><span class="n">redis</span>"<span class="p">,</span>
+ "<span class="n">ports</span>"<span
class="p">:</span> <span class="p">[{</span>
+ "<span
class="n">containerPort</span>" <span class="p">:</span>
"6379"<span class="p">,</span>
+ "<span
class="n">hostPort</span>"<span class="p">:</span> "6379"
+
+ <span class="p">}]</span>
+
+ <span class="p">}]</span>
+ <span class="p">}</span>
+
+ <span class="p">]</span>
<span class="p">}</span>
+
<span class="p">}</span>
</pre></div>
-<p>Please note that in this example, we are specifying a different docker
command path than the default <code>/usr/bin/docker</code> in appConfig.json.
We also specify the options that we need to include in the <code>docker
run</code> command (<code>-d</code> is by default included). This is just a
demo as how to include docker run command options. We also need to expose a
port to listen on for the application. Thus, we are specifying
<code>site.global.listen_port</code> in appConfig.json and the export config in
metainfo.json below</p>
-<p>The config structure in appConfig.json(map) -> components(list) ->
component(map) -> containers(list)-> container(map) have to match the one
in metainfo.json. </p>
-<p>We are also adding a few configurations for the component
<em>memcached</em> in appConfig.json here. These configurations are runtime
parameters of the component, as opposed to being part of the Docker image
definition in metainfo.json.</p>
-<p><strong>metainfo.json</strong></p>
+<p>Metainfo.json is the specification about how your image should run. All
fields in the config file are not different than those in a non Docker
application, except for those under "dockerContainers". All fields under
"dockerContainers" are required unless specified "optional" and they described
as below:</p>
+<ul>
+<li><strong>name</strong>: Name of your container. It doesn't affect how you
launch your application, but Slider will use it to identify the overriding
value in appConfig.json as shown later</li>
+<li><strong>image</strong>: The full name of you Docker image.</li>
+<li><strong>additionalParam</strong>: (optional) Additional params that need
to be passed to the command to be invoked during starting your Docker
container</li>
+<li><strong>commandPath</strong>: The path to your docker command</li>
+<li><strong>statusCommand</strong>: (optional) The command you can specify to
check the health of your running application. Slider relies on the return value
of the command to judge if healthy. 0 as healthy and non-0 as unhealthy. If not
provided, Slider will execute "docker top ${CONTAINER_ID} | grep"</li>
+<li><strong>port, containerPort and hostPort</strong>: (optional) the port of
the container that will be binded to the hostPort field, which will be
translated into <code>-p hostPort:containerPort</code> when starting the
container with <code>docker run</code></li>
+<li><strong>mount, containerMount and hostMount</strong>: (optional) the
directories from the host that will be mounted into the container, which will
be translated into â-v hostMount:containerMountâ when starting the
container</li>
+<li><strong>options</strong>: (optional) allow users to specify any additional
docker run command options to use. Those will be passed to 'docker run' command
when starting your application. If not specified, "-d" will be used</li>
+</ul>
+<p>Please note configs specified in metainfo.json can be overridden in
appConfig.json: <strong>commandPath, options, statusCommand, inputFiles,
mounts, ports</strong></p>
+<p><strong>appConfig.json</strong></p>
<div class="codehilite"><pre>{
- "schemaVersion": "2.1",
- "application": {
- "name": "FAKEAPP",
- "components": [
- {
- "name": "MEMCACHED",
- "type": "docker",
- "dockerContainers": [
- {
- "name": "memcached",
- "image": "borja/memcached",
- "additionalParam": "--appendonly
yes",
- "commandPath": "/usr/bin/docker",
- "statusCommand": "docker inspect <span
class="cp">${</span><span class="n">container_id</span><span
class="cp">}</span>"
- }
- ]
- },
- {
- "name": "REDIS",
- "type": "docker",
- "dockerContainers": [
- {
- "name": "redis",
- "image": "dockerhub/redis",
- "additionalParam": "--appendonly
yes",
- "commandPath": "/usr/bin/docker",
- "ports": [
- {
- "containerPort": "11211",
- "hostPort": "<span
class="cp">${</span><span class="n">conf</span><span class="p">:</span><span
class="n">configuration</span><span class="o">/</span><span
class="k">global</span><span class="o">/</span><span
class="n">port1</span><span class="cp">}</span>"
- }
- ],
- "mounts": [
- {
- "containerMount":
"/tmp/confg",
- "hostMount": "<span
class="cp">${</span><span class="n">conf</span><span class="p">:</span><span
class="n">configuration</span><span class="o">/</span><span
class="k">global</span><span class="o">/</span><span
class="n">app_root</span><span class="cp">}</span>/conf"
- }
- ]
- }
- ]
+ "schema": "http://example.org/specification/v2.0.0",
+ "metadata": {
+ },
+ "global": {
+ },
+ "components": {
+ "NODEJS": {
+ "nodejs.commandPath":
"/user/local/bin/docker",
+ "nodejs.options":"-d -e REDIS_HOST=<span
class="cp">${</span><span class="n">REDIS_HOST</span><span class="cp">}</span>
--net=host",
+ "nodejs.statusCommand":"docker inspect -f
{{.State.Running}} <span class="cp">${</span><span
class="n">CONTAINER_ID</span><span class="cp">}</span> | grep true"
+ },
+ "REDIS": {
+ "redis.options":"-d -e
REDIS_PASS=**None**",
+ "redis.statusCommand":"docker top <span
class="cp">${</span><span class="n">CONTAINER_ID</span><span
class="cp">}</span> | grep redis-server"
}
- ]
}
}
</pre></div>
-<p>Please note, in metainfo.json (we are trying to migrate from metainfo.xml
to metainfo.json as an improvement to the packaging approach) we are adding
some new fields in the component section to support Docker based
applications:</p>
-<ul>
-<li><strong>type</strong>: if specified as <code>docker</code>, Slider will
start Docker containers for the application; by default it is
<code>process</code>, which means Slider will instantiate the application as
normal process as today</li>
-<li><strong>containers/container</strong>: the Docker image of the
application, the name of which will be specified in <code>image</code>
field</li>
-<li><strong>port, containerPort and hostPort</strong>: the port of the
container that will be binded to the hostPort field, which will be translated
into <code>-p hostPort:containerPort</code> when starting the container with
<code>docker run</code></li>
-<li><strong>mount, containerMount and hostMount</strong>: the directories from
the host that will be mounted into the container, which will be translated into
â-v hostMount:containerMountâ when starting the container</li>
-<li><strong>options</strong>: allow users to specify any additional docker run
command options to use</li>
-<li><strong>status_command</strong>: the command that Slider can use to query
the status of the application component running in the container</li>
-<li>Properties specified in metainfo.json can be overridden in appConfig.json:
<strong>commandPath, options, statusCommand, inputFiles, mounts,
ports</strong></li>
-</ul>
+<p>AppConfig.json is where you can provide overriding config value for those
you defined in metainfo.json. This may be needed when you want to provide some
runtime variance.</p>
+<p>Please note that you need to specify the container name before your config
key: "nodejs.commandPath" instead of "commandPath". This is to distinguish
different containers under the same component as defined in metainfo.json</p>
+<p>In this example, we are specifying a different docker command path than the
default <code>/usr/bin/docker</code> in metainfo.json and a different set of
options that we need to include in the <code>docker run</code> command. </p>
+<p>The config structure in appConfig.json(map) -> components(list) ->
component(map) -> containers(list)-> container(map) have to match the one
in metainfo.json. </p>
<p><strong>resources.json</strong></p>
<div class="codehilite"><pre><span class="p">{</span>
"<span class="n">schema</span>"<span class="p">:</span>
"<span class="n">http</span><span class="p">:</span><span
class="o">//</span><span class="n">example</span><span class="p">.</span><span
class="n">org</span><span class="o">/</span><span
class="n">specification</span><span class="o">/</span><span
class="n">v2</span><span class="p">.</span>0<span
class="p">.</span>0"<span class="p">,</span>
@@ -328,12 +338,12 @@ Latest release: <strong>0.80.0-incubatin
"<span class="n">slider</span><span class="o">-</span><span
class="n">appmaster</span>"<span class="p">:</span> <span
class="p">{</span> <span class="p">},</span>
"<span class="n">REDIS</span>"<span class="p">:</span> <span
class="p">{</span>
"<span class="n">yarn</span><span class="p">.</span><span
class="n">role</span><span class="p">.</span><span
class="n">priority</span>"<span class="p">:</span> "1"<span
class="p">,</span>
- "<span class="n">yarn</span><span class="p">.</span><span
class="n">component</span><span class="p">.</span><span
class="n">instances</span>"<span class="p">:</span> "2"<span
class="p">,</span>
+ "<span class="n">yarn</span><span class="p">.</span><span
class="n">component</span><span class="p">.</span><span
class="n">instances</span>"<span class="p">:</span> "1"<span
class="p">,</span>
"<span class="n">yarn</span><span class="p">.</span><span
class="n">memory</span>"<span class="p">:</span> "512"
<span class="p">},</span>
- "<span class="n">MEMCACHED</span>"<span class="p">:</span>
<span class="p">{</span>
- "<span class="n">yarn</span><span class="p">.</span><span
class="n">role</span><span class="p">.</span><span
class="n">priority</span>"<span class="p">:</span> "1"<span
class="p">,</span>
- "<span class="n">yarn</span><span class="p">.</span><span
class="n">component</span><span class="p">.</span><span
class="n">instances</span>"<span class="p">:</span> "2"<span
class="p">,</span>
+ "<span class="n">NODEJS</span>"<span class="p">:</span>
<span class="p">{</span>
+ "<span class="n">yarn</span><span class="p">.</span><span
class="n">role</span><span class="p">.</span><span
class="n">priority</span>"<span class="p">:</span> "2"<span
class="p">,</span>
+ "<span class="n">yarn</span><span class="p">.</span><span
class="n">component</span><span class="p">.</span><span
class="n">instances</span>"<span class="p">:</span> "1"<span
class="p">,</span>
"<span class="n">yarn</span><span class="p">.</span><span
class="n">memory</span>"<span class="p">:</span> "512"
<span class="p">}</span>
<span class="p">}</span>
@@ -345,11 +355,11 @@ Latest release: <strong>0.80.0-incubatin
<ul>
<li>For redis components, it will run:</li>
</ul>
-<p><code>docker run -d -p hostPort:containerPort -v hostMount:containerMount
-net=bridge dockerhub/redis</code></p>
+<p><code>docker run -d -e REDIS_PASS=**None** -p hostPort:containerPort -name
${CONTAINER_ID} tutum/redis</code></p>
<ul>
-<li>For memcached components, it will run:</li>
+<li>For nodejs components, it will run:</li>
</ul>
-<p><code>docker run -d borja/memcached memcached_server.sh
--appendonly</code></p>
+<p><code>docker run -d -e REDIS_HOST=${REDIS_HOST} -name ${CONTAINER_ID}
--net=host rsahahw/centos-node-redis</code></p>
</div>
<div id="footer">
