Author: gourksaha
Date: Tue May 19 18:17:16 2015
New Revision: 1680364
URL: http://svn.apache.org/r1680364
Log:
SLIDER-780 Support for Docker based application packaging in Slider - site
documentation (Thomas Liu via gourksaha)
Added:
incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_docker.md
Modified:
incubator/slider/site/trunk/templates/nav.html
Added:
incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_docker.md
URL:
http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_docker.md?rev=1680364&view=auto
==============================================================================
---
incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_docker.md
(added)
+++
incubator/slider/site/trunk/content/docs/slider_specs/application_pkg_docker.md
Tue May 19 18:17:16 2015
@@ -0,0 +1,192 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Apache Slider Docker Based Application Packaging Support
+
+##Introduction
+
+Docker is an open platform for developers and sysadmins to build, ship, and
run distributed applications.
+
+- Users can use their existing Docker images without any modifications.
+- They need to create configuration files according to Slider's template for
Docker based applications
+- Slider can deploy their Docker containers within YARN cluster, with resource
management capability from YARN
+- They can use Slider to monitor and manage their Docker
containers/applications
+
+## Terminology
+
+### Docker Image
+- Applications definition file
+- Is composed of one or multiple layers
+- Users create Docker containers out of Docker images
+
+### Docker Container
+- A running instance of a Docker image
+- Comprises the application and its dependencies
+- Enjoys the resource isolation and allocation benefits of VMs but is much
more portable and efficient.
+
+##Specifications
+
+Refer to [*What is Docker*](https://www.docker.com/whatisdocker/) for a quick
overview of Docker and why it excels VM.
+
+**Goal**
+
+- Slider shall be able to deploy an application defined as Docker image,
monitor its running status, fetch exported configs, and maintain its lifecycle
+- Users can use their existing Docker images without any modifications
+- Slider shall be able to pull Docker images from Docker Hubs
+- Slider shall be able to aggregate application logs from and feed in input
files into Docker containers
+- Slider shall be able to launch applications containing multiple Docker
images, as well as maintaining multiple Docker containers on the same physical
host
+- Users shall be able to specify the Docker images, configurations and if
necessary, the special docker command to start/query/stop the container in
Slider configuration files
+- Slider shall be able to deploy security credentials into the container if
the application is running in secure mode
+
+**The scope of this JIRA doesnât yet include**
+
+- Application Docker containers integrated with Ambari client
+- Applications deployed in secure mode
+- Running application Docker containers in OS other than Linux
+- Docker images stored in private Docker hub - we would leave this work to
administrators to configure Docker daemon to be able to pull from those hubs
+- Application Docker containers started with `docker run -i` option which
requires interaction with the user
+- Docker images that require multiple initialization steps, more than just one
`docker run`
+- YARN can only monitor resources consumed by SliderAgent, instead of that of
the Docker containers
+
+**Commands to run**
+
+ slider create [app-name] --template appConfig.json --metainfo
metainfo.json --resources resources.json
+
+Below is an example of how we can use Slider to deploy a multi-component
Dockerized application
+
+**appConfig.json**
+```
+ {
+ "schema": "http://example.org/specification/v2.0.0";,
+ "metadata": {
+ },
+ "global": {
+ },
+ "components": {
+ "REDIS": {
+ "memcached.commandPath":"/user/local/bin/docker",
+ "memcached.options":"-d -e REDIS_PASS=\"**None**\"",
+ "memcached.statusCommand":"/usr/bin/docker ps",
+ "memcached.inputFiles": [
+ {
+ "containerPath": "/tmp/input",
+ "fileLocalPath": "/Users/peter/config.json"
+ }
+ ]
+ }
+ }
+ }
+```
+Please note that in this example, we are specifying a different docker command
path than the default `/usr/bin/docker` in appConfig.json. We also specify the
options that we need to include in the `docker run` command (`-d` is by default
included). This is just a demo as how to include docker run command options. We
also need to expose a port to listen on for the application. Thus, we are
specifying `site.global.listen_port` in appConfig.json and the export config in
metainfo.json below
+
+ The config structure in appConfig.json(map) -> components(list) ->
component(map) -> containers(list)-> container(map) have to match the one in
metainfo.json.
+
+We are also adding a few configurations for the component *memcached* in
appConfig.json here. These configurations are runtime parameters of the
component, as opposed to being part of the Docker image definition in
metainfo.json.
+
+**metainfo.json**
+```
+ {
+ "schemaVersion": "2.1",
+ "application": {
+ "name": "FAKEAPP",
+ "components": [
+ {
+ "name": "MEMCACHED",
+ "type": "docker",
+ "dockerContainers": [
+ {
+ "name": "memcached",
+ "image": "borja/memcached",
+ "additionalParam": "--appendonly yes",
+ "commandPath": "/usr/bin/docker",
+ "statusCommand": "docker inspect ${container_id}"
+ }
+ ]
+ },
+ {
+ "name": "REDIS",
+ "type": "docker",
+ "dockerContainers": [
+ {
+ "name": "redis",
+ "image": "dockerhub/redis",
+ "additionalParam": "--appendonly yes",
+ "commandPath": "/usr/bin/docker",
+ "ports": [
+ {
+ "containerPort" : "11211",
+ "hostPort":
"${conf:configuration/global/port1}"
+ }
+ ],
+ "mounts": [
+ {
+ "containerMount": "/tmp/confg",
+ "hostMount":
"${conf:configuration/global/app_root}/conf"
+ }
+ ]
+ }
+ ]
+ }
+ ]
+ }
+ }
+```
+Please note, in metainfo.json (we are trying to migrate from metainfo.xml to
metainfo.json as an improvement to the packaging approach) we are adding some
new fields in the component section to support Docker based applications:
+
+- **type**: if specified as `docker`, Slider will start Docker containers for
the application; by default it is `process`, which means Slider will
instantiate the application as normal process as today
+- **containers/container**: the Docker image of the application, the name of
which will be specified in `image` field
+- **port, containerPort and hostPort**: the port of the container that will be
binded to the hostPort field, which will be translated into `-p
hostPort:containerPort` when starting the container with `docker run`
+- **mount, containerMount and hostMount**: the directories from the host that
will be mounted into the container, which will be translated into â-v
hostMount:containerMountâ when starting the container
+- **options**: allow users to specify any additional docker run command
options to use
+- **status_command**: the command that Slider can use to query the status of
the application component running in the container
+- Properties specified in metainfo.json can be overridden in appConfig.json:
**commandPath, options, statusCommand, inputFiles, mounts, ports**
+
+**resources.json**
+```
+{
+ "schema" : "http://example.org/specification/v2.0.0";,
+ "metadata" : {
+ },
+ "global" : {
+ },
+ "components": {
+ "slider-appmaster": {
+ },
+ "REDIS": {
+ "yarn.role.priority": "1",
+ "yarn.component.instances": "2",
+ "yarn.memory": "512"
+ },
+ "MEMCACHED": {
+ "yarn.role.priority": "1",
+ "yarn.component.instances": "2",
+ "yarn.memory": "512"
+ }
+
+ }
+}
+```
+With the configuration files above, Slider will ask for the required number of
containers from YARN, as specified in resources.json. In each of those YARN
containers, Slider will run `docker pull` to download the Docker image
specified in metainfo.json. In the scope of this JIRA ticket, we are not
supporting Docker images stored in private Docker hubs that require credentials
to run docker pull. After downloading completes, Slider will start the
containers by running, in this case, `/usr/bin/docker run -d borja/memcached`
+
+- For redis components, it will run:
+
+`docker run -d -p hostPort:containerPort -v hostMount:containerMount
-net=bridge dockerhub/redis`
+
+- For memcached components, it will run:
+
+`docker run -d borja/memcached memcached_server.sh --appendonly`
+
Modified: incubator/slider/site/trunk/templates/nav.html
URL:
http://svn.apache.org/viewvc/incubator/slider/site/trunk/templates/nav.html?rev=1680364&r1=1680363&r2=1680364&view=diff
==============================================================================
--- incubator/slider/site/trunk/templates/nav.html (original)
+++ incubator/slider/site/trunk/templates/nav.html Tue May 19 18:17:16 2015
@@ -72,6 +72,7 @@
<li role="presentation" class="divider"></li>
<li id="nav_use_app_package"><a href="/docs/slider_specs/index.html">Creating
App Packages</a></li>
<li id="nav_use_app_package_upgrade"><a
href="/docs/slider_specs/application_pkg_upgrade.html">Upgrading App
Packages</a></li>
+<li id="nav_use_app_package_docker"><a
href="/docs/slider_specs/application_pkg_docker.html">Docker App
Packages</a></li>
<li role="presentation" class="divider"></li>
<li id="nav_use_client_config"><a
href="/docs/client-configuration.html">Client Configuration</a></li>
<li id="nav_use_exitcodes"><a href="/docs/exitcodes.html">Client Exit
Codes</a></li>
