Added: aurora/site/publish/documentation/0.12.0/build-system/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.12.0/build-system/index.html?rev=1733548&view=auto ============================================================================== --- aurora/site/publish/documentation/0.12.0/build-system/index.html (added) +++ aurora/site/publish/documentation/0.12.0/build-system/index.html Fri Mar 4 02:43:01 2016 @@ -0,0 +1,206 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/build-system/'" + value="0.12.0"> + <option value="0.12.0" + selected="selected"> + 0.12.0 + (latest) + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<p>The Python components of Aurora are built using <a href="https://pantsbuild.github.io";>Pants</a>.</p> + +<h1 id="python-build-conventions">Python Build Conventions</h1> + +<p>The Python code is laid out according to the following conventions: </p> + +<ol> +<li><p>1 <code>BUILD</code> per 3rd level directory. For a list of current top-level packages run:</p> +<pre class="highlight plaintext"><code>% find src/main/python -maxdepth 3 -mindepth 3 -type d |\ +while read dname; do echo $dname |\ + sed 's@src/main/python/\(.*\)/\(.*\)/\(.*\).*@\1.\2.\3@'; done +</code></pre></li> +<li><p>Each <code>BUILD</code> file exports 1 +<a href="https://pantsbuild.github.io/build_dictionary.html#bdict_python_library";><code>python_library</code></a> +that provides a +<a href="https://pantsbuild.github.io/build_dictionary.html#setup_py";><code>setup_py</code></a> +containing each +<a href="https://pantsbuild.github.io/build_dictionary.html#python_binary";><code>python_binary</code></a> +in the <code>BUILD</code> file, named the same as the directory it’s in so that it can be referenced +without a ’:’ character. The <code>sources</code> field in the <code>python_library</code> will almost always be +<code>rglobs('*.py')</code>.</p></li> +<li><p>Other BUILD files may only depend on this single public <code>python_library</code> +target. Any other target is considered a private implementation detail and +should be prefixed with an <code>_</code>.</p></li> +<li><p><code>python_binary</code> targets are always named the same as the exported console script.</p></li> +<li><p><code>python_binary</code> targets must have identical <code>dependencies</code> to the <code>python_library</code> exported +by the package and must use <code>entry_point</code>.</p> + +<p>The means a PEX file generated by pants will contain exactly the same files that will be +available on the <code>PYTHONPATH</code> in the case of <code>pip install</code> of the corresponding library +target. This will help our migration off of Pants in the future.</p></li> +</ol> + +<h2 id="annotated-example-apache-thermos-runner">Annotated example - apache.thermos.runner</h2> +<pre class="highlight plaintext"><code>% find src/main/python/apache/thermos/runner +src/main/python/apache/thermos/runner +src/main/python/apache/thermos/runner/__init__.py +src/main/python/apache/thermos/runner/thermos_runner.py +src/main/python/apache/thermos/runner/BUILD +% cat src/main/python/apache/thermos/runner/BUILD +# License boilerplate omitted +import os + + +# Private target so that a setup_py can exist without a circular dependency. Only targets within +# this file should depend on this. +python_library( + name = '_runner', + # The target covers every python file under this directory and subdirectories. + sources = rglobs('*.py'), + dependencies = [ + '3rdparty/python:twitter.common.app', + '3rdparty/python:twitter.common.log', + # Source dependencies are always referenced without a ':'. + 'src/main/python/apache/thermos/common', + 'src/main/python/apache/thermos/config', + 'src/main/python/apache/thermos/core', + ], +) + +# Binary target for thermos_runner.pex. Nothing should depend on this - it's only used as an +# argument to ./pants binary. +python_binary( + name = 'thermos_runner', + # Use entry_point, not source so the files used here are the same ones tests see. + entry_point = 'apache.thermos.bin.thermos_runner', + dependencies = [ + # Notice that we depend only on the single private target from this BUILD file here. + ':_runner', + ], +) + +# The public library that everyone importing the runner symbols uses. +# The test targets and any other dependent source code should depend on this. +python_library( + name = 'runner', + dependencies = [ + # Again, notice that we depend only on the single private target from this BUILD file here. + ':_runner', + ], + # We always provide a setup_py. This will cause any dependee libraries to automatically + # reference this library in their requirements.txt rather than copy the source files into their + # sdist. + provides = setup_py( + # Conventionally named and versioned. + name = 'apache.thermos.runner', + version = open(os.path.join(get_buildroot(), '.auroraversion')).read().strip().upper(), + ).with_binaries({ + # Every binary in this file should also be repeated here. + # Always use the dict-form of .with_binaries so that commands with dashes in their names are + # supported. + # The console script name is always the same as the PEX with .pex stripped. + 'thermos_runner': ':thermos_runner', + }), +) +</code></pre> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html>
Added: aurora/site/publish/documentation/0.12.0/client-cluster-configuration/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.12.0/client-cluster-configuration/index.html?rev=1733548&view=auto ============================================================================== --- aurora/site/publish/documentation/0.12.0/client-cluster-configuration/index.html (added) +++ aurora/site/publish/documentation/0.12.0/client-cluster-configuration/index.html Fri Mar 4 02:43:01 2016 @@ -0,0 +1,222 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/client-cluster-configuration/'" + value="0.12.0"> + <option value="0.12.0" + selected="selected"> + 0.12.0 + (latest) + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="client-cluster-configuration">Client Cluster Configuration</h1> + +<p>A cluster configuration file is used by the Aurora client to describe the Aurora clusters with +which it can communicate. Ultimately this allows client users to reference clusters with short names +like us-east and eu. The following properties may be set:</p> + +<table><thead> +<tr> +<th style="text-align: left"><strong>Property</strong></th> +<th style="text-align: left"><strong>Type</strong></th> +<th style="text-align: left"><strong>Description</strong></th> +</tr> +</thead><tbody> +<tr> +<td style="text-align: left"><strong>name</strong></td> +<td style="text-align: left">String</td> +<td style="text-align: left">Cluster name (Required)</td> +</tr> +<tr> +<td style="text-align: left"><strong>slave_root</strong></td> +<td style="text-align: left">String</td> +<td style="text-align: left">Path to mesos slave work dir (Required)</td> +</tr> +<tr> +<td style="text-align: left"><strong>slave<em>run</em>directory</strong></td> +<td style="text-align: left">String</td> +<td style="text-align: left">Name of mesos slave run dir (Required)</td> +</tr> +<tr> +<td style="text-align: left"><strong>zk</strong></td> +<td style="text-align: left">String</td> +<td style="text-align: left">Hostname of ZooKeeper instance used to resolve Aurora schedulers.</td> +</tr> +<tr> +<td style="text-align: left"><strong>zk_port</strong></td> +<td style="text-align: left">Integer</td> +<td style="text-align: left">Port of ZooKeeper instance used to locate Aurora schedulers (Default: 2181)</td> +</tr> +<tr> +<td style="text-align: left"><strong>scheduler<em>zk</em>path</strong></td> +<td style="text-align: left">String</td> +<td style="text-align: left">ZooKeeper path under which scheduler instances are registered.</td> +</tr> +<tr> +<td style="text-align: left"><strong>scheduler_uri</strong></td> +<td style="text-align: left">String</td> +<td style="text-align: left">URI of Aurora scheduler instance.</td> +</tr> +<tr> +<td style="text-align: left"><strong>proxy_url</strong></td> +<td style="text-align: left">String</td> +<td style="text-align: left">Used by the client to format URLs for display.</td> +</tr> +<tr> +<td style="text-align: left"><strong>auth_mechanism</strong></td> +<td style="text-align: left">String</td> +<td style="text-align: left">The authentication mechanism to use when communicating with the scheduler. (Default: UNAUTHENTICATED)</td> +</tr> +</tbody></table> + +<h4 id="name">name</h4> + +<p>The name of the Aurora cluster represented by this entry. This name will be the <code>cluster</code> portion of +any job keys identifying jobs running within the cluster.</p> + +<h4 id="slave_root">slave_root</h4> + +<p>The path on the mesos slaves where executing tasks can be found. It is used in combination with the +<code>slave_run_directory</code> property by <code>aurora task run</code> and <code>aurora task ssh</code> to change into the sandbox +directory after connecting to the host. This value should match the value passed to <code>mesos-slave</code> +as <code>-work_dir</code>.</p> + +<h4 id="slaverundirectory">slave<em>run</em>directory</h4> + +<p>The name of the directory where the task run can be found. This is used in combination with the +<code>slave_root</code> property by <code>aurora task run</code> and <code>aurora task ssh</code> to change into the sandbox +directory after connecting to the host. This should almost always be set to <code>latest</code>.</p> + +<h4 id="zk">zk</h4> + +<p>The hostname of the ZooKeeper instance used to resolve the Aurora scheduler. Aurora uses ZooKeeper +to elect a leader. The client will connect to this ZooKeeper instance to determine the current +leader. This host should match the host passed to the scheduler as <code>-zk_endpoints</code>.</p> + +<h4 id="zk_port">zk_port</h4> + +<p>The port on which the ZooKeeper instance is running. If not set this will default to the standard +ZooKeeper port of 2181. This port should match the port in the host passed to the scheduler as +<code>-zk_endpoints</code>.</p> + +<h4 id="schedulerzkpath">scheduler<em>zk</em>path</h4> + +<p>The path on the ZooKeeper instance under which the Aurora serverset is registered. This value should +match the value passed to the scheduler as <code>-serverset_path</code>.</p> + +<h4 id="scheduler_uri">scheduler_uri</h4> + +<p>The URI of the scheduler. This would be used in place of the ZooKeeper related configuration above +in circumstances where direct communication with a single scheduler is needed (e.g. testing +environments). It is strongly advised to <strong>never</strong> use this property for production deploys.</p> + +<h4 id="proxy_url">proxy_url</h4> + +<p>Instead of using the hostname of the leading scheduler as the base url, if <code>proxy_url</code> is set, its +value will be used instead. In that scenario the value for <code>proxy_url</code> would be, for example, the +URL of your VIP in a loadbalancer or a roundrobin DNS name.</p> + +<h4 id="auth_mechanism">auth_mechanism</h4> + +<p>The identifier of an authentication mechanism that the client should use when communicating with the +scheduler. Support for values other than <code>UNAUTHENTICATED</code> requires a matching scheduler-side +<a href="/documentation/0.12.0/security/">security configuration</a>.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.12.0/client-commands/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.12.0/client-commands/index.html?rev=1733548&view=auto ============================================================================== --- aurora/site/publish/documentation/0.12.0/client-commands/index.html (added) +++ aurora/site/publish/documentation/0.12.0/client-commands/index.html Fri Mar 4 02:43:01 2016 @@ -0,0 +1,490 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/client-commands/'" + value="0.12.0"> + <option value="0.12.0" + selected="selected"> + 0.12.0 + (latest) + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="aurora-client-commands">Aurora Client Commands</h1> + +<ul> +<li><a href="#introduction">Introduction</a></li> +<li><a href="#cluster-configuration">Cluster Configuration</a></li> +<li><a href="#job-keys">Job Keys</a></li> +<li><a href="#modifying-aurora-client-commands">Modifying Aurora Client Commands</a></li> +<li><a href="#regular-jobs">Regular Jobs</a> + +<ul> +<li><a href="#creating-and-running-a-job">Creating and Running a Job</a></li> +<li><a href="#running-a-command-on-a-running-job">Running a Command On a Running Job</a></li> +<li><a href="#killing-a-job">Killing a Job</a></li> +<li><a href="#updating-a-job">Updating a Job</a> + +<ul> +<li><a href="#coordinated-job-updates">Coordinated job updates</a></li> +</ul></li> +<li><a href="#renaming-a-job">Renaming a Job</a></li> +<li><a href="#restarting-jobs">Restarting Jobs</a></li> +</ul></li> +<li><a href="#cron-jobs">Cron Jobs</a></li> +<li><a href="#comparing-jobs">Comparing Jobs</a></li> +<li><a href="#viewingexamining-jobs">Viewing/Examining Jobs</a> + +<ul> +<li><a href="#listing-jobs">Listing Jobs</a></li> +<li><a href="#inspecting-a-job">Inspecting a Job</a></li> +<li><a href="#versions">Versions</a></li> +<li><a href="#checking-your-quota">Checking Your Quota</a></li> +<li><a href="#finding-a-job-on-web-ui">Finding a Job on Web UI</a></li> +<li><a href="#getting-job-status">Getting Job Status</a></li> +<li><a href="#opening-the-web-ui">Opening the Web UI</a></li> +<li><a href="#sshing-to-a-specific-task-machine">SSHing to a Specific Task Machine</a></li> +<li><a href="#templating-command-arguments">Templating Command Arguments</a></li> +</ul></li> +</ul> + +<h2 id="introduction">Introduction</h2> + +<p>Once you have written an <code>.aurora</code> configuration file that describes +your Job and its parameters and functionality, you interact with Aurora +using Aurora Client commands. This document describes all of these commands +and how and when to use them. All Aurora Client commands start with +<code>aurora</code>, followed by the name of the specific command and its +arguments.</p> + +<p><em>Job keys</em> are a very common argument to Aurora commands, as well as the +gateway to useful information about a Job. Before using Aurora, you +should read the next section which describes them in detail. The section +after that briefly describes how you can modify the behavior of certain +Aurora Client commands, linking to a detailed document about how to do +that.</p> + +<p>This is followed by the Regular Jobs section, which describes the basic +Client commands for creating, running, and manipulating Aurora Jobs. +After that are sections on Comparing Jobs and Viewing/Examining Jobs. In +other words, various commands for getting information and metadata about +Aurora Jobs.</p> + +<h2 id="cluster-configuration">Cluster Configuration</h2> + +<p>The client must be able to find a configuration file that specifies available clusters. This file +declares shorthand names for clusters, which are in turn referenced by job configuration files +and client commands.</p> + +<p>The client will load at most two configuration files, making both of their defined clusters +available. The first is intended to be a system-installed cluster, using the path specified in +the environment variable <code>AURORA_CONFIG_ROOT</code>, defaulting to <code>/etc/aurora/clusters.json</code> if the +environment variable is not set. The second is a user-installed file, located at +<code>~/.aurora/clusters.json</code>.</p> + +<p>A cluster configuration is formatted as JSON. The simplest cluster configuration is one that +communicates with a single (non-leader-elected) scheduler. For example:</p> +<pre class="highlight javascript"><code><span style="background-color: #f8f8f8">[{</span> + <span style="color: #d14">"name"</span><span style="background-color: #f8f8f8">:</span> <span style="color: #d14">"example"</span><span style="background-color: #f8f8f8">,</span> + <span style="color: #d14">"scheduler_uri"</span><span style="background-color: #f8f8f8">:</span> <span style="color: #d14">"localhost:55555"</span><span style="background-color: #f8f8f8">,</span> +<span style="background-color: #f8f8f8">}]</span> +</code></pre> + +<p>A configuration for a leader-elected scheduler would contain something like:</p> +<pre class="highlight javascript"><code><span style="background-color: #f8f8f8">[{</span> + <span style="color: #d14">"name"</span><span style="background-color: #f8f8f8">:</span> <span style="color: #d14">"example"</span><span style="background-color: #f8f8f8">,</span> + <span style="color: #d14">"zk"</span><span style="background-color: #f8f8f8">:</span> <span style="color: #d14">"192.168.33.7"</span><span style="background-color: #f8f8f8">,</span> + <span style="color: #d14">"scheduler_zk_path"</span><span style="background-color: #f8f8f8">:</span> <span style="color: #d14">"/aurora/scheduler"</span> +<span style="background-color: #f8f8f8">}]</span> +</code></pre> + +<p>For more details on cluster configuration see the +<a href="/documentation/0.12.0/client-cluster-configuration/">Client Cluster Configuration</a> documentation.</p> + +<h2 id="job-keys">Job Keys</h2> + +<p>A job key is a unique system-wide identifier for an Aurora-managed +Job, for example <code>cluster1/web-team/test/experiment204</code>. It is a 4-tuple +consisting of, in order, <em>cluster</em>, <em>role</em>, <em>environment</em>, and +<em>jobname</em>, separated by /s. Cluster is the name of an Aurora +cluster. Role is the Unix service account under which the Job +runs. Environment is a namespace component like <code>devel</code>, <code>test</code>, +<code>prod</code>, or <code>stagingN.</code> Jobname is the Job’s name.</p> + +<p>The combination of all four values uniquely specifies the Job. If any +one value is different from that of another job key, the two job keys +refer to different Jobs. For example, job key +<code>cluster1/tyg/prod/workhorse</code> is different from +<code>cluster1/tyg/prod/workcamel</code> is different from +<code>cluster2/tyg/prod/workhorse</code> is different from +<code>cluster2/foo/prod/workhorse</code> is different from +<code>cluster1/tyg/test/workhorse.</code></p> + +<p>Role names are user accounts existing on the slave machines. If you don’t know what accounts +are available, contact your sysadmin.</p> + +<p>Environment names are namespaces; you can count on <code>prod</code>, <code>devel</code> and <code>test</code> existing.</p> + +<h2 id="modifying-aurora-client-commands">Modifying Aurora Client Commands</h2> + +<p>For certain Aurora Client commands, you can define hook methods that run +either before or after an action that takes place during the command’s +execution, as well as based on whether the action finished successfully or failed +during execution. Basically, a hook is code that lets you extend the +command’s actions. The hook executes on the client side, specifically on +the machine executing Aurora commands.</p> + +<p>Hooks can be associated with these Aurora Client commands.</p> + +<ul> +<li><code>job create</code></li> +<li><code>job kill</code></li> +<li><code>job restart</code></li> +</ul> + +<p>The process for writing and activating them is complex enough +that we explain it in a devoted document, <a href="/documentation/0.12.0/hooks/">Hooks for Aurora Client API</a>.</p> + +<h2 id="regular-jobs">Regular Jobs</h2> + +<p>This section covers Aurora commands related to running, killing, +renaming, updating, and restarting a basic Aurora Job.</p> + +<h3 id="creating-and-running-a-job">Creating and Running a Job</h3> +<pre class="highlight plaintext"><code>aurora job create <job key> <configuration file> +</code></pre> + +<p>Creates and then runs a Job with the specified job key based on a <code>.aurora</code> configuration file. +The configuration file may also contain and activate hook definitions.</p> + +<h3 id="running-a-command-on-a-running-job">Running a Command On a Running Job</h3> +<pre class="highlight plaintext"><code>aurora task run CLUSTER/ROLE/ENV/NAME[/INSTANCES] <cmd> +</code></pre> + +<p>Runs a shell command on all machines currently hosting shards of a +single Job.</p> + +<p><code>run</code> supports the same command line wildcards used to populate a Job’s +commands; i.e. anything in the <code>{{mesos.*}}</code> and <code>{{thermos.*}}</code> +namespaces.</p> + +<h3 id="killing-a-job">Killing a Job</h3> +<pre class="highlight plaintext"><code>aurora job killall CLUSTER/ROLE/ENV/NAME +</code></pre> + +<p>Kills all Tasks associated with the specified Job, blocking until all +are terminated. Defaults to killing all instances in the Job.</p> + +<p>The <code><configuration file></code> argument for <code>kill</code> is optional. Use it only +if it contains hook definitions and activations that affect the +kill command.</p> + +<h3 id="updating-a-job">Updating a Job</h3> + +<p>There are several sub-commands to manage job updates:</p> +<pre class="highlight plaintext"><code>aurora update start <job key> <configuration file> +aurora update info <job key> +aurora update pause <job key> +aurora update resume <job key> +aurora update abort <job key> +aurora update list <cluster> +</code></pre> + +<p>When you <code>start</code> a job update, the command will return once it has sent the +instructions to the scheduler. At that point, you may view detailed +progress for the update with the <code>info</code> subcommand, in addition to viewing +graphical progress in the web browser. You may also get a full listing of +in-progress updates in a cluster with <code>list</code>.</p> + +<p>Once an update has been started, you can <code>pause</code> to keep the update but halt +progress. This can be useful for doing things like debug a partially-updated +job to determine whether you would like to proceed. You can <code>resume</code> to +proceed.</p> + +<p>You may <code>abort</code> a job update regardless of the state it is in. This will +instruct the scheduler to completely abandon the job update and leave the job +in the current (possibly partially-updated) state.</p> + +<h4 id="coordinated-job-updates">Coordinated job updates</h4> + +<p>Some Aurora services may benefit from having more control over updates by explicitly +acknowledging (“heartbeating”) job update progress. This may be helpful for mission-critical +service updates where explicit job health monitoring is vital during the entire job update +lifecycle. Such job updates would rely on an external service (or a custom client) periodically +pulsing an active coordinated job update via a +<a href="https://github.com/apache/aurora/blob/#{git_tag}/api/src/main/thrift/org/apache/aurora/gen/api.thrift";>pulseJobUpdate RPC</a>).</p> + +<p>A coordinated update is defined by setting a positive +<a href="/documentation/0.12.0/configuration-reference/#updateconfig-objects">pulse<em>interval</em>secs</a> value in job configuration +file. If no pulses are received within specified interval the update will be blocked. A blocked +update is unable to continue rolling forward (or rolling back) but retains its active status. +It may only be unblocked by a fresh <code>pulseJobUpdate</code> call.</p> + +<p>NOTE: A coordinated update starts in <code>ROLL_FORWARD_AWAITING_PULSE</code> state and will not make any +progress until the first pulse arrives. However, a paused update (<code>ROLL_FORWARD_PAUSED</code> or +<code>ROLL_BACK_PAUSED</code>) is still considered active and upon resuming will immediately make progress +provided the pulse interval has not expired.</p> + +<h3 id="renaming-a-job">Renaming a Job</h3> + +<p>Renaming is a tricky operation as downstream clients must be informed of +the new name. A conservative approach +to renaming suitable for production services is:</p> + +<ol> +<li> Modify the Aurora configuration file to change the role, +environment, and/or name as appropriate to the standardized naming +scheme.</li> +<li><p>Check that only these naming components have changed +with <code>aurora diff</code>.</p> +<pre class="highlight plaintext"><code>aurora job diff CLUSTER/ROLE/ENV/NAME <job_configuration> +</code></pre></li> +<li><p>Create the (identical) job at the new key. You may need to request a +temporary quota increase.</p> +<pre class="highlight plaintext"><code>aurora job create CLUSTER/ROLE/ENV/NEW_NAME <job_configuration> +</code></pre></li> +<li><p>Migrate all clients over to the new job key. Update all links and +dashboards. Ensure that both job keys run identical versions of the +code while in this state.</p></li> +<li><p>After verifying that all clients have successfully moved over, kill +the old job.</p> +<pre class="highlight plaintext"><code>aurora job killall CLUSTER/ROLE/ENV/NAME +</code></pre></li> +<li><p>If you received a temporary quota increase, be sure to let the +powers that be know you no longer need the additional capacity.</p></li> +</ol> + +<h3 id="restarting-jobs">Restarting Jobs</h3> + +<p><code>restart</code> restarts all of a job key identified Job’s shards:</p> +<pre class="highlight plaintext"><code>aurora job restart CLUSTER/ROLE/ENV/NAME[/INSTANCES] +</code></pre> + +<p>Restarts are controlled on the client side, so aborting +the <code>job restart</code> command halts the restart operation.</p> + +<p><strong>Note</strong>: <code>job restart</code> only applies its command line arguments and does not +use or is affected by <code>update.config</code>. Restarting +does <strong><em>not</em></strong> involve a configuration change. To update the +configuration, use <code>update.config</code>.</p> + +<p>The <code>--config</code> argument for restart is optional. Use it only +if it contains hook definitions and activations that affect the +<code>job restart</code> command.</p> + +<h2 id="cron-jobs">Cron Jobs</h2> + +<p>You can manage cron jobs using the <code>aurora cron</code> command. Please see +<a href="/documentation/0.12.0/cron-jobs/">cron-jobs.md</a> for more details.</p> + +<p>You will see various commands and options relating to cron jobs in +<code>aurora -h</code> and similar. Ignore them, as they’re not yet implemented.</p> + +<h2 id="comparing-jobs">Comparing Jobs</h2> +<pre class="highlight plaintext"><code>aurora job diff CLUSTER/ROLE/ENV/NAME <job configuration> +</code></pre> + +<p>Compares a job configuration against a running job. By default the diff +is determined using <code>diff</code>, though you may choose an alternate + diff program by specifying the <code>DIFF_VIEWER</code> environment variable.</p> + +<h2 id="viewing-examining-jobs">Viewing/Examining Jobs</h2> + +<p>Above we discussed creating, killing, and updating Jobs. Here we discuss +how to view and examine Jobs.</p> + +<h3 id="listing-jobs">Listing Jobs</h3> +<pre class="highlight plaintext"><code>aurora config list <job configuration> +</code></pre> + +<p>Lists all Jobs registered with the Aurora scheduler in the named cluster for the named role.</p> + +<h3 id="inspecting-a-job">Inspecting a Job</h3> +<pre class="highlight plaintext"><code>aurora job inspect CLUSTER/ROLE/ENV/NAME <job configuration> +</code></pre> + +<p><code>inspect</code> verifies that its specified job can be parsed from a +configuration file, and displays the parsed configuration.</p> + +<h3 id="checking-your-quota">Checking Your Quota</h3> +<pre class="highlight plaintext"><code>aurora quota get CLUSTER/ROLE +</code></pre> + +<p>Prints the production quota allocated to the role’s value at the given +cluster. Only non-<a href="/documentation/0.12.0/deploying-aurora-scheduler/#dedicated-attribute">dedicated</a> +<a href="/documentation/0.12.0/configuration-reference/#job-objects">production</a> jobs consume quota.</p> + +<h3 id="finding-a-job-on-web-ui">Finding a Job on Web UI</h3> + +<p>When you create a job, part of the output response contains a URL that goes +to the job’s scheduler UI page. For example:</p> +<pre class="highlight plaintext"><code>vagrant@precise64:~$ aurora job create devcluster/www-data/prod/hello /vagrant/examples/jobs/hello_world.aurora +INFO] Creating job hello +INFO] Response from scheduler: OK (message: 1 new tasks pending for job www-data/prod/hello) +INFO] Job url: http://precise64:8081/scheduler/www-data/prod/hello +</code></pre> + +<p>You can go to the scheduler UI page for this job via <code>http://precise64:8081/scheduler/www-data/prod/hello</code> +You can go to the overall scheduler UI page by going to the part of that URL that ends at <code>scheduler</code>; <code>http://precise64:8081/scheduler</code></p> + +<p>Once you click through to a role page, you see Jobs arranged +separately by pending jobs, active jobs and finished jobs. +Jobs are arranged by role, typically a service account for +production jobs and user accounts for test or development jobs.</p> + +<h3 id="getting-job-status">Getting Job Status</h3> +<pre class="highlight plaintext"><code>aurora job status <job_key> +</code></pre> + +<p>Returns the status of recent tasks associated with the +<code>job_key</code> specified Job in its supplied cluster. Typically this includes +a mix of active tasks (running or assigned) and inactive tasks +(successful, failed, and lost.)</p> + +<h3 id="opening-the-web-ui">Opening the Web UI</h3> + +<p>Use the Job’s web UI scheduler URL or the <code>aurora status</code> command to find out on which +machines individual tasks are scheduled. You can open the web UI via the +<code>open</code> command line command if invoked from your machine:</p> +<pre class="highlight plaintext"><code>aurora job open [<cluster>[/<role>[/<env>/<job_name>]]] +</code></pre> + +<p>If only the cluster is specified, it goes directly to that cluster’s +scheduler main page. If the role is specified, it goes to the top-level +role page. If the full job key is specified, it goes directly to the job +page where you can inspect individual tasks.</p> + +<h3 id="sshing-to-a-specific-task-machine">SSHing to a Specific Task Machine</h3> +<pre class="highlight plaintext"><code>aurora task ssh <job_key> <shard number> +</code></pre> + +<p>You can have the Aurora client ssh directly to the machine that has been +assigned a particular Job/shard number. This may be useful for quickly +diagnosing issues such as performance issues or abnormal behavior on a +particular machine.</p> + +<h3 id="templating-command-arguments">Templating Command Arguments</h3> +<pre class="highlight plaintext"><code>aurora task run [-e] [-t THREADS] <job_key> -- <<command-line>> +</code></pre> + +<p>Given a job specification, run the supplied command on all hosts and +return the output. You may use the standard Mustache templating rules:</p> + +<ul> +<li><code>{{thermos.ports[name]}}</code> substitutes the specific named port of the +task assigned to this machine</li> +<li><code>{{mesos.instance}}</code> substitutes the shard id of the job’s task +assigned to this machine</li> +<li><code>{{thermos.task_id}}</code> substitutes the task id of the job’s task +assigned to this machine</li> +</ul> + +<p>For example, the following type of pattern can be a powerful diagnostic +tool:</p> +<pre class="highlight plaintext"><code>aurora task run -t5 cluster1/tyg/devel/seizure -- \ + 'curl -s -m1 localhost:{{thermos.ports[http]}}/vars | grep uptime' +</code></pre> + +<p>By default, the command runs in the Task’s sandbox. The <code>-e</code> option can +run the command in the executor’s sandbox. This is mostly useful for +Aurora administrators.</p> + +<p>You can parallelize the runs by using the <code>-t</code> option.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.12.0/committers/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.12.0/committers/index.html?rev=1733548&view=auto ============================================================================== --- aurora/site/publish/documentation/0.12.0/committers/index.html (added) +++ aurora/site/publish/documentation/0.12.0/committers/index.html Fri Mar 4 02:43:01 2016 @@ -0,0 +1,183 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/committers/'" + value="0.12.0"> + <option value="0.12.0" + selected="selected"> + 0.12.0 + (latest) + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h2 id="setting-up-your-email-account">Setting up your email account</h2> + +<p>Once your Apache ID has been set up you can configure your account and add ssh keys and setup an +email forwarding address at</p> + +<p><a href="http://id.apache.org";>http://id.apache.org</a></p> + +<p>Additional instructions for setting up your new committer email can be found at</p> + +<p><a href="http://www.apache.org/dev/user-email.html";>http://www.apache.org/dev/user-email.html</a></p> + +<p>The recommended setup is to configure all services (mailing lists, JIRA, ReviewBoard) to send +emails to your @apache.org email address.</p> + +<h2 id="creating-a-gpg-key-for-releases">Creating a gpg key for releases</h2> + +<p>In order to create a release candidate you will need a gpg key published to an external key server +and that key will need to be added to our KEYS file as well.</p> + +<ol> +<li><p>Create a key:</p> +<pre class="highlight plaintext"><code> gpg --gen-key +</code></pre></li> +<li><p>Add your gpg key to the Apache Aurora KEYS file:</p> +<pre class="highlight plaintext"><code> git clone https://git-wip-us.apache.org/repos/asf/aurora.git + (gpg --list-sigs <KEY ID> && gpg --armor --export <KEY ID>) >> KEYS + git add KEYS && git commit -m "Adding gpg key for <APACHE ID>" + ./rbt post -o -g +</code></pre></li> +<li><p>Publish the key to an external key server:</p> +<pre class="highlight plaintext"><code> gpg --keyserver pgp.mit.edu --send-keys <KEY ID> +</code></pre></li> +<li><p>Update the changes to the KEYS file to the Apache Aurora svn dist locations listed below:</p> +<pre class="highlight plaintext"><code> https://dist.apache.org/repos/dist/dev/aurora/KEYS + https://dist.apache.org/repos/dist/release/aurora/KEYS +</code></pre></li> +<li><p>Add your key to git config for use with the release scripts:</p> +<pre class="highlight plaintext"><code> git config --global user.signingkey <KEY ID> +</code></pre></li> +</ol> + +<h2 id="creating-a-release">Creating a release</h2> + +<p>The following will guide you through the steps to create a release candidate, vote, and finally an +official Apache Aurora release. Before starting your gpg key should be in the KEYS file and you +must have access to commit to the dist.a.o repositories.</p> + +<ol> +<li><p>Ensure that all issues resolved for this release candidate are tagged with the correct Fix +Version in Jira, the changelog script will use this to generate the CHANGELOG in step #2.</p></li> +<li><p>Create a release candidate. This will automatically update the CHANGELOG and commit it, create a +branch and update the current version within the trunk. To create a minor version update and publish +it run</p> +<pre class="highlight plaintext"><code> ./build-support/release/release-candidate -l m -p +</code></pre></li> +<li><p>Update, if necessary, the draft email created from the <code>release-candidate</code> script in step #2 and +send the [VOTE] email to the dev@ mailing list. You can verify the release signature and checksums +by running</p> +<pre class="highlight plaintext"><code> ./build-support/release/verify-release-candidate +</code></pre></li> +<li><p>Wait for the vote to complete. If the vote fails close the vote by replying to the initial [VOTE] +email sent in step #3 by editing the subject to [RESULT][VOTE] … and noting the failure reason +(example <a href="http://markmail.org/message/d4d6xtvj7vgwi76f";>here</a>). Now address any issues and go back to +step #1 and run again, this time you will use the -r flag to increment the release candidate +version. This will automatically clean up the release candidate rc0 branch and source distribution.</p> +<pre class="highlight plaintext"><code> ./build-support/release/release-candidate -l m -r 1 -p +</code></pre></li> +<li><p>Once the vote has successfully passed create the release</p> +<pre class="highlight plaintext"><code> ./build-support/release/release +</code></pre></li> +<li><p>Update the draft email created fom the <code>release</code> script in step #5 to include the Apache ID’s for +all binding votes and send the [RESULT][VOTE] email to the dev@ mailing list.</p></li> +</ol> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html>
