Added: aurora/site/publish/documentation/0.14.0/operations/storage/index.html
URL:
http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.14.0/operations/storage/index.html?rev=1748470&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.14.0/operations/storage/index.html
(added)
+++ aurora/site/publish/documentation/0.14.0/operations/storage/index.html Tue
Jun 14 21:35:25 2016
@@ -0,0 +1,222 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";>
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type =
'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ?
'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300"
alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value +
'/operations/storage/'"
+ value="0.14.0">
+ <option value="0.14.0"
+ selected="selected">
+ 0.14.0
+ (latest)
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="aurora-scheduler-storage">Aurora Scheduler Storage</h1>
+
+<ul>
+<li><a href="#overview">Overview</a></li>
+<li><a href="#replicated-log-configuration">Replicated Log
Configuration</a></li>
+<li><a href="#replicated-log-configuration">Backup Configuration</a></li>
+<li><a href="#storage-semantics">Storage Semantics</a>
+
+<ul>
+<li><a href="#reads-writes-modifications">Reads, writes, modifications</a></li>
+<li><a href="#read-lifecycle">Read lifecycle</a></li>
+<li><a href="#write-lifecycle">Write lifecycle</a></li>
+<li><a href="#atomicity-consistency-and-isolation">Atomicity, consistency and
isolation</a></li>
+<li><a href="#population-on-restart">Population on restart</a></li>
+</ul></li>
+</ul>
+
+<h2 id="overview">Overview</h2>
+
+<p>Aurora scheduler maintains data that need to be persisted to survive
failovers and restarts.
+For example:</p>
+
+<ul>
+<li>Task configurations and scheduled task instances</li>
+<li>Job update configurations and update progress</li>
+<li>Production resource quotas</li>
+<li>Mesos resource offer host attributes</li>
+</ul>
+
+<p>Aurora solves its persistence needs by leveraging the Mesos implementation
of a Paxos replicated
+log <a href="https://ramcloud.stanford.edu/~ongaro/userstudy/paxos.pdf";>[1]</a>
+<a href="http://en.wikipedia.org/wiki/State_machine_replication";>[2]</a> with
a key-value
+<a href="https://github.com/google/leveldb";>LevelDB</a> storage as persistence
media.</p>
+
+<p>Conceptually, it can be represented by the following major components:</p>
+
+<ul>
+<li>Volatile storage: in-memory cache of all available data. Implemented via
in-memory
+<a href="http://www.h2database.com/html/main.html";>H2 Database</a> and
accessed via
+<a href="http://mybatis.github.io/mybatis-3/";>MyBatis</a>.</li>
+<li>Log manager: interface between Aurora storage and Mesos replicated log.
The default schema format
+is <a href="https://github.com/apache/thrift";>thrift</a>. Data is stored in
serialized binary form.</li>
+<li>Snapshot manager: all data is periodically persisted in Mesos replicated
log in a single snapshot.
+This helps establishing periodic recovery checkpoints and speeds up volatile
storage recovery on
+restart.</li>
+<li>Backup manager: as a precaution, snapshots are periodically written out
into backup files.
+This solves a <a href="../backup-restore/">disaster recovery problem</a>
+in case of a complete loss or corruption of Mesos log files.</li>
+</ul>
+
+<p><img alt="Storage hierarchy" src="../../images/storage_hierarchy.png" /></p>
+
+<h2 id="storage-semantics">Storage Semantics</h2>
+
+<p>Implementation details of the Aurora storage system. Understanding those
can sometimes be useful
+when investigating performance issues.</p>
+
+<h3 id="reads-writes-modifications">Reads, writes, modifications</h3>
+
+<p>All services in Aurora access data via a set of predefined store interfaces
(aka stores) logically
+grouped by the type of data they serve. Every interface defines a specific set
of operations allowed
+on the data thus abstracting out the storage access and the actual persistence
implementation. The
+latter is especially important in view of a general immutability of persisted
data. With the Mesos
+replicated log as the underlying persistence solution, data can be read and
written easily but not
+modified. All modifications are simulated by saving new versions of modified
objects. This feature
+and general performance considerations justify the existence of the volatile
in-memory store.</p>
+
+<h4 id="read-lifecycle">Read lifecycle</h4>
+
+<p>There are two types of reads available in Aurora: consistent and
weakly-consistent. The difference
+is explained <a href="#atomicity-consistency-and-isolation">below</a>.</p>
+
+<p>All reads are served from the volatile storage making reads generally cheap
storage operations
+from the performance standpoint. The majority of the volatile stores are
represented by the
+in-memory H2 database. This allows for rich schema definitions, queries and
relationships that
+key-value storage is unable to match.</p>
+
+<h4 id="write-lifecycle">Write lifecycle</h4>
+
+<p>Writes are more involved operations since in addition to updating the
volatile store data has to be
+appended to the replicated log. Data is not available for reads until fully
ack-ed by both
+replicated log and volatile storage.</p>
+
+<h3 id="atomicity-consistency-and-isolation">Atomicity, consistency and
isolation</h3>
+
+<p>Aurora uses <a
href="http://en.wikipedia.org/wiki/Write-ahead_logging";>write-ahead logging</a>
to ensure
+consistency between replicated and volatile storage. In Aurora, data is first
written into the
+replicated log and only then updated in the volatile store.</p>
+
+<p>Aurora storage uses read-write locks to serialize data mutations and
provide consistent view of the
+available data. The available <code>Storage</code> interface exposes 3 major
types of operations:
+* <code>consistentRead</code> - access is locked using reader’s lock and
provides consistent view on read
+* <code>weaklyConsistentRead</code> - access is lock-less. Delivers best
contention performance but may result
+in stale reads
+* <code>write</code> - access is fully serialized by using writer’s
lock. Operation success requires both
+volatile and replicated writes to succeed.</p>
+
+<p>The consistency of the volatile store is enforced via H2 transactional
isolation.</p>
+
+<h3 id="population-on-restart">Population on restart</h3>
+
+<p>Any time a scheduler restarts, it restores its volatile state from the most
recent position recorded
+in the replicated log by restoring the snapshot and replaying individual log
entries on top to fully
+recover the state up to the last write.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a
href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How
To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/";>License</a></li>
+ <li><a
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ <li><a href="http://www.apache.org/security/";>Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a
href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under
the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a
href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX
photo</a> displayed on the homepage is available under a <a
href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons
BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo
are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Propchange:
aurora/site/publish/documentation/0.14.0/operations/storage/index.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange:
aurora/site/publish/documentation/0.14.0/operations/storage/index.html
------------------------------------------------------------------------------
svn:keywords = Id
Propchange:
aurora/site/publish/documentation/0.14.0/operations/storage/index.html
------------------------------------------------------------------------------
svn:mime-type = text/html
Added:
aurora/site/publish/documentation/0.14.0/reference/client-cluster-configuration/index.html
URL:
http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.14.0/reference/client-cluster-configuration/index.html?rev=1748470&view=auto
==============================================================================
---
aurora/site/publish/documentation/0.14.0/reference/client-cluster-configuration/index.html
(added)
+++
aurora/site/publish/documentation/0.14.0/reference/client-cluster-configuration/index.html
Tue Jun 14 21:35:25 2016
@@ -0,0 +1,250 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";>
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type =
'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ?
'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300"
alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value +
'/reference/client-cluster-configuration/'"
+ value="0.14.0">
+ <option value="0.14.0"
+ selected="selected">
+ 0.14.0
+ (latest)
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="client-cluster-configuration">Client Cluster Configuration</h1>
+
+<p>A cluster configuration file is used by the Aurora client to describe the
Aurora clusters with
+which it can communicate. Ultimately this allows client users to reference
clusters with short names
+like us-east and eu.</p>
+
+<p>A cluster configuration is formatted as JSON. The simplest cluster
configuration is one that
+communicates with a single (non-leader-elected) scheduler. For example:</p>
+<pre class="highlight plaintext"><code>[{
+ "name": "example",
+ "scheduler_uri": "http://localhost:55555";,
+}]
+</code></pre>
+
+<p>A configuration for a leader-elected scheduler would contain something
like:</p>
+<pre class="highlight plaintext"><code>[{
+ "name": "example",
+ "zk": "192.168.33.7",
+ "scheduler_zk_path": "/aurora/scheduler"
+}]
+</code></pre>
+
+<p>The following properties may be set:</p>
+
+<table><thead>
+<tr>
+<th style="text-align: left"><strong>Property</strong></th>
+<th style="text-align: left"><strong>Type</strong></th>
+<th style="text-align: left"><strong>Description</strong></th>
+</tr>
+</thead><tbody>
+<tr>
+<td style="text-align: left"><strong>name</strong></td>
+<td style="text-align: left">String</td>
+<td style="text-align: left">Cluster name (Required)</td>
+</tr>
+<tr>
+<td style="text-align: left"><strong>slave_root</strong></td>
+<td style="text-align: left">String</td>
+<td style="text-align: left">Path to Mesos agent work dir (Required)</td>
+</tr>
+<tr>
+<td style="text-align: left"><strong>slave<em>run</em>directory</strong></td>
+<td style="text-align: left">String</td>
+<td style="text-align: left">Name of Mesos agent run dir (Required)</td>
+</tr>
+<tr>
+<td style="text-align: left"><strong>zk</strong></td>
+<td style="text-align: left">String</td>
+<td style="text-align: left">Hostname of ZooKeeper instance used to resolve
Aurora schedulers.</td>
+</tr>
+<tr>
+<td style="text-align: left"><strong>zk_port</strong></td>
+<td style="text-align: left">Integer</td>
+<td style="text-align: left">Port of ZooKeeper instance used to locate Aurora
schedulers (Default: 2181)</td>
+</tr>
+<tr>
+<td style="text-align: left"><strong>scheduler<em>zk</em>path</strong></td>
+<td style="text-align: left">String</td>
+<td style="text-align: left">ZooKeeper path under which scheduler instances
are registered.</td>
+</tr>
+<tr>
+<td style="text-align: left"><strong>scheduler_uri</strong></td>
+<td style="text-align: left">String</td>
+<td style="text-align: left">URI of Aurora scheduler instance.</td>
+</tr>
+<tr>
+<td style="text-align: left"><strong>proxy_url</strong></td>
+<td style="text-align: left">String</td>
+<td style="text-align: left">Used by the client to format URLs for
display.</td>
+</tr>
+<tr>
+<td style="text-align: left"><strong>auth_mechanism</strong></td>
+<td style="text-align: left">String</td>
+<td style="text-align: left">The authentication mechanism to use when
communicating with the scheduler. (Default: UNAUTHENTICATED)</td>
+</tr>
+</tbody></table>
+
+<h2 id="details">Details</h2>
+
+<h3 id="name"><code>name</code></h3>
+
+<p>The name of the Aurora cluster represented by this entry. This name will be
the <code>cluster</code> portion of
+any job keys identifying jobs running within the cluster.</p>
+
+<h3 id="slave_root"><code>slave_root</code></h3>
+
+<p>The path on the Mesos agents where executing tasks can be found. It is used
in combination with the
+<code>slave_run_directory</code> property by <code>aurora task run</code> and
<code>aurora task ssh</code> to change into the sandbox
+directory after connecting to the host. This value should match the value
passed to <code>mesos-slave</code>
+as <code>-work_dir</code>.</p>
+
+<h3 id="slave_run_directory"><code>slave_run_directory</code></h3>
+
+<p>The name of the directory where the task run can be found. This is used in
combination with the
+<code>slave_root</code> property by <code>aurora task run</code> and
<code>aurora task ssh</code> to change into the sandbox
+directory after connecting to the host. This should almost always be set to
<code>latest</code>.</p>
+
+<h3 id="zk"><code>zk</code></h3>
+
+<p>The hostname of the ZooKeeper instance used to resolve the Aurora
scheduler. Aurora uses ZooKeeper
+to elect a leader. The client will connect to this ZooKeeper instance to
determine the current
+leader. This host should match the host passed to the scheduler as
<code>-zk_endpoints</code>.</p>
+
+<h3 id="zk_port"><code>zk_port</code></h3>
+
+<p>The port on which the ZooKeeper instance is running. If not set this will
default to the standard
+ZooKeeper port of 2181. This port should match the port in the host passed to
the scheduler as
+<code>-zk_endpoints</code>.</p>
+
+<h3 id="scheduler_zk_path"><code>scheduler_zk_path</code></h3>
+
+<p>The path on the ZooKeeper instance under which the Aurora serverset is
registered. This value should
+match the value passed to the scheduler as <code>-serverset_path</code>.</p>
+
+<h3 id="scheduler_uri"><code>scheduler_uri</code></h3>
+
+<p>The URI of the scheduler. This would be used in place of the ZooKeeper
related configuration above
+in circumstances where direct communication with a single scheduler is needed
(e.g. testing
+environments). It is strongly advised to <strong>never</strong> use this
property for production deploys.</p>
+
+<h3 id="proxy_url"><code>proxy_url</code></h3>
+
+<p>Instead of using the hostname of the leading scheduler as the base url, if
<code>proxy_url</code> is set, its
+value will be used instead. In that scenario the value for
<code>proxy_url</code> would be, for example, the
+URL of your VIP in a loadbalancer or a roundrobin DNS name.</p>
+
+<h3 id="auth_mechanism"><code>auth_mechanism</code></h3>
+
+<p>The identifier of an authentication mechanism that the client should use
when communicating with the
+scheduler. Support for values other than <code>UNAUTHENTICATED</code> requires
a matching scheduler-side
+<a href="../../operations/security/">security configuration</a>.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a
href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How
To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/";>License</a></li>
+ <li><a
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ <li><a href="http://www.apache.org/security/";>Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a
href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under
the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a
href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX
photo</a> displayed on the homepage is available under a <a
href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons
BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo
are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Propchange:
aurora/site/publish/documentation/0.14.0/reference/client-cluster-configuration/index.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange:
aurora/site/publish/documentation/0.14.0/reference/client-cluster-configuration/index.html
------------------------------------------------------------------------------
svn:keywords = Id
Propchange:
aurora/site/publish/documentation/0.14.0/reference/client-cluster-configuration/index.html
------------------------------------------------------------------------------
svn:mime-type = text/html
Added:
aurora/site/publish/documentation/0.14.0/reference/client-commands/index.html
URL:
http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.14.0/reference/client-commands/index.html?rev=1748470&view=auto
==============================================================================
---
aurora/site/publish/documentation/0.14.0/reference/client-commands/index.html
(added)
+++
aurora/site/publish/documentation/0.14.0/reference/client-commands/index.html
Tue Jun 14 21:35:25 2016
@@ -0,0 +1,448 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";>
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type =
'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ?
'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300"
alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value +
'/reference/client-commands/'"
+ value="0.14.0">
+ <option value="0.14.0"
+ selected="selected">
+ 0.14.0
+ (latest)
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="aurora-client-commands">Aurora Client Commands</h1>
+
+<ul>
+<li><a href="#introduction">Introduction</a></li>
+<li><a href="#cluster-configuration">Cluster Configuration</a></li>
+<li><a href="#job-keys">Job Keys</a></li>
+<li><a href="#modifying-aurora-client-commands">Modifying Aurora Client
Commands</a></li>
+<li><a href="#regular-jobs">Regular Jobs</a>
+
+<ul>
+<li><a href="#creating-and-running-a-job">Creating and Running a Job</a></li>
+<li><a href="#running-a-command-on-a-running-job">Running a Command On a
Running Job</a></li>
+<li><a href="#killing-a-job">Killing a Job</a></li>
+<li><a href="#adding-instances">Adding Instances</a></li>
+<li><a href="#updating-a-job">Updating a Job</a>
+
+<ul>
+<li><a href="#coordinated-job-updates">Coordinated job updates</a></li>
+</ul></li>
+<li><a href="#renaming-a-job">Renaming a Job</a></li>
+<li><a href="#restarting-jobs">Restarting Jobs</a></li>
+</ul></li>
+<li><a href="#cron-jobs">Cron Jobs</a></li>
+<li><a href="#comparing-jobs">Comparing Jobs</a></li>
+<li><a href="#viewingexamining-jobs">Viewing/Examining Jobs</a>
+
+<ul>
+<li><a href="#listing-jobs">Listing Jobs</a></li>
+<li><a href="#inspecting-a-job">Inspecting a Job</a></li>
+<li><a href="#versions">Versions</a></li>
+<li><a href="#checking-your-quota">Checking Your Quota</a></li>
+<li><a href="#finding-a-job-on-web-ui">Finding a Job on Web UI</a></li>
+<li><a href="#getting-job-status">Getting Job Status</a></li>
+<li><a href="#opening-the-web-ui">Opening the Web UI</a></li>
+<li><a href="#sshing-to-a-specific-task-machine">SSHing to a Specific Task
Machine</a></li>
+<li><a href="#templating-command-arguments">Templating Command
Arguments</a></li>
+</ul></li>
+</ul>
+
+<h2 id="introduction">Introduction</h2>
+
+<p>Once you have written an <code>.aurora</code> configuration file that
describes
+your Job and its parameters and functionality, you interact with Aurora
+using Aurora Client commands. This document describes all of these commands
+and how and when to use them. All Aurora Client commands start with
+<code>aurora</code>, followed by the name of the specific command and its
+arguments.</p>
+
+<p><em>Job keys</em> are a very common argument to Aurora commands, as well as
the
+gateway to useful information about a Job. Before using Aurora, you
+should read the next section which describes them in detail. The section
+after that briefly describes how you can modify the behavior of certain
+Aurora Client commands, linking to a detailed document about how to do
+that.</p>
+
+<p>This is followed by the Regular Jobs section, which describes the basic
+Client commands for creating, running, and manipulating Aurora Jobs.
+After that are sections on Comparing Jobs and Viewing/Examining Jobs. In
+other words, various commands for getting information and metadata about
+Aurora Jobs.</p>
+
+<h2 id="cluster-configuration">Cluster Configuration</h2>
+
+<p>The client must be able to find a configuration file that specifies
available clusters. This file
+declares shorthand names for clusters, which are in turn referenced by job
configuration files
+and client commands.</p>
+
+<p>The client will load at most two configuration files, making both of their
defined clusters
+available. The first is intended to be a system-installed cluster, using the
path specified in
+the environment variable <code>AURORA_CONFIG_ROOT</code>, defaulting to
<code>/etc/aurora/clusters.json</code> if the
+environment variable is not set. The second is a user-installed file, located
at
+<code>~/.aurora/clusters.json</code>.</p>
+
+<p>For more details on cluster configuration see the
+<a href="../client-cluster-configuration/">Client Cluster Configuration</a>
documentation.</p>
+
+<h2 id="job-keys">Job Keys</h2>
+
+<p>A job key is a unique system-wide identifier for an Aurora-managed
+Job, for example <code>cluster1/web-team/test/experiment204</code>. It is a
4-tuple
+consisting of, in order, <em>cluster</em>, <em>role</em>,
<em>environment</em>, and
+<em>jobname</em>, separated by /s. Cluster is the name of an Aurora
+cluster. Role is the Unix service account under which the Job
+runs. Environment is a namespace component like <code>devel</code>,
<code>test</code>,
+<code>prod</code>, or <code>stagingN.</code> Jobname is the Job’s
name.</p>
+
+<p>The combination of all four values uniquely specifies the Job. If any
+one value is different from that of another job key, the two job keys
+refer to different Jobs. For example, job key
+<code>cluster1/tyg/prod/workhorse</code> is different from
+<code>cluster1/tyg/prod/workcamel</code> is different from
+<code>cluster2/tyg/prod/workhorse</code> is different from
+<code>cluster2/foo/prod/workhorse</code> is different from
+<code>cluster1/tyg/test/workhorse.</code></p>
+
+<p>Role names are user accounts existing on the agent machines. If you
don’t know what accounts
+are available, contact your sysadmin.</p>
+
+<p>Environment names are namespaces; you can count on <code>prod</code>,
<code>devel</code> and <code>test</code> existing.</p>
+
+<h2 id="modifying-aurora-client-commands">Modifying Aurora Client Commands</h2>
+
+<p>For certain Aurora Client commands, you can define hook methods that run
+either before or after an action that takes place during the command’s
+execution, as well as based on whether the action finished successfully or
failed
+during execution. Basically, a hook is code that lets you extend the
+command’s actions. The hook executes on the client side, specifically on
+the machine executing Aurora commands.</p>
+
+<p>Hooks can be associated with these Aurora Client commands.</p>
+
+<ul>
+<li><code>job create</code></li>
+<li><code>job kill</code></li>
+<li><code>job restart</code></li>
+</ul>
+
+<p>The process for writing and activating them is complex enough
+that we explain it in a devoted document, <a href="../client-hooks/">Hooks for
Aurora Client API</a>.</p>
+
+<h2 id="regular-jobs">Regular Jobs</h2>
+
+<p>This section covers Aurora commands related to running, killing,
+renaming, updating, and restarting a basic Aurora Job.</p>
+
+<h3 id="creating-and-running-a-job">Creating and Running a Job</h3>
+<pre class="highlight plaintext"><code>aurora job create <job key>
<configuration file>
+</code></pre>
+
+<p>Creates and then runs a Job with the specified job key based on a
<code>.aurora</code> configuration file.
+The configuration file may also contain and activate hook definitions.</p>
+
+<h3 id="running-a-command-on-a-running-job">Running a Command On a Running
Job</h3>
+<pre class="highlight plaintext"><code>aurora task run
CLUSTER/ROLE/ENV/NAME[/INSTANCES] <cmd>
+</code></pre>
+
+<p>Runs a shell command on all machines currently hosting shards of a
+single Job.</p>
+
+<p><code>run</code> supports the same command line wildcards used to populate
a Job’s
+commands; i.e. anything in the <code>{{mesos.*}}</code> and
<code>{{thermos.*}}</code>
+namespaces.</p>
+
+<h3 id="killing-a-job">Killing a Job</h3>
+<pre class="highlight plaintext"><code>aurora job killall CLUSTER/ROLE/ENV/NAME
+</code></pre>
+
+<p>Kills all Tasks associated with the specified Job, blocking until all
+are terminated. Defaults to killing all instances in the Job.</p>
+
+<p>The <code><configuration file></code> argument for <code>kill</code>
is optional. Use it only
+if it contains hook definitions and activations that affect the
+kill command.</p>
+
+<h3 id="adding-instances">Adding Instances</h3>
+<pre class="highlight plaintext"><code>aurora job add
CLUSTER/ROLE/ENV/NAME/INSTANCE <count>
+</code></pre>
+
+<p>Adds <code><count></code> instances to the existing job. The
configuration of the new instances is derived from
+an active job instance pointed by the <code>/INSTANCE</code> part of the job
specification. This command is
+a simpler way to scale out an existing job when an instance with desired task
configuration
+already exists. Use <code>aurora update start</code> to add instances with a
new (updated) configuration.</p>
+
+<h3 id="updating-a-job">Updating a Job</h3>
+
+<p>You can manage job updates using the <code>aurora update</code> command.
Please see
+<a href="../../features/job-updates/">the Job Update documentation</a> for
more details.</p>
+
+<h3 id="renaming-a-job">Renaming a Job</h3>
+
+<p>Renaming is a tricky operation as downstream clients must be informed of
+the new name. A conservative approach
+to renaming suitable for production services is:</p>
+
+<ol>
+<li> Modify the Aurora configuration file to change the role,
+environment, and/or name as appropriate to the standardized naming
+scheme.</li>
+<li><p>Check that only these naming components have changed
+with <code>aurora diff</code>.</p>
+<pre class="highlight plaintext"><code>aurora job diff CLUSTER/ROLE/ENV/NAME
<job_configuration>
+</code></pre></li>
+<li><p>Create the (identical) job at the new key. You may need to request a
+temporary quota increase.</p>
+<pre class="highlight plaintext"><code>aurora job create
CLUSTER/ROLE/ENV/NEW_NAME <job_configuration>
+</code></pre></li>
+<li><p>Migrate all clients over to the new job key. Update all links and
+dashboards. Ensure that both job keys run identical versions of the
+code while in this state.</p></li>
+<li><p>After verifying that all clients have successfully moved over, kill
+the old job.</p>
+<pre class="highlight plaintext"><code>aurora job killall CLUSTER/ROLE/ENV/NAME
+</code></pre></li>
+<li><p>If you received a temporary quota increase, be sure to let the
+powers that be know you no longer need the additional capacity.</p></li>
+</ol>
+
+<h3 id="restarting-jobs">Restarting Jobs</h3>
+
+<p><code>restart</code> restarts all of a job key identified Job’s
shards:</p>
+<pre class="highlight plaintext"><code>aurora job restart
CLUSTER/ROLE/ENV/NAME[/INSTANCES]
+</code></pre>
+
+<p>Restarts are controlled on the client side, so aborting
+the <code>job restart</code> command halts the restart operation.</p>
+
+<p><strong>Note</strong>: <code>job restart</code> only applies its command
line arguments and does not
+use or is affected by <code>update.config</code>. Restarting
+does <strong><em>not</em></strong> involve a configuration change. To update
the
+configuration, use <code>update.config</code>.</p>
+
+<p>The <code>--config</code> argument for restart is optional. Use it only
+if it contains hook definitions and activations that affect the
+<code>job restart</code> command.</p>
+
+<h2 id="cron-jobs">Cron Jobs</h2>
+
+<p>You can manage cron jobs using the <code>aurora cron</code> command.
Please see
+<a href="../../features/cron-jobs/">the Cron Jobs Feature</a> for more
details.</p>
+
+<h2 id="comparing-jobs">Comparing Jobs</h2>
+<pre class="highlight plaintext"><code>aurora job diff CLUSTER/ROLE/ENV/NAME
<job configuration>
+</code></pre>
+
+<p>Compares a job configuration against a running job. By default the diff
+is determined using <code>diff</code>, though you may choose an alternate
+ diff program by specifying the <code>DIFF_VIEWER</code> environment
variable.</p>
+
+<h2 id="viewing-examining-jobs">Viewing/Examining Jobs</h2>
+
+<p>Above we discussed creating, killing, and updating Jobs. Here we discuss
+how to view and examine Jobs.</p>
+
+<h3 id="listing-jobs">Listing Jobs</h3>
+<pre class="highlight plaintext"><code>aurora config list <job
configuration>
+</code></pre>
+
+<p>Lists all Jobs registered with the Aurora scheduler in the named cluster
for the named role.</p>
+
+<h3 id="inspecting-a-job">Inspecting a Job</h3>
+<pre class="highlight plaintext"><code>aurora job inspect
CLUSTER/ROLE/ENV/NAME <job configuration>
+</code></pre>
+
+<p><code>inspect</code> verifies that its specified job can be parsed from a
+configuration file, and displays the parsed configuration.</p>
+
+<h3 id="checking-your-quota">Checking Your Quota</h3>
+<pre class="highlight plaintext"><code>aurora quota get CLUSTER/ROLE
+</code></pre>
+
+<p>Prints the production quota allocated to the role’s value at the given
+cluster. Only non-<a
href="../../features/constraints/#dedicated-attribute">dedicated</a>
+<a href="../configuration/#job-objects">production</a> jobs consume quota.</p>
+
+<h3 id="finding-a-job-on-web-ui">Finding a Job on Web UI</h3>
+
+<p>When you create a job, part of the output response contains a URL that goes
+to the job’s scheduler UI page. For example:</p>
+<pre class="highlight plaintext"><code>vagrant@precise64:~$ aurora job create
devcluster/www-data/prod/hello /vagrant/examples/jobs/hello_world.aurora
+INFO] Creating job hello
+INFO] Response from scheduler: OK (message: 1 new tasks pending for job
www-data/prod/hello)
+INFO] Job url: http://precise64:8081/scheduler/www-data/prod/hello
+</code></pre>
+
+<p>You can go to the scheduler UI page for this job via
<code>http://precise64:8081/scheduler/www-data/prod/hello</code>
+You can go to the overall scheduler UI page by going to the part of that URL
that ends at <code>scheduler</code>;
<code>http://precise64:8081/scheduler</code></p>
+
+<p>Once you click through to a role page, you see Jobs arranged
+separately by pending jobs, active jobs and finished jobs.
+Jobs are arranged by role, typically a service account for
+production jobs and user accounts for test or development jobs.</p>
+
+<h3 id="getting-job-status">Getting Job Status</h3>
+<pre class="highlight plaintext"><code>aurora job status <job_key>
+</code></pre>
+
+<p>Returns the status of recent tasks associated with the
+<code>job_key</code> specified Job in its supplied cluster. Typically this
includes
+a mix of active tasks (running or assigned) and inactive tasks
+(successful, failed, and lost.)</p>
+
+<h3 id="opening-the-web-ui">Opening the Web UI</h3>
+
+<p>Use the Job’s web UI scheduler URL or the <code>aurora status</code>
command to find out on which
+machines individual tasks are scheduled. You can open the web UI via the
+<code>open</code> command line command if invoked from your machine:</p>
+<pre class="highlight plaintext"><code>aurora job open
[<cluster>[/<role>[/<env>/<job_name>]]]
+</code></pre>
+
+<p>If only the cluster is specified, it goes directly to that cluster’s
+scheduler main page. If the role is specified, it goes to the top-level
+role page. If the full job key is specified, it goes directly to the job
+page where you can inspect individual tasks.</p>
+
+<h3 id="sshing-to-a-specific-task-machine">SSHing to a Specific Task
Machine</h3>
+<pre class="highlight plaintext"><code>aurora task ssh <job_key>
<shard number>
+</code></pre>
+
+<p>You can have the Aurora client ssh directly to the machine that has been
+assigned a particular Job/shard number. This may be useful for quickly
+diagnosing issues such as performance issues or abnormal behavior on a
+particular machine.</p>
+
+<h3 id="templating-command-arguments">Templating Command Arguments</h3>
+<pre class="highlight plaintext"><code>aurora task run [-e] [-t THREADS]
<job_key> -- <<command-line>>
+</code></pre>
+
+<p>Given a job specification, run the supplied command on all hosts and
+return the output. You may use the standard Mustache templating rules:</p>
+
+<ul>
+<li><code>{{thermos.ports[name]}}</code> substitutes the specific named port
of the
+task assigned to this machine</li>
+<li><code>{{mesos.instance}}</code> substitutes the shard id of the
job’s task
+assigned to this machine</li>
+<li><code>{{thermos.task_id}}</code> substitutes the task id of the
job’s task
+assigned to this machine</li>
+</ul>
+
+<p>For example, the following type of pattern can be a powerful diagnostic
+tool:</p>
+<pre class="highlight plaintext"><code>aurora task run -t5
cluster1/tyg/devel/seizure -- \
+ 'curl -s -m1 localhost:{{thermos.ports[http]}}/vars | grep uptime'
+</code></pre>
+
+<p>By default, the command runs in the Task’s sandbox. The
<code>-e</code> option can
+run the command in the executor’s sandbox. This is mostly useful for
+Aurora administrators.</p>
+
+<p>You can parallelize the runs by using the <code>-t</code> option.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a
href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How
To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/";>License</a></li>
+ <li><a
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ <li><a href="http://www.apache.org/security/";>Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a
href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under
the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a
href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX
photo</a> displayed on the homepage is available under a <a
href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons
BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo
are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Propchange:
aurora/site/publish/documentation/0.14.0/reference/client-commands/index.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange:
aurora/site/publish/documentation/0.14.0/reference/client-commands/index.html
------------------------------------------------------------------------------
svn:keywords = Id
Propchange:
aurora/site/publish/documentation/0.14.0/reference/client-commands/index.html
------------------------------------------------------------------------------
svn:mime-type = text/html
Added:
aurora/site/publish/documentation/0.14.0/reference/client-hooks/index.html
URL:
http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.14.0/reference/client-hooks/index.html?rev=1748470&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.14.0/reference/client-hooks/index.html
(added)
+++ aurora/site/publish/documentation/0.14.0/reference/client-hooks/index.html
Tue Jun 14 21:35:25 2016
@@ -0,0 +1,379 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";>
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type =
'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ?
'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300"
alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value +
'/reference/client-hooks/'"
+ value="0.14.0">
+ <option value="0.14.0"
+ selected="selected">
+ 0.14.0
+ (latest)
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="hooks-for-aurora-client-api">Hooks for Aurora Client API</h1>
+
+<p>You can execute hook methods around Aurora API Client methods when they are
called by the Aurora Command Line commands.</p>
+
+<p>Explaining how hooks work is a bit tricky because of some indirection about
what they apply to. Basically, a hook is code that executes when a particular
Aurora Client API method runs, letting you extend the method’s actions.
The hook executes on the client side, specifically on the machine executing
Aurora commands.</p>
+
+<p>The catch is that hooks are associated with Aurora Client API methods,
which users don’t directly call. Instead, users call Aurora Command Line
commands, which call Client API methods during their execution. Since which
hooks run depend on which Client API methods get called, you will need to know
which Command Line commands call which API methods. Later on, there is a table
showing the various associations.</p>
+
+<p><strong>Terminology Note</strong>: From now on, “method(s)”
refer to Client API methods, and “command(s)” refer to Command Line
commands.</p>
+
+<ul>
+<li><a href="#hook-types">Hook Types</a></li>
+<li><a href="#execution-order">Execution Order</a></li>
+<li><a href="#hookable-methods">Hookable Methods</a></li>
+<li><a href="#activating-and-using-hooks">Activating and Using Hooks</a></li>
+<li><a href="#aurora-config-file-settings">.aurora Config File
Settings</a></li>
+<li><a href="#command-line">Command Line</a></li>
+<li><a href="#hooks-protocol">Hooks Protocol</a>
+
+<ul>
+<li><a href="#pre_-methods">pre_ Methods</a></li>
+<li><a href="#err_-methods">err_ Methods</a></li>
+<li><a href="#post_-methods">post_ Methods</a></li>
+</ul></li>
+<li><a href="#generic-hooks">Generic Hooks</a></li>
+<li><a href="#hooks-process-checklist">Hooks Process Checklist</a></li>
+</ul>
+
+<h2 id="hook-types">Hook Types</h2>
+
+<p>Hooks have three basic types, differing by when they run with respect to
their associated method.</p>
+
+<p><code>pre_<method_name></code>: When its associated method is called,
the <code>pre_</code> hook executes first, then the called method. If the
<code>pre_</code> hook fails, the method never runs. Later code that expected
the method to succeed may be affected by this, and result in terminating the
Aurora client.</p>
+
+<p>Note that a <code>pre_</code> hook can error-trap internally so it does not
+return <code>False</code>. Designers/contributors of new <code>pre_</code>
hooks should
+consider whether or not to error-trap them. You can error trap at the
+highest level very generally and always pass the <code>pre_</code> hook by
+returning <code>True</code>. For example:</p>
+<pre class="highlight plaintext"><code>def pre_create(...):
+ do_something() # if do_something fails with an exception, the create_job is
not attempted!
+ return True
+
+# However...
+def pre_create(...):
+ try:
+ do_something() # may cause exception
+ except Exception: # generic error trap will catch it
+ pass # and ignore the exception
+ return True # create_job will run in any case!
+</code></pre>
+
+<p><code>post_<method_name></code>: A <code>post_</code> hook executes
after its associated method successfully finishes running. If it fails, the
already executed method is unaffected. A <code>post_</code> hook’s error
is trapped, and any later operations are unaffected.</p>
+
+<p><code>err_<method_name></code>: Executes only when its associated
method returns a status other than OK or throws an exception. If an
<code>err_</code> hook fails, the already executed method is unaffected. An
<code>err_</code> hook’s error is trapped, and any later operations are
unaffected.</p>
+
+<h2 id="execution-order">Execution Order</h2>
+
+<p>A command with <code>pre_</code>, <code>post_</code>, and <code>err_</code>
hooks defined and activated for its called method executes in the following
order when the method successfully executes:</p>
+
+<ol>
+<li>Command called</li>
+<li>Command code executes</li>
+<li>Method Called</li>
+<li><code>pre_</code> method hook runs</li>
+<li>Method runs and successfully finishes</li>
+<li><code>post_</code> method hook runs</li>
+<li>Command code executes</li>
+<li>Command execution ends</li>
+</ol>
+
+<p>The following is what happens when, for the same command and hooks, the
method associated with the command suffers an error and does not successfully
finish executing:</p>
+
+<ol>
+<li>Command called</li>
+<li>Command code executes</li>
+<li>Method Called</li>
+<li><code>pre_</code> method hook runs</li>
+<li>Method runs and fails</li>
+<li><code>err_</code> method hook runs</li>
+<li>Command Code executes (if <code>err_</code> method does not end the
command execution)</li>
+<li>Command execution ends</li>
+</ol>
+
+<p>Note that the <code>post_</code> and <code>err_</code> hooks for the same
method can never both run for a single execution of that method.</p>
+
+<h2 id="hookable-methods">Hookable Methods</h2>
+
+<p>You can associate <code>pre_</code>, <code>post_</code>, and
<code>err_</code> hooks with the following methods. Since you do not directly
interact with the methods, but rather the Aurora Command Line commands that
call them, for each method we also list the command(s) that can call the
method. Note that a different method or methods may be called by a command
depending on how the command’s other code executes. Similarly, multiple
commands can call the same method. We also list the methods’ argument
signatures, which are used by their associated hooks. <a name="Chart"></a></p>
+
+<table><thead>
+<tr>
+<th>Aurora Client API Method</th>
+<th>Client API Method Argument Signature</th>
+<th>Aurora Command Line Command</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>create_job</code></td>
+<td><code>self</code>, <code>config</code></td>
+<td><code>job create</code>, <code>runtask</td>
+</tr>
+<tr>
+<td><code>restart</code></td>
+<td><code>self</code>, <code>job_key</code>, <code>shards</code>,
<code>update_config</code>, <code>health_check_interval_seconds</code></td>
+<td><code>job restart</code></td>
+</tr>
+<tr>
+<td><code>kill_job</code></td>
+<td><code>self</code>, <code>job_key</code>, <code>shards=None</code></td>
+<td><code>job kill</code></td>
+</tr>
+<tr>
+<td><code>start_cronjob</code></td>
+<td><code>self</code>, <code>job_key</code></td>
+<td><code>cron start</code></td>
+</tr>
+<tr>
+<td><code>start_job_update</code></td>
+<td><code>self</code>, <code>config</code>, <code>instances=None</code></td>
+<td><code>update start</code></td>
+</tr>
+</tbody></table>
+
+<p>Some specific examples:</p>
+
+<ul>
+<li><p><code>pre_create_job</code> executes when a <code>create_job</code>
method is called, and before the <code>create_job</code> method itself
executes.</p></li>
+<li><p><code>post_cancel_update</code> executes after a
<code>cancel_update</code> method has successfully finished running.</p></li>
+<li><p><code>err_kill_job</code> executes when the <code>kill_job</code>
method is called, but doesn’t successfully finish running.</p></li>
+</ul>
+
+<h2 id="activating-and-using-hooks">Activating and Using Hooks</h2>
+
+<p>By default, hooks are inactive. If you do not want to use hooks, you do not
need to make any changes to your code. If you do want to use hooks, you will
need to alter your <code>.aurora</code> config file to activate them both for
the configuration as a whole as well as for individual <code>Job</code>s. And,
of course, you will need to define in your config file what happens when a
particular hook executes.</p>
+
+<h2 id="aurora-config-file-settings">.aurora Config File Settings</h2>
+
+<p>You can define a top-level <code>hooks</code> variable in any
<code>.aurora</code> config file. <code>hooks</code> is a list of all objects
that define hooks used by <code>Job</code>s defined in that config file. If you
do not want to define any hooks for a configuration, <code>hooks</code> is
optional.</p>
+<pre class="highlight plaintext"><code>hooks = [Object_with_defined_hooks1,
Object_with_defined_hooks2]
+</code></pre>
+
+<p>Be careful when assembling a config file using <code>include</code> on
multiple smaller config files. If there are multiple files that assign a value
to <code>hooks</code>, only the last assignment made will stick. For example,
if <code>x.aurora</code> has <code>hooks = [a, b, c]</code> and
<code>y.aurora</code> has <code>hooks = [d, e, f]</code> and
<code>z.aurora</code> has, in this order, <code>include x.aurora</code> and
<code>include y.aurora</code>, the <code>hooks</code> value will be <code>[d,
e, f]</code>.</p>
+
+<p>Also, for any <code>Job</code> that you want to use hooks with, its
<code>Job</code> definition in the <code>.aurora</code> config file must set an
<code>enable_hooks</code> flag to <code>True</code> (it defaults to
<code>False</code>). By default, hooks are disabled and you must enable them
for <code>Job</code>s of your choice.</p>
+
+<p>To summarize, to use hooks for a particular job, you must both activate
hooks for your config file as a whole, and for that job. Activating hooks only
for individual jobs won’t work, nor will only activating hooks for your
config file as a whole. You must also specify the hooks’ defining object
in the <code>hooks</code> variable.</p>
+
+<p>Recall that <code>.aurora</code> config files are written in Pystachio. So
the following turns on hooks for production jobs at cluster1 and cluster2, but
leaves them off for similar jobs with a defined user role. Of course, you also
need to list the objects that define the hooks in your config file’s
<code>hooks</code> variable.</p>
+<pre class="highlight plaintext"><code>jobs = [
+ Job(enable_hooks = True, cluster = c, env = 'prod') for c in
('cluster1', 'cluster2')
+ ]
+jobs.extend(
+ Job(cluster = c, env = 'prod', role = getpass.getuser()) for c in
('cluster1', 'cluster2'))
+ # Hooks disabled for these jobs
+</code></pre>
+
+<h2 id="command-line">Command Line</h2>
+
+<p>All Aurora Command Line commands now accept an <code>.aurora</code> config
file as an optional parameter (some, of course, accept it as a required
parameter). Whenever a command has a <code>.aurora</code> file parameter, any
hooks specified and activated in the <code>.aurora</code> file can be used. For
example:</p>
+<pre class="highlight plaintext"><code>aurora job restart
cluster1/role/env/app myapp.aurora
+</code></pre>
+
+<p>The command activates any hooks specified and activated in
<code>myapp.aurora</code>. For the <code>restart</code> command, that is the
only thing the <code>myapp.aurora</code> parameter does. So, if the command was
the following, since there is no <code>.aurora</code> config file to specify
any hooks, no hooks on the <code>restart</code> command can run.</p>
+<pre class="highlight plaintext"><code>aurora job restart cluster1/role/env/app
+</code></pre>
+
+<h2 id="hooks-protocol">Hooks Protocol</h2>
+
+<p>Any object defined in the <code>.aurora</code> config file can define hook
methods. You should define your hook methods within a class, and then use the
class name as a value in the <code>hooks</code> list in your config file.</p>
+
+<p>Note that you can define other methods in the class that its hook methods
can call; all the logic of a hook does not have to be in its definition.</p>
+
+<p>The following example defines a class containing a
<code>pre_kill_job</code> hook definition that calls another method defined in
the class.</p>
+<pre class="highlight plaintext"><code># Defines a method pre_kill_job
+class KillConfirmer(object):
+ def confirm(self, msg):
+ return raw_input(msg).lower() == 'yes'
+
+ def pre_kill_job(self, job_key, shards=None):
+ shards = ('shards %s' % shards) if shards is not None else 'all shards'
+ return self.confirm('Are you sure you want to kill %s (%s)? (yes/no): '
+ % (job_key, shards))
+</code></pre>
+
+<h3 id="pre_-methods">pre_ Methods</h3>
+
+<p><code>pre_</code> methods have the signature:</p>
+<pre class="highlight plaintext"><code>pre_<API method name>(self,
<associated method's signature>)
+</code></pre>
+
+<p><code>pre_</code> methods have the same signature as their associated
method, with the addition of <code>self</code> as the first parameter. See the
<a href="#Chart">chart</a> above for the mapping of parameters to methods. When
writing <code>pre_</code> methods, you can use the <code>*</code> and
<code>**</code> syntax to designate that all unspecified parameters are passed
in a list to the <code>*</code>ed variable and all named parameters with values
are passed as name/value pairs to the <code>**</code>ed variable.</p>
+
+<p>If this method returns False, the API command call aborts.</p>
+
+<h3 id="err_-methods">err_ Methods</h3>
+
+<p><code>err_</code> methods have the signature:</p>
+<pre class="highlight plaintext"><code>err_<API method name>(self, exc,
<associated method's signature>)
+</code></pre>
+
+<p><code>err_</code> methods have the same signature as their associated
method, with the addition of a first parameter <code>self</code> and a second
parameter <code>exc</code>. <code>exc</code> is either a result with
responseCode other than <code>ResponseCode.OK</code> or an
<code>Exception</code>. See the <a href="#Chart">chart</a> above for the
mapping of parameters to methods. When writing <code>err</code>_ methods, you
can use the <code>*</code> and <code>**</code> syntax to designate that all
unspecified parameters are passed in a list to the <code>*</code>ed variable
and all named parameters with values are passed as name/value pairs to the
<code>**</code>ed variable.</p>
+
+<p><code>err_</code> method return codes are ignored.</p>
+
+<h3 id="post_-methods">post_ Methods</h3>
+
+<p><code>post_</code> methods have the signature:</p>
+<pre class="highlight plaintext"><code>post_<API method name>(self,
result, <associated method signature>)
+</code></pre>
+
+<p><code>post_</code> method parameters are <code>self</code>, then
<code>result</code>, followed by the same parameter signature as their
associated method. <code>result</code> is the result of the associated method
call. See the <a href="#chart">chart</a> above for the mapping of parameters to
methods. When writing <code>post_</code> methods, you can use the
<code>*</code> and <code>**</code> syntax to designate that all unspecified
arguments are passed in a list to the <code>*</code>ed parameter and all
unspecified named arguments with values are passed as name/value pairs to the
<code>**</code>ed parameter.</p>
+
+<p><code>post_</code> method return codes are ignored.</p>
+
+<h2 id="generic-hooks">Generic Hooks</h2>
+
+<p>There are seven Aurora API Methods which any of the three hook types can
attach to. Thus, there are 21 possible hook/method combinations for a single
<code>.aurora</code> config file. Say that you define <code>pre_</code> and
<code>post_</code> hooks for the <code>restart</code> method. That leaves 19
undefined hook/method combinations; <code>err_restart</code> and the 3
<code>pre_</code>, <code>post_</code>, and <code>err_</code> hooks for each of
the other 6 hookable methods. You can define what happens when any of these
otherwise undefined 19 hooks execute via a generic hook, whose signature is:</p>
+<pre class="highlight plaintext"><code>generic_hook(self, hook_config, event,
method_name, result_or_err, args*, kw**)
+</code></pre>
+
+<p>where:</p>
+
+<ul>
+<li><p><code>hook_config</code> is a named tuple of <code>config</code> (the
Pystashio <code>config</code> object) and <code>job_key</code>.</p></li>
+<li><p><code>event</code> is one of <code>pre</code>, <code>err</code>, or
<code>post</code>, indicating which type of hook the genetic hook is standing
in for. For example, assume no specific hooks were defined for the
<code>restart</code> API command. If <code>generic_hook</code> is defined and
activated, and <code>restart</code> is called, <code>generic_hook</code> will
effectively run as <code>pre_restart</code>, <code>post_restart</code>, and
<code>err_restart</code>. You can use a selection statement on this value so
that <code>generic_hook</code> will act differently based on whether it is
standing in for a <code>pre_</code>, <code>post_</code>, or <code>err_</code>
hook.</p></li>
+<li><p><code>method_name</code> is the Client API method name whose execution
is causing this execution of the <code>generic_hook</code>.</p></li>
+<li><p><code>args*</code>, <code>kw**</code> are the API method arguments and
keyword arguments respectively.</p></li>
+<li><p><code>result_or_err</code> is a tri-state parameter taking one of these
three values:</p>
+
+<ol>
+<li>None for <code>pre_</code>hooks</li>
+<li><code>result</code> for <code>post_</code> nooks</li>
+<li><code>exc</code> for <code>err_</code> hooks</li>
+</ol></li>
+</ul>
+
+<p>Example:</p>
+<pre class="highlight plaintext"><code># Overrides the standard do-nothing
generic_hook by adding a log writing operation.
+from twitter.common import log
+ class Logger(object):
+ '''Adds to the log every time a hookable API method is called'''
+ def generic_hook(self, hook_config, event, method_name, result_or_err,
*args, **kw)
+ log.info('%s: %s_%s of %s'
+ % (self.__class__.__name__, event, method_name,
hook_config.job_key))
+</code></pre>
+
+<h2 id="hooks-process-checklist">Hooks Process Checklist</h2>
+
+<ol>
+<li><p>In your <code>.aurora</code> config file, add a <code>hooks</code>
variable. Note that you may want to define a <code>.aurora</code> file only for
hook definitions and then include this file in multiple other config files that
you want to use the same hooks.</p>
+
+<p>hooks = []</p></li>
+<li><p>In the <code>hooks</code> variable, list all objects that define hooks
used by <code>Job</code>s defined in this config:</p>
+
+<p>hooks = [Object<em>hook</em>definer1, Object<em>hook</em>definer2]</p></li>
+<li><p>For each job that uses hooks in this config file, add
<code>enable_hooks = True</code> to the <code>Job</code> definition. Note that
this is necessary even if you only want to use the generic hook.</p></li>
+<li><p>Write your <code>pre_</code>, <code>post_</code>, and <code>err_</code>
hook definitions as part of an object definition in your <code>.aurora</code>
config file.</p></li>
+<li><p>If desired, write your <code>generic_hook</code> definition as part of
an object definition in your <code>.aurora</code> config file. Remember, the
object must be listed as a member of <code>hooks</code>.</p></li>
+<li><p>If your Aurora command line command does not otherwise take an
<code>.aurora</code> config file argument, add the appropriate
<code>.aurora</code> file as an argument in order to define and activate the
configuration’s hooks.</p></li>
+</ol>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a
href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How
To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/";>License</a></li>
+ <li><a
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ <li><a href="http://www.apache.org/security/";>Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a
href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under
the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a
href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX
photo</a> displayed on the homepage is available under a <a
href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons
BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo
are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Propchange:
aurora/site/publish/documentation/0.14.0/reference/client-hooks/index.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange:
aurora/site/publish/documentation/0.14.0/reference/client-hooks/index.html
------------------------------------------------------------------------------
svn:keywords = Id
Propchange:
aurora/site/publish/documentation/0.14.0/reference/client-hooks/index.html
------------------------------------------------------------------------------
svn:mime-type = text/html
Added:
aurora/site/publish/documentation/0.14.0/reference/configuration-best-practices/index.html
URL:
http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.14.0/reference/configuration-best-practices/index.html?rev=1748470&view=auto
==============================================================================
---
aurora/site/publish/documentation/0.14.0/reference/configuration-best-practices/index.html
(added)
+++
aurora/site/publish/documentation/0.14.0/reference/configuration-best-practices/index.html
Tue Jun 14 21:35:25 2016
@@ -0,0 +1,297 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";>
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type =
'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ?
'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300"
alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value +
'/reference/configuration-best-practices/'"
+ value="0.14.0">
+ <option value="0.14.0"
+ selected="selected">
+ 0.14.0
+ (latest)
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="aurora-configuration-best-practices">Aurora Configuration Best
Practices</h1>
+
+<h2 id="use-as-few-aurora-files-as-possible">Use As Few .aurora Files As
Possible</h2>
+
+<p>When creating your <code>.aurora</code> configuration, try to keep all
versions of
+a particular job within the same <code>.aurora</code> file. For example, if you
+have separate jobs for <code>cluster1</code>, <code>cluster1</code> staging,
<code>cluster1</code>
+testing, and<code>cluster2</code>, keep them as close together as possible.</p>
+
+<p>Constructs shared across multiple jobs owned by your team (e.g.
+team-level defaults or structural templates) can be split into separate
+<code>.aurora</code>files and included via the <code>include</code>
directive.</p>
+
+<h2 id="avoid-boilerplate">Avoid Boilerplate</h2>
+
+<p>If you see repetition or find yourself copy and pasting any parts of
+your configuration, it’s likely an opportunity for templating. Take the
+example below:</p>
+
+<p><code>redundant.aurora</code> contains:</p>
+<pre class="highlight plaintext"><code>download = Process(
+ name = 'download',
+ cmdline = 'wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2',
+ max_failures = 5,
+ min_duration = 1)
+
+unpack = Process(
+ name = 'unpack',
+ cmdline = 'rm -rf Python-2.7.3 && tar xzf Python-2.7.3.tar.bz2',
+ max_failures = 5,
+ min_duration = 1)
+
+build = Process(
+ name = 'build',
+ cmdline = 'pushd Python-2.7.3 && ./configure && make
&& popd',
+ max_failures = 1)
+
+email = Process(
+ name = 'email',
+ cmdline = 'echo Success | mail [email protected]',
+ max_failures = 5,
+ min_duration = 1)
+
+build_python = Task(
+ name = 'build_python',
+ processes = [download, unpack, build, email],
+ constraints = [Constraint(order = ['download', 'unpack', 'build', 'email'])])
+</code></pre>
+
+<p>As you’ll notice, there’s a lot of repetition in the
<code>Process</code>
+definitions. For example, almost every process sets a <code>max_failures</code>
+limit to 5 and a <code>min_duration</code> to 1. This is an opportunity for
factoring
+into a common process template.</p>
+
+<p>Furthermore, the Python version is repeated everywhere. This can be
+bound via structural templating as described in the <a
href="../configuration-templating/#AdvancedBinding">Advanced Binding</a>
+section.</p>
+
+<p><code>less_redundant.aurora</code> contains:</p>
+<pre class="highlight plaintext"><code>class Python(Struct):
+ version = Required(String)
+ base = Default(String, 'Python-{{version}}')
+ package = Default(String, '{{base}}.tar.bz2')
+
+ReliableProcess = Process(
+ max_failures = 5,
+ min_duration = 1)
+
+download = ReliableProcess(
+ name = 'download',
+ cmdline = 'wget
http://www.python.org/ftp/python/{{python.version}}/{{python.package}}')
+
+unpack = ReliableProcess(
+ name = 'unpack',
+ cmdline = 'rm -rf {{python.base}} && tar xzf {{python.package}}')
+
+build = ReliableProcess(
+ name = 'build',
+ cmdline = 'pushd {{python.base}} && ./configure && make
&& popd',
+ max_failures = 1)
+
+email = ReliableProcess(
+ name = 'email',
+ cmdline = 'echo Success | mail {{role}}@foocorp.com')
+
+build_python = SequentialTask(
+ name = 'build_python',
+ processes = [download, unpack, build, email]).bind(python = Python(version =
"2.7.3"))
+</code></pre>
+
+<h2 id="thermos-uses-bash-but-thermos-is-not-bash">Thermos Uses bash, But
Thermos Is Not bash</h2>
+
+<h4 id="bad">Bad</h4>
+
+<p>Many tiny Processes makes for harder to manage configurations.</p>
+<pre class="highlight plaintext"><code>copy = Process(
+ name = 'copy',
+ cmdline = 'rcp user@my_machine:my_application .'
+ )
+
+ unpack = Process(
+ name = 'unpack',
+ cmdline = 'unzip app.zip'
+ )
+
+ remove = Process(
+ name = 'remove',
+ cmdline = 'rm -f app.zip'
+ )
+
+ run = Process(
+ name = 'app',
+ cmdline = 'java -jar app.jar'
+ )
+
+ run_task = Task(
+ processes = [copy, unpack, remove, run],
+ constraints = order(copy, unpack, remove, run)
+ )
+</code></pre>
+
+<h4 id="good">Good</h4>
+
+<p>Each <code>cmdline</code> runs in a bash subshell, so you have the full
power of
+bash. Chaining commands with <code>&&</code> or <code>||</code> is
almost always the right
+thing to do.</p>
+
+<p>Also for Tasks that are simply a list of processes that run one after
+another, consider using the <code>SequentialTask</code> helper which applies a
+linear ordering constraint for you.</p>
+<pre class="highlight plaintext"><code>stage = Process(
+ name = 'stage',
+ cmdline = 'rcp user@my_machine:my_application . && unzip app.zip
&& rm -f app.zip')
+
+run = Process(name = 'app', cmdline = 'java -jar app.jar')
+
+run_task = SequentialTask(processes = [stage, run])
+</code></pre>
+
+<h2 id="rarely-use-functions-in-your-configurations">Rarely Use Functions In
Your Configurations</h2>
+
+<p>90% of the time you define a function in a <code>.aurora</code> file,
you’re
+probably Doing It Wrong™.</p>
+
+<h4 id="bad">Bad</h4>
+<pre class="highlight plaintext"><code>def get_my_task(name, user, cpu, ram,
disk):
+ return Task(
+ name = name,
+ user = user,
+ processes = [STAGE_PROCESS, RUN_PROCESS],
+ constraints = order(STAGE_PROCESS, RUN_PROCESS),
+ resources = Resources(cpu = cpu, ram = ram, disk = disk)
+ )
+
+ task_one = get_my_task('task_one', 'feynman', 1.0, 32*MB, 1*GB)
+ task_two = get_my_task('task_two', 'feynman', 2.0, 64*MB, 1*GB)
+</code></pre>
+
+<h4 id="good">Good</h4>
+
+<p>This one is more idiomatic. Forced keyword arguments prevents accidents,
+e.g. constructing a task with “32*MB” when you mean 32MB of ram
and not
+disk. Less proliferation of task-construction techniques means
+easier-to-read, quicker-to-understand, and a more composable
+configuration.</p>
+<pre class="highlight plaintext"><code>TASK_TEMPLATE = SequentialTask(
+ user = 'wickman',
+ processes = [STAGE_PROCESS, RUN_PROCESS],
+)
+
+task_one = TASK_TEMPLATE(
+ name = 'task_one',
+ resources = Resources(cpu = 1.0, ram = 32*MB, disk = 1*GB) )
+
+task_two = TASK_TEMPLATE(
+ name = 'task_two',
+ resources = Resources(cpu = 2.0, ram = 64*MB, disk = 1*GB)
+)
+</code></pre>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a
href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How
To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/";>License</a></li>
+ <li><a
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ <li><a href="http://www.apache.org/security/";>Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a
href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under
the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a
href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX
photo</a> displayed on the homepage is available under a <a
href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons
BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo
are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Propchange:
aurora/site/publish/documentation/0.14.0/reference/configuration-best-practices/index.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange:
aurora/site/publish/documentation/0.14.0/reference/configuration-best-practices/index.html
------------------------------------------------------------------------------
svn:keywords = Id
Propchange:
aurora/site/publish/documentation/0.14.0/reference/configuration-best-practices/index.html
------------------------------------------------------------------------------
svn:mime-type = text/html
