Added:
aurora/site/publish/documentation/0.14.0/features/resource-isolation/index.html
URL:
http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.14.0/features/resource-isolation/index.html?rev=1748470&view=auto
==============================================================================
---
aurora/site/publish/documentation/0.14.0/features/resource-isolation/index.html
(added)
+++
aurora/site/publish/documentation/0.14.0/features/resource-isolation/index.html
Tue Jun 14 21:35:25 2016
@@ -0,0 +1,294 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";>
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type =
'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ?
'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300"
alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value +
'/features/resource-isolation/'"
+ value="0.14.0">
+ <option value="0.14.0"
+ selected="selected">
+ 0.14.0
+ (latest)
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="resources-isolation-and-sizing">Resources Isolation and Sizing</h1>
+
+<ul>
+<li><a href="#isolation">Isolation</a></li>
+<li><a href="#sizing">Sizing</a></li>
+<li><a href="#oversubscription">Oversubscription</a></li>
+</ul>
+
+<h2 id="isolation">Isolation</h2>
+
+<p>Aurora is a multi-tenant system; a single software instance runs on a
+server, serving multiple clients/tenants. To share resources among
+tenants, it implements isolation of:</p>
+
+<ul>
+<li>CPU</li>
+<li>memory</li>
+<li>disk space</li>
+</ul>
+
+<p>CPU is a soft limit, and handled differently from memory and disk space.
+Too low a CPU value results in throttling your application and
+slowing it down. Memory and disk space are both hard limits; when your
+application goes over these values, it’s killed.</p>
+
+<h3 id="cpu-isolation">CPU Isolation</h3>
+
+<p>Mesos uses a quota based CPU scheduler (the <em>Completely Fair
Scheduler</em>)
+to provide consistent and predictable performance. This is effectively
+a guarantee of resources – you receive at least what you requested, but
+also no more than you’ve requested.</p>
+
+<p>The scheduler gives applications a CPU quota for every 100 ms interval.
+When an application uses its quota for an interval, it is throttled for
+the rest of the 100 ms. Usage resets for each interval and unused
+quota does not carry over.</p>
+
+<p>For example, an application specifying 4.0 CPU has access to 400 ms of
+CPU time every 100 ms. This CPU quota can be used in different ways,
+depending on the application and available resources. Consider the
+scenarios shown in this diagram.</p>
+
+<p><img alt="CPU Availability" src="../../images/CPUavailability.png" /></p>
+
+<ul>
+<li><p><em>Scenario A</em>: the application can use up to 4 cores continuously
for
+every 100 ms interval. It is never throttled and starts processing
+new requests immediately.</p></li>
+<li><p><em>Scenario B</em> : the application uses up to 8 cores (depending on
+availability) but is throttled after 50 ms. The CPU quota resets at the
+start of each new 100 ms interval.</p></li>
+<li><p><em>Scenario C</em> : is like Scenario A, but there is a garbage
collection
+event in the second interval that consumes all CPU quota. The
+application throttles for the remaining 75 ms of that interval and
+cannot service requests until the next interval. In this example, the
+garbage collection finished in one interval but, depending on how much
+garbage needs collecting, it may take more than one interval and further
+delay service of requests.</p></li>
+</ul>
+
+<p><em>Technical Note</em>: Mesos considers logical cores, also known as
+hyperthreading or SMT cores, as the unit of CPU.</p>
+
+<h3 id="memory-isolation">Memory Isolation</h3>
+
+<p>Mesos uses dedicated memory allocation. Your application always has
+access to the amount of memory specified in your configuration. The
+application’s memory use is defined as the sum of the resident set size
+(RSS) of all processes in a shard. Each shard is considered
+independently.</p>
+
+<p>In other words, say you specified a memory size of 10GB. Each shard
+would receive 10GB of memory. If an individual shard’s memory demands
+exceed 10GB, that shard is killed, but the other shards continue
+working.</p>
+
+<p><em>Technical note</em>: Total memory size is not enforced at allocation
time,
+so your application can request more than its allocation without getting
+an ENOMEM. However, it will be killed shortly after.</p>
+
+<h3 id="disk-space">Disk Space</h3>
+
+<p>Disk space used by your application is defined as the sum of the
files’
+disk space in your application’s directory, including the
<code>stdout</code> and
+<code>stderr</code> logged from your application. Each shard is considered
+independently. You should use off-node storage for your application’s
+data whenever possible.</p>
+
+<p>In other words, say you specified disk space size of 100MB. Each shard
+would receive 100MB of disk space. If an individual shard’s disk space
+demands exceed 100MB, that shard is killed, but the other shards
+continue working.</p>
+
+<p>After your application finishes running, its allocated disk space is
+reclaimed. Thus, your job’s final action should move any disk content
+that you want to keep, such as logs, to your home file system or other
+less transitory storage. Disk reclamation takes place an undefined
+period after the application finish time; until then, the disk contents
+are still available but you shouldn’t count on them being so.</p>
+
+<p><em>Technical note</em> : Disk space is not enforced at write so your
+application can write above its quota without getting an ENOSPC, but it
+will be killed shortly after. This is subject to change.</p>
+
+<h3 id="gpu-isolation">GPU Isolation</h3>
+
+<p>GPU isolation will be supported for Nvidia devices starting from Mesos
0.29.0.
+Access to the allocated units will be exclusive with no sharing between tasks
+allowed (e.g. no fractional GPU allocation). Until official documentation is
released,
+see <a
href="https://docs.google.com/document/d/10GJ1A80x4nIEo8kfdeo9B11PIbS1xJrrB4Z373Ifkpo/edit#heading=h.w84lz7p4eexl";>Mesos
design document</a>
+for more details.</p>
+
+<h3 id="other-resources">Other Resources</h3>
+
+<p>Other resources, such as network bandwidth, do not have any performance
+guarantees. For some resources, such as memory bandwidth, there are no
+practical sharing methods so some application combinations collocated on
+the same host may cause contention.</p>
+
+<h2 id="sizing">Sizing</h2>
+
+<h3 id="cpu-sizing">CPU Sizing</h3>
+
+<p>To correctly size Aurora-run Mesos tasks, specify a per-shard CPU value
+that lets the task run at its desired performance when at peak load
+distributed across all shards. Include reserve capacity of at least 50%,
+possibly more, depending on how critical your service is (or how
+confident you are about your original estimate : -)), ideally by
+increasing the number of shards to also improve resiliency. When running
+your application, observe its CPU stats over time. If consistently at or
+near your quota during peak load, you should consider increasing either
+per-shard CPU or the number of shards.</p>
+
+<h2 id="memory-sizing">Memory Sizing</h2>
+
+<p>Size for your application’s peak requirement. Observe the per-instance
+memory statistics over time, as memory requirements can vary over
+different periods. Remember that if your application exceeds its memory
+value, it will be killed, so you should also add a safety margin of
+around 10-20%. If you have the ability to do so, you may also want to
+put alerts on the per-instance memory.</p>
+
+<h2 id="disk-space-sizing">Disk Space Sizing</h2>
+
+<p>Size for your application’s peak requirement. Rotate and discard log
+files as needed to stay within your quota. When running a Java process,
+add the maximum size of the Java heap to your disk space requirement, in
+order to account for an out of memory error dumping the heap
+into the application’s sandbox space.</p>
+
+<h2 id="gpu-sizing">GPU Sizing</h2>
+
+<p>GPU is highly dependent on your application requirements and is only limited
+by the number of physical GPU units available on a target box.</p>
+
+<h2 id="oversubscription">Oversubscription</h2>
+
+<p><strong>WARNING</strong>: This feature is currently in alpha status. Do not
use it in production clusters!</p>
+
+<p>Mesos <a
href="http://mesos.apache.org/documentation/latest/oversubscription/";>supports
a concept of revocable tasks</a>
+by oversubscribing machine resources by the amount deemed safe to not affect
the existing
+non-revocable tasks. Aurora now supports revocable jobs via a
<code>tier</code> setting set to <code>revocable</code>
+value.</p>
+
+<p>The Aurora scheduler must be configured to receive revocable offers from
Mesos and accept revocable
+jobs. If not configured properly revocable tasks will never get assigned to
hosts and will stay in
+<code>PENDING</code>. Set these scheduler flag to allow receiving revocable
Mesos offers:</p>
+<pre class="highlight plaintext"><code>-receive_revocable_resources=true
+</code></pre>
+
+<p>Specify a tier configuration file path (unless you want to use the <a
href="https://github.com/apache/aurora/blob/rel/0.14.0/src/main/resources/org/apache/aurora/scheduler/tiers.json";>default</a>):</p>
+<pre class="highlight plaintext"><code>-tier_config=path/to/tiers/config.json
+</code></pre>
+
+<p>See the <a href="../../reference/configuration/">Configuration
Reference</a> for details on how to mark a job
+as being revocable.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a
href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How
To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/";>License</a></li>
+ <li><a
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ <li><a href="http://www.apache.org/security/";>Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a
href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under
the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a
href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX
photo</a> displayed on the homepage is available under a <a
href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons
BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo
are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Propchange:
aurora/site/publish/documentation/0.14.0/features/resource-isolation/index.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange:
aurora/site/publish/documentation/0.14.0/features/resource-isolation/index.html
------------------------------------------------------------------------------
svn:keywords = Id
Propchange:
aurora/site/publish/documentation/0.14.0/features/resource-isolation/index.html
------------------------------------------------------------------------------
svn:mime-type = text/html
Added:
aurora/site/publish/documentation/0.14.0/features/service-discovery/index.html
URL:
http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.14.0/features/service-discovery/index.html?rev=1748470&view=auto
==============================================================================
---
aurora/site/publish/documentation/0.14.0/features/service-discovery/index.html
(added)
+++
aurora/site/publish/documentation/0.14.0/features/service-discovery/index.html
Tue Jun 14 21:35:25 2016
@@ -0,0 +1,166 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";>
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type =
'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ?
'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300"
alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value +
'/features/service-discovery/'"
+ value="0.14.0">
+ <option value="0.14.0"
+ selected="selected">
+ 0.14.0
+ (latest)
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="service-discovery">Service Discovery</h1>
+
+<p>It is possible for the Aurora executor to announce tasks into ServerSets for
+the purpose of service discovery. ServerSets use the Zookeeper <a
href="http://zookeeper.apache.org/doc/trunk/recipes.html#sc_outOfTheBox";>group
membership pattern</a>
+of which there are several reference implementations:</p>
+
+<ul>
+<li><a
href="https://github.com/apache/mesos/blob/master/src/zookeeper/group.cpp";>C++</a></li>
+<li><a
href="https://github.com/twitter/commons/blob/master/src/java/com/twitter/common/zookeeper/ServerSetImpl.java#L221";>Java</a></li>
+<li><a
href="https://github.com/twitter/commons/blob/master/src/python/twitter/common/zookeeper/serverset/serverset.py#L51";>Python</a></li>
+</ul>
+
+<p>These can also be used natively in Finagle using the <a
href="https://github.com/twitter/finagle/blob/master/finagle-serversets/src/main/scala/com/twitter/finagle/zookeeper/ZookeeperServerSetCluster.scala";>ZookeeperServerSetCluster</a>.</p>
+
+<p>For more information about how to configure announcing, see the <a
href="../../reference/configuration/">Configuration Reference</a>.</p>
+
+<h2 id="using-mesos-discoveryinfo">Using Mesos DiscoveryInfo</h2>
+
+<p>Experimental support for populating DiscoveryInfo in Mesos is introduced in
Aurora. This can be used to build
+custom service discovery system not using zookeeper. Please see <code>Service
Discovery</code> section in
+<a
href="http://mesos.apache.org/documentation/latest/app-framework-development-guide/";>Mesos
Framework Development guide</a> for
+explanation of the protobuf message in Mesos.</p>
+
+<p>To use this feature, please enable <code>--populate_discovery_info</code>
flag on scheduler. All jobs started by scheduler
+afterwards will have their portmap populated to Mesos and discoverable in
<code>/state</code> endpoint in Mesos master and agent.</p>
+
+<h3 id="using-mesos-dns">Using Mesos DNS</h3>
+
+<p>An example is using <a
href="https://github.com/mesosphere/mesos-dns";>Mesos-DNS</a>, which is able to
generate multiple DNS
+records. With current implementation, the example job with key
<code>devcluster/vagrant/test/http-example</code> generates at
+least the following:</p>
+
+<ol>
+<li>An A record for <code>http_example.test.vagrant.aurora.mesos</code> (which
only includes IP address);</li>
+<li>A <a href="https://en.wikipedia.org/wiki/SRV_record";>SRV record</a> for
+<code>_http_example.test.vagrant._tcp.aurora.mesos</code>, which includes IP
address and every port. This should only
+be used if the service has one port.</li>
+<li>A SRV record
<code>_{port-name}._http_example.test.vagrant._tcp.aurora.mesos</code> for each
port name
+defined. This should be used when the service has multiple ports.</li>
+</ol>
+
+<p>Things to note:</p>
+
+<ol>
+<li>The domain part (“.mesos” in above example) can be configured
in <a
href="http://mesosphere.github.io/mesos-dns/docs/configuration-parameters.html";>Mesos
DNS</a>;</li>
+<li>Right now, portmap and port aliases in announcer object are not reflected
in DiscoveryInfo, therefore not visible in
+Mesos DNS records either. This is because they are only resolved in thermos
executors.</li>
+</ol>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a
href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How
To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/";>License</a></li>
+ <li><a
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ <li><a href="http://www.apache.org/security/";>Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a
href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under
the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a
href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX
photo</a> displayed on the homepage is available under a <a
href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons
BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo
are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Propchange:
aurora/site/publish/documentation/0.14.0/features/service-discovery/index.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange:
aurora/site/publish/documentation/0.14.0/features/service-discovery/index.html
------------------------------------------------------------------------------
svn:keywords = Id
Propchange:
aurora/site/publish/documentation/0.14.0/features/service-discovery/index.html
------------------------------------------------------------------------------
svn:mime-type = text/html
Added: aurora/site/publish/documentation/0.14.0/features/services/index.html
URL:
http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.14.0/features/services/index.html?rev=1748470&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.14.0/features/services/index.html
(added)
+++ aurora/site/publish/documentation/0.14.0/features/services/index.html Tue
Jun 14 21:35:25 2016
@@ -0,0 +1,207 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";>
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type =
'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ?
'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300"
alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value +
'/features/services/'"
+ value="0.14.0">
+ <option value="0.14.0"
+ selected="selected">
+ 0.14.0
+ (latest)
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="long-running-services">Long-running Services</h1>
+
+<p>Jobs that are always restart on completion, whether successful or
unsuccessful,
+are called services. This is useful for long-running processes
+such as webservices that should always be running, unless stopped
explicitly.</p>
+
+<h2 id="service-specification">Service Specification</h2>
+
+<p>A job is identified as a service by the presence of the flag
+`<code>service=True</code> in the <a
href="../../reference/configuration/#job-objects"><code>Job</code></a> object.
+The <code>Service</code> alias can be used as shorthand for <code>Job</code>
with <code>service=True</code>.</p>
+
+<p>Example (available in the <a href="../../getting-started/vagrant/">Vagrant
environment</a>):</p>
+<pre class="highlight plaintext"><code>$ cat
/vagrant/examples/jobs/hello_world.aurora
+hello = Process(
+ name = 'hello',
+ cmdline = """
+ while true; do
+ echo hello world
+ sleep 10
+ done
+ """)
+
+task = SequentialTask(
+ processes = [hello],
+ resources = Resources(cpu = 1.0, ram = 128*MB, disk = 128*MB)
+)
+
+jobs = [
+ Service(
+ task = task,
+ cluster = 'devcluster',
+ role = 'www-data',
+ environment = 'prod',
+ name = 'hello'
+ )
+]
+</code></pre>
+
+<p>Jobs without the service bit set only restart up to
<code>max_task_failures</code> times and only if they
+terminated unsuccessfully either due to human error or machine failure (see the
+<a href="../../reference/configuration/#job-objects"><code>Job</code></a>
object for details).</p>
+
+<h2 id="ports">Ports</h2>
+
+<p>In order to be useful, most services have to bind to one or more ports.
Aurora enables this
+usecase via the <a
href="../../reference/configuration/#thermos-namespace"><code>thermos.ports</code>
namespace</a> that
+allows to request arbitrarily named ports:</p>
+<pre class="highlight plaintext"><code>nginx = Process(
+ name = 'nginx',
+ cmdline = './run_nginx.sh -port {{thermos.ports[http]}}'
+)
+</code></pre>
+
+<p>When this process is included in a job, the job will be allocated a port,
and the command line
+will be replaced with something like:</p>
+<pre class="highlight plaintext"><code>./run_nginx.sh -port 42816
+</code></pre>
+
+<p>Where 42816 happens to be the allocated port.</p>
+
+<p>For details on how to enable clients to discover this dynamically assigned
port, see our
+<a href="../service-discovery/">Service Discovery</a> documentation.</p>
+
+<h2 id="health-checking">Health Checking</h2>
+
+<p>Typically, the Thermos executor monitors processes within a task only by
liveness of the forked
+process. In addition to that, Aurora has support for rudimentary health
checking: Either via HTTP
+via custom shell scripts.</p>
+
+<p>For example, simply by requesting a <code>health</code> port, a process can
request to be health checked
+via repeated calls to the <code>/health</code> endpoint:</p>
+<pre class="highlight plaintext"><code>nginx = Process(
+ name = 'nginx',
+ cmdline = './run_nginx.sh -port {{thermos.ports[health]}}'
+)
+</code></pre>
+
+<p>Please see the
+<a
href="../../reference/configuration/#healthcheckconfig-objects">configuration
reference</a>
+for configuration options for this feature.</p>
+
+<p>You can pause health checking by touching a file inside of your sandbox,
named <code>.healthchecksnooze</code>.
+As long as that file is present, health checks will be disabled, enabling
users to gather core
+dumps or other performance measurements without worrying about Aurora’s
health check killing
+their process.</p>
+
+<p>WARNING: Remember to remove this when you are done, otherwise your instance
will have permanently
+disabled health checks.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a
href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How
To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/";>License</a></li>
+ <li><a
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ <li><a href="http://www.apache.org/security/";>Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a
href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under
the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a
href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX
photo</a> displayed on the homepage is available under a <a
href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons
BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo
are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Propchange:
aurora/site/publish/documentation/0.14.0/features/services/index.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange:
aurora/site/publish/documentation/0.14.0/features/services/index.html
------------------------------------------------------------------------------
svn:keywords = Id
Propchange:
aurora/site/publish/documentation/0.14.0/features/services/index.html
------------------------------------------------------------------------------
svn:mime-type = text/html
Added: aurora/site/publish/documentation/0.14.0/features/sla-metrics/index.html
URL:
http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.14.0/features/sla-metrics/index.html?rev=1748470&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.14.0/features/sla-metrics/index.html
(added)
+++ aurora/site/publish/documentation/0.14.0/features/sla-metrics/index.html
Tue Jun 14 21:35:25 2016
@@ -0,0 +1,314 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";>
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type =
'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ?
'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300"
alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value +
'/features/sla-metrics/'"
+ value="0.14.0">
+ <option value="0.14.0"
+ selected="selected">
+ 0.14.0
+ (latest)
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="aurora-sla-measurement">Aurora SLA Measurement</h1>
+
+<ul>
+<li><a href="#overview">Overview</a></li>
+<li><a href="#metric-details">Metric Details</a>
+
+<ul>
+<li><a href="#platform-uptime">Platform Uptime</a></li>
+<li><a href="#job-uptime">Job Uptime</a></li>
+<li><a href="#median-time-to-assigned-(mtta)">Median Time To Assigned
(MTTA)</a></li>
+<li><a href="#median-time-to-running-(mttr)">Median Time To Running
(MTTR)</a></li>
+</ul></li>
+<li><a href="#limitations">Limitations</a></li>
+</ul>
+
+<h2 id="overview">Overview</h2>
+
+<p>The primary goal of the feature is collection and monitoring of Aurora job
SLA (Service Level
+Agreements) metrics that defining a contractual relationship between the
Aurora/Mesos platform
+and hosted services.</p>
+
+<p>The Aurora SLA feature is by default only enabled for service (non-cron)
+production jobs (<code>"production=True"</code> in your
<code>.aurora</code> config). It can be enabled for
+non-production services by an operator via the scheduler command line flag
<code>-sla_non_prod_metrics</code>.</p>
+
+<p>Counters that track SLA measurements are computed periodically within the
scheduler.
+The individual instance metrics are refreshed every minute (configurable via
+<code>sla_stat_refresh_interval</code>). The instance counters are
subsequently aggregated by
+relevant grouping types before exporting to scheduler <code>/vars</code>
endpoint (when using <code>vagrant</code>
+that would be <code>http://192.168.33.7:8081/vars</code>)</p>
+
+<h2 id="metric-details">Metric Details</h2>
+
+<h3 id="platform-uptime">Platform Uptime</h3>
+
+<p><em>Aggregate amount of time a job spends in a non-runnable state due to
platform unavailability
+or scheduling delays. This metric tracks Aurora/Mesos uptime performance and
reflects on any
+system-caused downtime events (tasks LOST or DRAINED). Any user-initiated task
kills/restarts
+will not degrade this metric.</em></p>
+
+<p><strong>Collection scope:</strong></p>
+
+<ul>
+<li>Per job - <code>sla_<job_key>_platform_uptime_percent</code></li>
+<li>Per cluster - <code>sla_cluster_platform_uptime_percent</code></li>
+</ul>
+
+<p><strong>Units:</strong> percent</p>
+
+<p>A fault in the task environment may cause the Aurora/Mesos to have
different views on the task state
+or lose track of the task existence. In such cases, the service task is marked
as LOST and
+rescheduled by Aurora. For example, this may happen when the task stays in
ASSIGNED or STARTING
+for too long or the Mesos agent becomes unhealthy (or disappears completely).
The time between
+task entering LOST and its replacement reaching RUNNING state is counted
towards platform downtime.</p>
+
+<p>Another example of a platform downtime event is the administrator-requested
task rescheduling. This
+happens during planned Mesos agent maintenance when all agent tasks are marked
as DRAINED and
+rescheduled elsewhere.</p>
+
+<p>To accurately calculate Platform Uptime, we must separate platform incurred
downtime from user
+actions that put a service instance in a non-operational state. It is simpler
to isolate
+user-incurred downtime and treat all other downtime as platform incurred.</p>
+
+<p>Currently, a user can cause a healthy service (task) downtime in only two
ways: via <code>killTasks</code>
+or <code>restartShards</code> RPCs. For both, their affected tasks leave an
audit state transition trail
+relevant to uptime calculations. By applying a special “SLA
meaning” to exposed task state
+transition records, we can build a deterministic downtime trace for every
given service instance.</p>
+
+<p>A task going through a state transition carries one of three possible SLA
meanings
+(see <a
href="https://github.com/apache/aurora/blob/rel/0.14.0/src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java";>SlaAlgorithm.java</a>
for
+sla-to-task-state mapping):</p>
+
+<ul>
+<li><p>Task is UP: starts a period where the task is considered to be up and
running from the Aurora
+platform standpoint.</p></li>
+<li><p>Task is DOWN: starts a period where the task cannot reach the UP state
for some
+non-user-related reason. Counts towards instance downtime.</p></li>
+<li><p>Task is REMOVED from SLA: starts a period where the task is not
expected to be UP due to
+user initiated action or failure. We ignore this period for the uptime
calculation purposes.</p></li>
+</ul>
+
+<p>This metric is recalculated over the last sampling period (last minute) to
account for
+any UP/DOWN/REMOVED events. It ignores any UP/DOWN events not immediately
adjacent to the
+sampling interval as well as adjacent REMOVED events.</p>
+
+<h3 id="job-uptime">Job Uptime</h3>
+
+<p><em>Percentage of the job instances considered to be in RUNNING state for
the specified duration
+relative to request time. This is a purely application side metric that is
considering aggregate
+uptime of all RUNNING instances. Any user- or platform initiated restarts
directly affect
+this metric.</em></p>
+
+<p><strong>Collection scope:</strong> We currently expose job uptime values at
5 pre-defined
+percentiles (50th,75th,90th,95th and 99th):</p>
+
+<ul>
+<li><code>sla_<job_key>_job_uptime_50_00_sec</code></li>
+<li><code>sla_<job_key>_job_uptime_75_00_sec</code></li>
+<li><code>sla_<job_key>_job_uptime_90_00_sec</code></li>
+<li><code>sla_<job_key>_job_uptime_95_00_sec</code></li>
+<li><code>sla_<job_key>_job_uptime_99_00_sec</code></li>
+</ul>
+
+<p><strong>Units:</strong> seconds
+You can also get customized real-time stats from aurora client. See
<code>aurora sla -h</code> for
+more details.</p>
+
+<h3 id="median-time-to-assigned-mtta">Median Time To Assigned (MTTA)</h3>
+
+<p><em>Median time a job spends waiting for its tasks to be assigned to a
host. This is a combined
+metric that helps track the dependency of scheduling performance on the
requested resources
+(user scope) as well as the internal scheduler bin-packing algorithm
efficiency (platform scope).</em></p>
+
+<p><strong>Collection scope:</strong></p>
+
+<ul>
+<li>Per job - <code>sla_<job_key>_mtta_ms</code></li>
+<li>Per cluster - <code>sla_cluster_mtta_ms</code></li>
+<li>Per instance size (small, medium, large, x-large, xx-large). Size are
defined in:
+<a
href="https://github.com/apache/aurora/blob/rel/0.14.0/src/main/java/org/apache/aurora/scheduler/base/ResourceAggregates.java";>ResourceAggregates.java</a>
+
+<ul>
+<li>By CPU:</li>
+<li><code>sla_cpu_small_mtta_ms</code></li>
+<li><code>sla_cpu_medium_mtta_ms</code></li>
+<li><code>sla_cpu_large_mtta_ms</code></li>
+<li><code>sla_cpu_xlarge_mtta_ms</code></li>
+<li><code>sla_cpu_xxlarge_mtta_ms</code></li>
+<li>By RAM:</li>
+<li><code>sla_ram_small_mtta_ms</code></li>
+<li><code>sla_ram_medium_mtta_ms</code></li>
+<li><code>sla_ram_large_mtta_ms</code></li>
+<li><code>sla_ram_xlarge_mtta_ms</code></li>
+<li><code>sla_ram_xxlarge_mtta_ms</code></li>
+<li>By DISK:</li>
+<li><code>sla_disk_small_mtta_ms</code></li>
+<li><code>sla_disk_medium_mtta_ms</code></li>
+<li><code>sla_disk_large_mtta_ms</code></li>
+<li><code>sla_disk_xlarge_mtta_ms</code></li>
+<li><code>sla_disk_xxlarge_mtta_ms</code></li>
+</ul></li>
+</ul>
+
+<p><strong>Units:</strong> milliseconds</p>
+
+<p>MTTA only considers instances that have already reached ASSIGNED state and
ignores those
+that are still PENDING. This ensures straggler instances (e.g. with
unreasonable resource
+constraints) do not affect metric curves.</p>
+
+<h3 id="median-time-to-running-mttr">Median Time To Running (MTTR)</h3>
+
+<p><em>Median time a job waits for its tasks to reach RUNNING state. This is a
comprehensive metric
+reflecting on the overall time it takes for the Aurora/Mesos to start
executing user content.</em></p>
+
+<p><strong>Collection scope:</strong></p>
+
+<ul>
+<li>Per job - <code>sla_<job_key>_mttr_ms</code></li>
+<li>Per cluster - <code>sla_cluster_mttr_ms</code></li>
+<li>Per instance size (small, medium, large, x-large, xx-large). Size are
defined in:
+<a
href="https://github.com/apache/aurora/blob/rel/0.14.0/src/main/java/org/apache/aurora/scheduler/base/ResourceAggregates.java";>ResourceAggregates.java</a>
+
+<ul>
+<li>By CPU:</li>
+<li><code>sla_cpu_small_mttr_ms</code></li>
+<li><code>sla_cpu_medium_mttr_ms</code></li>
+<li><code>sla_cpu_large_mttr_ms</code></li>
+<li><code>sla_cpu_xlarge_mttr_ms</code></li>
+<li><code>sla_cpu_xxlarge_mttr_ms</code></li>
+<li>By RAM:</li>
+<li><code>sla_ram_small_mttr_ms</code></li>
+<li><code>sla_ram_medium_mttr_ms</code></li>
+<li><code>sla_ram_large_mttr_ms</code></li>
+<li><code>sla_ram_xlarge_mttr_ms</code></li>
+<li><code>sla_ram_xxlarge_mttr_ms</code></li>
+<li>By DISK:</li>
+<li><code>sla_disk_small_mttr_ms</code></li>
+<li><code>sla_disk_medium_mttr_ms</code></li>
+<li><code>sla_disk_large_mttr_ms</code></li>
+<li><code>sla_disk_xlarge_mttr_ms</code></li>
+<li><code>sla_disk_xxlarge_mttr_ms</code></li>
+</ul></li>
+</ul>
+
+<p><strong>Units:</strong> milliseconds</p>
+
+<p>MTTR only considers instances in RUNNING state. This ensures straggler
instances (e.g. with
+unreasonable resource constraints) do not affect metric curves.</p>
+
+<h2 id="limitations">Limitations</h2>
+
+<ul>
+<li><p>The availability of Aurora SLA metrics is bound by the scheduler
availability.</p></li>
+<li><p>All metrics are calculated at a pre-defined interval (currently set at
1 minute).
+Scheduler restarts may result in missed collections.</p></li>
+</ul>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a
href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How
To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/";>License</a></li>
+ <li><a
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ <li><a href="http://www.apache.org/security/";>Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a
href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under
the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a
href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX
photo</a> displayed on the homepage is available under a <a
href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons
BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo
are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Propchange:
aurora/site/publish/documentation/0.14.0/features/sla-metrics/index.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange:
aurora/site/publish/documentation/0.14.0/features/sla-metrics/index.html
------------------------------------------------------------------------------
svn:keywords = Id
Propchange:
aurora/site/publish/documentation/0.14.0/features/sla-metrics/index.html
------------------------------------------------------------------------------
svn:mime-type = text/html
Added: aurora/site/publish/documentation/0.14.0/features/webhooks/index.html
URL:
http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.14.0/features/webhooks/index.html?rev=1748470&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.14.0/features/webhooks/index.html
(added)
+++ aurora/site/publish/documentation/0.14.0/features/webhooks/index.html Tue
Jun 14 21:35:25 2016
@@ -0,0 +1,194 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";>
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type =
'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ?
'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300"
alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value +
'/features/webhooks/'"
+ value="0.14.0">
+ <option value="0.14.0"
+ selected="selected">
+ 0.14.0
+ (latest)
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="webhooks">Webhooks</h1>
+
+<p>Aurora has an optional feature which allows operator to specify a file to
configure a HTTP webhook
+to receive task state change events. It can be enabled with a scheduler flag eg
+<code>-webhook_config=/path/to/webhook.json</code>. At this point, webhooks
are still considered <em>experimental</em>.</p>
+
+<p>Below is a sample configuration:</p>
+<pre class="highlight json"><code><span style="background-color:
#f8f8f8">{</span><span style="color: #bbbbbb">
+ </span><span style="color: #000080">"headers"</span><span
style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb">
</span><span style="background-color: #f8f8f8">{</span><span style="color:
#bbbbbb">
+ </span><span style="color: #000080">"Content-Type"</span><span
style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb">
</span><span style="color:
#d14">"application/vnd.kafka.json.v1+json"</span><span style="background-color:
#f8f8f8">,</span><span style="color: #bbbbbb">
+ </span><span style="color: #000080">"Producer-Type"</span><span
style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb">
</span><span style="color: #d14">"reliable"</span><span style="color: #bbbbbb">
+ </span><span style="background-color: #f8f8f8">},</span><span style="color:
#bbbbbb">
+ </span><span style="color: #000080">"targetURL"</span><span
style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb">
</span><span style="color: #d14">"http://localhost:5000/";</span><span
style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb">
+ </span><span style="color: #000080">"timeoutMsec"</span><span
style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb">
</span><span style="color: #009999">5</span><span style="color: #bbbbbb">
+</span><span style="background-color: #f8f8f8">}</span><span style="color:
#bbbbbb">
+</span></code></pre>
+
+<p>And an example of a response that you will get back:
+<code>json
+{
+ "task":
+ {
+ "cachedHashCode":0,
+ "assignedTask": {
+ "cachedHashCode":0,
+
"taskId":"vagrant-test-http_example-8-a6cf7ec5-d793-49c7-b10f-0e14ab80bfff",
+ "task": {
+ "cachedHashCode":-1819348376,
+ "job": {
+ "cachedHashCode":803049425,
+ "role":"vagrant",
+ "environment":"test",
+ "name":"http_example"
+ },
+ "owner": {
+ "cachedHashCode":226895216,
+ "user":"vagrant"
+ },
+ "isService":true,
+ "numCpus":0.1,
+ "ramMb":16,
+ "diskMb":8,
+ "priority":0,
+ "maxTaskFailures":1,
+ "production":false,
+ "resources":[
+
{"cachedHashCode":729800451,"setField":"NUM_CPUS","value":0.1},
+
{"cachedHashCode":552899914,"setField":"RAM_MB","value":16},
+
{"cachedHashCode":-1547868317,"setField":"DISK_MB","value":8},
+
{"cachedHashCode":1957328227,"setField":"NAMED_PORT","value":"http"},
+
{"cachedHashCode":1954229436,"setField":"NAMED_PORT","value":"tcp"}
+ ],
+ "constraints":[],
+ "requestedPorts":["http","tcp"],
+
"taskLinks":{"http":"http://%host%:%port:http%"},
+ "contactEmail":"vagrant@localhost",
+ "executorConfig": {
+ "cachedHashCode":-1194797325,
+ "name":"AuroraExecutor",
+ "data": "{\"environment\":
\"test\", \"health_check_config\":
{\"initial_interval_secs\": 5.0, \"health_checker\": {
\"http\": {\"expected_response_code\": 0,
\"endpoint\": \"/health\", \"expected_response\":
\"ok\"}}, \"max_consecutive_failures\": 0,
\"timeout_secs\": 1.0, \"interval_secs\": 1.0},
\"name\": \"http_example\", \"service\": true,
\"max_task_failures\": 1, \"cron_collision_policy\":
\"KILL_EXISTING\", \"enable_hooks\": false,
\"cluster\": \"devcluster\", \"task\":
{\"processes\": [{\"daemon\": false, \"name\":
\"echo_ports\", \"ephemeral\": false,
\"max_failures\": 1, \"min_duration\": 5,
\"cmdline\": \"echo \\\&quo
t;tcp port: {{thermos.ports[tcp]}}; http port: {{thermos.ports[http]}}; alias:
{{thermos.ports[alias]}}\\\"\", \"final\": false},
{\"daemon\": false, \"name\": \"stage_server\",
\"ephemeral\": false, \"max_failures\": 1,
\"min_duration\": 5, \"cmdline\": \"cp
/vagrant/src/test/sh/org/apache/aurora/e2e/http_example.py .\",
\"final\": false}, {\"daemon\": false, \"name\":
\"run_server\", \"ephemeral\": false,
\"max_failures\": 1, \"min_duration\": 5,
\"cmdline\": \"python http_example.py
{{thermos.ports[http]}}\", \"final\": false}],
\"name\": \"http_example\",
\"finalization_wait\": 30, \"max_failures\": 1,
\"max_concurrency\": 0, \"resources\": {\"disk\":
8388608, \"ram\": 16777216, \"cpu\": 0.1},
\"constraints\": [{\"order\": [\"echo_ports\",
\"stage_server\", \"run_server\"]}]},
\"production\": false, \"role\": \"vagrant\",
\"contact\": \"vagrant@localhost\", \"announce\":
{\"primary_port\": \"http\", \"portmap\":
{\"alias\": \"http\"}}, \"lifecycle\":
{\"http\": {\"graceful_shutdown_endpoint\":
\"/quitquitquit\", \"port\": \"health\",
\"shutdown_endpoint\": \"/abortabortabort\"}},
\"priority\": 0}"},
+ "metadata":[],
+ "container":{
+ "cachedHashCode":-1955376216,
+ "setField":"MESOS",
+ "value":{"cachedHashCode":31}}
+ },
+ "assignedPorts":{},
+ "instanceId":8
+ },
+ "status":"PENDING",
+ "failureCount":0,
+ "taskEvents":[
+
{"cachedHashCode":0,"timestamp":1464992060258,"status":"PENDING","scheduler":"aurora"}]
+ },
+ "oldState":{}}
+</code></p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a
href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How
To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/";>License</a></li>
+ <li><a
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ <li><a href="http://www.apache.org/security/";>Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a
href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under
the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a
href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX
photo</a> displayed on the homepage is available under a <a
href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons
BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo
are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Propchange:
aurora/site/publish/documentation/0.14.0/features/webhooks/index.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange:
aurora/site/publish/documentation/0.14.0/features/webhooks/index.html
------------------------------------------------------------------------------
svn:keywords = Id
Propchange:
aurora/site/publish/documentation/0.14.0/features/webhooks/index.html
------------------------------------------------------------------------------
svn:mime-type = text/html
Added:
aurora/site/publish/documentation/0.14.0/getting-started/overview/index.html
URL:
http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.14.0/getting-started/overview/index.html?rev=1748470&view=auto
==============================================================================
---
aurora/site/publish/documentation/0.14.0/getting-started/overview/index.html
(added)
+++
aurora/site/publish/documentation/0.14.0/getting-started/overview/index.html
Tue Jun 14 21:35:25 2016
@@ -0,0 +1,225 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet"
href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";>
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type =
'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ?
'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300"
alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value +
'/getting-started/overview/'"
+ value="0.14.0">
+ <option value="0.14.0"
+ selected="selected">
+ 0.14.0
+ (latest)
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="aurora-system-overview">Aurora System Overview</h1>
+
+<p>Apache Aurora is a service scheduler that runs on top of Apache Mesos,
enabling you to run
+long-running services, cron jobs, and ad-hoc jobs that take advantage of
Apache Mesos’ scalability,
+fault-tolerance, and resource isolation.</p>
+
+<h2 id="components">Components</h2>
+
+<p>It is important to have an understanding of the components that make up
+a functioning Aurora cluster.</p>
+
+<p><img alt="Aurora Components" src="../../images/components.png" /></p>
+
+<ul>
+<li><p><strong>Aurora scheduler</strong>
+The scheduler is your primary interface to the work you run in your cluster.
You will
+instruct it to run jobs, and it will manage them in Mesos for you. You will
also frequently use
+the scheduler’s read-only web interface as a heads-up display for
what’s running in your cluster.</p></li>
+<li><p><strong>Aurora client</strong>
+The client (<code>aurora</code> command) is a command line tool that exposes
primitives that you can use to
+interact with the scheduler. The client operates on</p></li>
+</ul>
+
+<p>Aurora also provides an admin client (<code>aurora_admin</code> command)
that contains commands built for
+ cluster administrators. You can use this tool to do things like manage user
quotas and manage
+ graceful maintenance on machines in cluster.</p>
+
+<ul>
+<li><p><strong>Aurora executor</strong>
+The executor (a.k.a. Thermos executor) is responsible for carrying out the
workloads described in
+the Aurora DSL (<code>.aurora</code> files). The executor is what actually
executes user processes. It will
+also perform health checking of tasks and register tasks in ZooKeeper for the
purposes of dynamic
+service discovery.</p></li>
+<li><p><strong>Aurora observer</strong>
+The observer provides browser-based access to the status of individual tasks
executing on worker
+machines. It gives insight into the processes executing, and facilitates
browsing of task sandbox
+directories.</p></li>
+<li><p><strong>ZooKeeper</strong>
+<a href="http://zookeeper.apache.org";>ZooKeeper</a> is a distributed consensus
system. In an Aurora cluster
+it is used for reliable election of the leading Aurora scheduler and Mesos
master. It is also
+used as a vehicle for service discovery, see <a
href="../../features/service-discovery/">Service Discovery</a></p></li>
+<li><p><strong>Mesos master</strong>
+The master is responsible for tracking worker machines and performing
accounting of their
+resources. The scheduler interfaces with the master to control the
cluster.</p></li>
+<li><p><strong>Mesos agent</strong>
+The agent receives work assigned by the scheduler and executes them. It
interfaces with Linux
+isolation systems like cgroups, namespaces and Docker to manage the resource
consumption of tasks.
+When a user task is launched, the agent will launch the executor (in the
context of a Linux cgroup
+or Docker container depending upon the environment), which will in turn fork
user processes.</p></li>
+</ul>
+
+<p>In earlier versions of Mesos and Aurora, the Mesos agent was known as the
Mesos slave.</p>
+
+<h2 id="jobs-tasks-and-processes">Jobs, Tasks and Processes</h2>
+
+<p>Aurora is a Mesos framework used to schedule <em>jobs</em> onto Mesos. Mesos
+cares about individual <em>tasks</em>, but typical jobs consist of dozens or
+hundreds of task replicas. Aurora provides a layer on top of Mesos with
+its <code>Job</code> abstraction. An Aurora <code>Job</code> consists of a
task template and
+instructions for creating near-identical replicas of that task (modulo
+things like “instance id” or specific port numbers which may
differ from
+machine to machine).</p>
+
+<p>How many tasks make up a Job is complicated. On a basic level, a Job
consists of
+one task template and instructions for creating near-identical replicas of
that task
+(otherwise referred to as “instances” or “shards”).</p>
+
+<p>A task can merely be a single <em>process</em> corresponding to a single
+command line, such as <code>python2.7 my_script.py</code>. However, a task can
also
+consist of many separate processes, which all run within a single
+sandbox. For example, running multiple cooperating agents together,
+such as <code>logrotate</code>, <code>installer</code>, master, or agent
processes. This is
+where Thermos comes in. While Aurora provides a <code>Job</code> abstraction on
+top of Mesos <code>Tasks</code>, Thermos provides a <code>Process</code>
abstraction
+underneath Mesos <code>Task</code>s and serves as part of the Aurora
framework’s
+executor.</p>
+
+<p>You define <code>Job</code>s,<code>Task</code>s, and <code>Process</code>es
in a configuration file.
+Configuration files are written in Python, and make use of the
+<a href="https://github.com/wickman/pystachio";>Pystachio</a> templating
language,
+along with specific Aurora, Mesos, and Thermos commands and methods.
+The configuration files typically end with a <code>.aurora</code>
extension.</p>
+
+<p>Summary:</p>
+
+<ul>
+<li>Aurora manages jobs made of tasks.</li>
+<li>Mesos manages tasks made of processes.</li>
+<li>Thermos manages processes.</li>
+<li>All that is defined in <code>.aurora</code> configuration files</li>
+</ul>
+
+<p><img alt="Aurora hierarchy" src="../../images/aurora_hierarchy.png" /></p>
+
+<p>Each <code>Task</code> has a <em>sandbox</em> created when the
<code>Task</code> starts and garbage
+collected when it finishes. All of a <code>Task'</code>s processes run in
its
+sandbox, so processes can share state by using a shared current working
+directory.</p>
+
+<p>The sandbox garbage collection policy considers many factors, most
+importantly age and size. It makes a best-effort attempt to keep
+sandboxes around as long as possible post-task in order for service
+owners to inspect data and logs, should the <code>Task</code> have completed
+abnormally. But you can’t design your applications assuming sandboxes
+will be around forever, e.g. by building log saving or other
+checkpointing mechanisms directly into your application or into your
+<code>Job</code> description.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a
href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How
To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/";>License</a></li>
+ <li><a
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ <li><a href="http://www.apache.org/security/";>Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a
href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under
the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a
href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX
photo</a> displayed on the homepage is available under a <a
href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons
BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo
are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Propchange:
aurora/site/publish/documentation/0.14.0/getting-started/overview/index.html
------------------------------------------------------------------------------
svn:eol-style = native
Propchange:
aurora/site/publish/documentation/0.14.0/getting-started/overview/index.html
------------------------------------------------------------------------------
svn:keywords = Id
Propchange:
aurora/site/publish/documentation/0.14.0/getting-started/overview/index.html
------------------------------------------------------------------------------
svn:mime-type = text/html
