Added: aurora/site/publish/documentation/0.19.1/development/ui/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/development/ui/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/development/ui/index.html (added) +++ aurora/site/publish/documentation/0.19.1/development/ui/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,194 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/ui/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="developing-the-aurora-scheduler-ui">Developing the Aurora Scheduler UI</h1> + +<h2 id="installing-bower-optional">Installing bower (optional)</h2> + +<p>Third party JS libraries used in Aurora (located at 3rdparty/javascript/bower_components) are +managed by bower, a JS dependency manager. Bower is only required if you plan to add, remove or +update JS libraries. Bower can be installed using the following command:</p> +<pre class="highlight plaintext"><code>npm install -g bower +</code></pre> + +<p>Bower depends on node.js and npm. The easiest way to install node on a mac is via brew:</p> +<pre class="highlight plaintext"><code>brew install node +</code></pre> + +<p>For more node.js installation options refer to <a href="https://github.com/joyent/node/wiki/Installation";>https://github.com/joyent/node/wiki/Installation</a>.</p> + +<p>More info on installing and using bower can be found at: <a href="http://bower.io/";>http://bower.io/</a>. Once installed, you can +use the following commands to view and modify the bower repo at +3rdparty/javascript/bower_components</p> +<pre class="highlight plaintext"><code>bower list +bower install <library name> +bower remove <library name> +bower update <library name> +bower help +</code></pre> + +<h2 id="faster-iteration-in-vagrant">Faster Iteration in Vagrant</h2> + +<p>The scheduler serves UI assets from the classpath. For production deployments this means the assets +are served from within a jar. However, for faster development iteration, the vagrant image is +configured to add the <code>scheduler</code> subtree of <code>/vagrant/dist/resources/main</code> to the head of +<code>CLASSPATH</code>. This path is configured as a shared filesystem to the path on the host system where +your Aurora repository lives. This means that any updates under <code>dist/resources/main/scheduler</code> in +your checkout will be reflected immediately in the UI served from within the vagrant image.</p> + +<p>The one caveat to this is that this path is under <code>dist</code> not <code>src</code>. This is because the assets must +be processed by gradle before they can be served. So, unfortunately, you cannot just save your local +changes and see them reflected in the UI, you must first run <code>./gradlew processResources</code>. This is +less than ideal, but better than having to restart the scheduler after every change. Additionally, +gradle makes this process somewhat easier with the use of the <code>--continuous</code> flag. If you run: +<code>./gradlew processResources --continuous</code> gradle will monitor the filesystem for changes and run the +task automatically as necessary. This doesn’t quite provide hot-reload capabilities, but it does +allow for <5s from save to changes being visibile in the UI with no further action required on the +part of the developer.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html>
Added: aurora/site/publish/documentation/0.19.1/features/constraints/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/features/constraints/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/features/constraints/index.html (added) +++ aurora/site/publish/documentation/0.19.1/features/constraints/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,264 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/features/constraints/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="scheduling-constraints">Scheduling Constraints</h1> + +<p>By default, Aurora will pick any random agent with sufficient resources +in order to schedule a task. This scheduling choice can be further +restricted with the help of constraints.</p> + +<h2 id="mesos-attributes">Mesos Attributes</h2> + +<p>Data centers are often organized with hierarchical failure domains. Common failure domains +include hosts, racks, rows, and PDUs. If you have this information available, it is wise to tag +the Mesos agent with them as +<a href="https://mesos.apache.org/documentation/attributes-resources/";>attributes</a>.</p> + +<p>The Mesos agent <code>--attributes</code> command line argument can be used to mark agents with +static key/value pairs, so called attributes (not to be confused with <code>--resources</code>, which are +dynamic and accounted).</p> + +<p>For example, consider the host <code>cluster1-aaa-03-sr2</code> and its following attributes (given in +key:value format): <code>host:cluster1-aaa-03-sr2</code> and <code>rack:aaa</code>.</p> + +<p>Aurora makes these attributes available for matching with scheduling constraints.</p> + +<h2 id="limit-constraints">Limit Constraints</h2> + +<p>Limit constraints allow to control machine diversity using constraints. The below +constraint ensures that no more than two instances of your job may run on a single host. +Think of this as a “group by” limit.</p> +<pre class="highlight plaintext"><code>Service( + name = 'webservice', + role = 'www-data', + constraints = { + 'host': 'limit:2', + } + ... +) +</code></pre> + +<p>Likewise, you can use constraints to control rack diversity, e.g. at +most one task per rack:</p> +<pre class="highlight plaintext"><code>constraints = { + 'rack': 'limit:1', +} +</code></pre> + +<p>Use these constraints sparingly as they can dramatically reduce Tasks’ schedulability. +Further details are available in the reference documentation on +<a href="../../reference/configuration/#specifying-scheduling-constraints">Scheduling Constraints</a>.</p> + +<h2 id="value-constraints">Value Constraints</h2> + +<p>Value constraints can be used to express that a certain attribute with a certain value +should be present on a Mesos agent. For example, the following job would only be +scheduled on nodes that claim to have an <code>SSD</code> as their disk.</p> +<pre class="highlight plaintext"><code>Service( + name = 'webservice', + role = 'www-data', + constraints = { + 'disk': 'SSD', + } + ... +) +</code></pre> + +<p>Further details are available in the reference documentation on +<a href="../../reference/configuration/#specifying-scheduling-constraints">Scheduling Constraints</a>.</p> + +<h2 id="running-stateful-services">Running stateful services</h2> + +<p>Aurora is best suited to run stateless applications, but it also accommodates for stateful services +like databases, or services that otherwise need to always run on the same machines.</p> + +<h3 id="dedicated-attribute">Dedicated attribute</h3> + +<p>Most of the Mesos attributes arbitrary and available for custom use. There is one exception, +though: the <code>dedicated</code> attribute. Aurora treats this specially, and only allows matching jobs to +run on these machines, and will only schedule matching jobs on these machines.</p> + +<h4 id="syntax">Syntax</h4> + +<p>The dedicated attribute has semantic meaning. The format is <code>$role(/.*)?</code>. When a job is created, +the scheduler requires that the <code>$role</code> component matches the <code>role</code> field in the job +configuration, and will reject the job creation otherwise. The remainder of the attribute is +free-form. We’ve developed the idiom of formatting this attribute as <code>$role/$job</code>, but do not +enforce this. For example: a job <code>devcluster/www-data/prod/hello</code> with a dedicated constraint set as +<code>www-data/web.multi</code> will have its tasks scheduled only on Mesos agents configured with: +<code>--attributes=dedicated:www-data/web.multi</code>.</p> + +<p>A wildcard (<code>*</code>) may be used for the role portion of the dedicated attribute, which will allow any +owner to elect for a job to run on the host(s). For example: tasks from both +<code>devcluster/www-data/prod/hello</code> and <code>devcluster/vagrant/test/hello</code> with a dedicated constraint +formatted as <code>*/web.multi</code> will be scheduled only on Mesos agents configured with +<code>--attributes=dedicated:*/web.multi</code>. This may be useful when assembling a virtual cluster of +machines sharing the same set of traits or requirements.</p> + +<h5 id="example">Example</h5> + +<p>Consider the following agent command line:</p> +<pre class="highlight plaintext"><code>mesos-slave --attributes="dedicated:db_team/redis" ... +</code></pre> + +<p>And this job configuration:</p> +<pre class="highlight plaintext"><code>Service( + name = 'redis', + role = 'db_team', + constraints = { + 'dedicated': 'db_team/redis' + } + ... +) +</code></pre> + +<p>The job configuration is indicating that it should only be scheduled on agents with the attribute +<code>dedicated:db_team/redis</code>. Additionally, Aurora will prevent any tasks that do <em>not</em> have that +constraint from running on those agents.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.19.1/features/containers/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/features/containers/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/features/containers/index.html (added) +++ aurora/site/publish/documentation/0.19.1/features/containers/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,276 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/features/containers/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="containers">Containers</h1> + +<p>Aurora supports several containerizers, notably the Mesos containerizer and the Docker +containerizer. The Mesos containerizer uses native OS features directly to provide isolation between +containers, while the Docker containerizer delegates container management to the Docker engine.</p> + +<p>The support for launching container images via both containerizers has to be +<a href="../../operations/configuration/#containers">enabled by a cluster operator</a>.</p> + +<h2 id="mesos-containerizer">Mesos Containerizer</h2> + +<p>The Mesos containerizer is the native Mesos containerizer solution. It allows tasks to be +run with an array of <a href="../resource-isolation/">pluggable isolators</a> and can launch tasks using +<a href="https://github.com/docker/docker/blob/master/image/spec/v1.md";>Docker</a> images, +<a href="https://github.com/appc/spec/blob/master/SPEC.md";>AppC</a> images, or directly on the agent host +filesystem.</p> + +<p>The following example (available in our <a href="../../getting-started/vagrant/">Vagrant environment</a>) +launches a hello world example within a <code>debian/jessie</code> Docker image:</p> +<pre class="highlight plaintext"><code>$ cat /vagrant/examples/jobs/hello_docker_image.aurora +hello_loop = Process( + name = 'hello', + cmdline = """ + while true; do + echo hello world + sleep 10 + done + """) + +task = Task( + processes = [hello_loop], + resources = Resources(cpu=1, ram=1*MB, disk=8*MB) +) + +jobs = [ + Service( + cluster = 'devcluster', + environment = 'devel', + role = 'www-data', + name = 'hello_docker_image', + task = task, + container = Mesos(image=DockerImage(name='debian', tag='jessie')) + ) +] +</code></pre> + +<p>Docker and Appc images are designated using an appropriate <code>image</code> property of the <code>Mesos</code> +configuration object. If either <code>container</code> or <code>image</code> is left unspecified, the host filesystem +will be used. Further details of how to specify images can be found in the +<a href="../../reference/configuration/#mesos-object">Reference Documentation</a>.</p> + +<p>By default, Aurora launches processes as the Linux user named like the used role (e.g. <code>www-data</code> +in the example above). This user has to exist on the host filesystem. If it does not exist within +the container image, it will be created automatically. Otherwise, this user and its primary group +has to exist in the image with matching uid/gid.</p> + +<p>For more information on the Mesos containerizer filesystem, namespace, and isolator features, visit +<a href="http://mesos.apache.org/documentation/latest/mesos-containerizer/";>Mesos Containerizer</a> and +<a href="http://mesos.apache.org/documentation/latest/container-image/";>Mesos Container Images</a>.</p> + +<h2 id="docker-containerizer">Docker Containerizer</h2> + +<p>The Docker containerizer launches container images using the Docker engine. It may often provide +more advanced features than the native Mesos containerizer, but has to be installed separately to +Mesos on each agent host.</p> + +<p>Starting with the 0.17.0 release, <code>image</code> can be specified with a <code>{{docker.image[name][tag]}}</code> binder so that +the tag can be resolved to a concrete image digest. This ensures that the job always uses the same image +across restarts, even if the version identified by the tag has been updated, guaranteeing that only job +updates can mutate configuration.</p> + +<p>Example (available in the <a href="../../getting-started/vagrant/">Vagrant environment</a>):</p> +<pre class="highlight plaintext"><code>$ cat /vagrant/examples/jobs/hello_docker_engine.aurora +hello_loop = Process( + name = 'hello', + cmdline = """ + while true; do + echo hello world + sleep 10 + done + """) + +task = Task( + processes = [hello_loop], + resources = Resources(cpu=1, ram=1*MB, disk=8*MB) +) + +jobs = [ + Service( + cluster = 'devcluster', + environment = 'devel', + role = 'www-data', + name = 'hello_docker', + task = task, + container = Docker(image = 'python:2.7') + ), Service( + cluster = 'devcluster', + environment = 'devel', + role = 'www-data', + name = 'hello_docker_engine_binding', + task = task, + container = Docker(image = '{{docker.image[library/python][2.7]}}') + ) +] +</code></pre> + +<p>Note, this feature requires a v2 Docker registry. If using a private Docker registry its url +must be specified in the <code>clusters.json</code> configuration file under the key <code>docker_registry</code>. +If not specified <code>docker_registry</code> defaults to <code>https://registry-1.docker.io</code> (Docker Hub).</p> + +<p>Example: + # clusters.json + [{ + “name”: “devcluster”, + … + “docker_registry”: “<a href="https://registry.example.com";>https://registry.example.com</a>” + }]</p> + +<p>Details of how to use Docker via the Docker engine can be found in the +<a href="../../reference/configuration/#docker-object">Reference Documentation</a>. Please note that in order to +correctly execute processes inside a job, the Docker container must have Python 2.7 and potentitally +further Mesos dependencies installed. This limitation does not hold for Docker containers used via +the Mesos containerizer.</p> + +<p>For more information on launching Docker containers through the Docker containerizer, visit +<a href="http://mesos.apache.org/documentation/latest/docker-containerizer/";>Docker Containerizer</a></p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.19.1/features/cron-jobs/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/features/cron-jobs/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/features/cron-jobs/index.html (added) +++ aurora/site/publish/documentation/0.19.1/features/cron-jobs/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,289 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/features/cron-jobs/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="cron-jobs">Cron Jobs</h1> + +<p>Aurora supports execution of scheduled jobs on a Mesos cluster using cron-style syntax.</p> + +<ul> +<li><a href="#overview">Overview</a></li> +<li><a href="#collision-policies">Collision Policies</a></li> +<li><a href="#failure-recovery">Failure recovery</a></li> +<li><a href="#interacting-with-cron-jobs-via-the-aurora-cli">Interacting with cron jobs via the Aurora CLI</a> + +<ul> +<li><a href="#cron-schedule">cron schedule</a></li> +<li><a href="#cron-deschedule">cron deschedule</a></li> +<li><a href="#cron-start">cron start</a></li> +<li><a href="#job-killall-job-restart-job-kill">job killall, job restart, job kill</a></li> +</ul></li> +<li><a href="#technical-note-about-syntax">Technical Note About Syntax</a></li> +<li><a href="#caveats">Caveats</a> + +<ul> +<li><a href="#failovers">Failovers</a></li> +<li><a href="#collision-policy-is-best-effort">Collision policy is best-effort</a></li> +<li><a href="#timezone-configuration">Timezone Configuration</a></li> +</ul></li> +</ul> + +<h2 id="overview">Overview</h2> + +<p>A job is identified as a cron job by the presence of a +<code>cron_schedule</code> attribute containing a cron-style schedule in the +<a href="../../reference/configuration/#job-objects"><code>Job</code></a> object. Examples of cron schedules +include “every 5 minutes” (<code>*/5 * * * *</code>), “Fridays at 17:00” (<code>* 17 * * FRI</code>), and +“the 1st and 15th day of the month at 03:00” (<code>0 3 1,15 *</code>).</p> + +<p>Example (available in the <a href="../../getting-started/vagrant/">Vagrant environment</a>):</p> +<pre class="highlight plaintext"><code>$ cat /vagrant/examples/jobs/cron_hello_world.aurora +# A cron job that runs every 5 minutes. +jobs = [ + Job( + cluster = 'devcluster', + role = 'www-data', + environment = 'test', + name = 'cron_hello_world', + cron_schedule = '*/5 * * * *', + task = SimpleTask( + 'cron_hello_world', + 'echo "Hello world from cron, the time is now $(date --rfc-822)"'), + ), +] +</code></pre> + +<h2 id="collision-policies">Collision Policies</h2> + +<p>The <code>cron_collision_policy</code> field specifies the scheduler’s behavior when a new cron job is +triggered while an older run hasn’t finished. The scheduler has two policies available:</p> + +<ul> +<li><code>KILL_EXISTING</code>: The default policy - on a collision the old instances are killed and a instances with the current +configuration are started.</li> +<li><code>CANCEL_NEW</code>: On a collision the new run is cancelled.</li> +</ul> + +<p>Note that the use of <code>CANCEL_NEW</code> is likely a code smell - interrupted cron jobs should be able +to recover their progress on a subsequent invocation, otherwise they risk having their work queue +grow faster than they can process it.</p> + +<h2 id="failure-recovery">Failure recovery</h2> + +<p>Unlike with services, which aurora will always re-execute regardless of exit status, instances of +cron jobs retry according to the <code>max_task_failures</code> attribute of the +<a href="../../reference/configuration/#task-object">Task</a> object. To get “run-until-success” semantics, +set <code>max_task_failures</code> to <code>-1</code>.</p> + +<h2 id="interacting-with-cron-jobs-via-the-aurora-cli">Interacting with cron jobs via the Aurora CLI</h2> + +<p>Most interaction with cron jobs takes place using the <code>cron</code> subcommand. See <code>aurora cron -h</code> +for up-to-date usage instructions.</p> + +<h3 id="cron-schedule">cron schedule</h3> + +<p>Schedules a new cron job on the Aurora cluster for later runs or replaces the existing cron template +with a new one. Only future runs will be affected, any existing active tasks are left intact.</p> +<pre class="highlight plaintext"><code>$ aurora cron schedule devcluster/www-data/test/cron_hello_world /vagrant/examples/jobs/cron_hello_world.aurora +</code></pre> + +<h3 id="cron-deschedule">cron deschedule</h3> + +<p>Deschedules a cron job, preventing future runs but allowing current runs to complete.</p> +<pre class="highlight plaintext"><code>$ aurora cron deschedule devcluster/www-data/test/cron_hello_world +</code></pre> + +<h3 id="cron-start">cron start</h3> + +<p>Start a cron job immediately, outside of its normal cron schedule.</p> +<pre class="highlight plaintext"><code>$ aurora cron start devcluster/www-data/test/cron_hello_world +</code></pre> + +<h3 id="job-killall-job-restart-job-kill">job killall, job restart, job kill</h3> + +<p>Cron jobs create instances running on the cluster that you can interact with like normal Aurora +tasks with <code>job kill</code> and <code>job restart</code>.</p> + +<h2 id="technical-note-about-syntax">Technical Note About Syntax</h2> + +<p><code>cron_schedule</code> uses a restricted subset of BSD crontab syntax. While the +execution engine currently uses Quartz, the schedule parsing is custom, a subset of FreeBSD +<a href="http://www.freebsd.org/cgi/man.cgi?crontab(5)">crontab(5)</a> syntax. See +<a href="https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/cron/CrontabEntry.java#L106-L124";>the source</a> +for details.</p> + +<h2 id="caveats">Caveats</h2> + +<h3 id="failovers">Failovers</h3> + +<p>No failover recovery. Aurora does not record the latest minute it fired +triggers for across failovers. Therefore it’s possible to miss triggers +on failover. Note that this behavior may change in the future.</p> + +<p>It’s necessary to sync time between schedulers with something like <code>ntpd</code>. +Clock skew could cause double or missed triggers in the case of a failover.</p> + +<h3 id="collision-policy-is-best-effort">Collision policy is best-effort</h3> + +<p>Aurora aims to always have <em>at least one copy</em> of a given instance running at a time - it’s +an AP system, meaning it chooses Availability and Partition Tolerance at the expense of +Consistency.</p> + +<p>If your collision policy was <code>CANCEL_NEW</code> and a task has terminated but +Aurora has not noticed this Aurora will go ahead and create your new +task.</p> + +<p>If your collision policy was <code>KILL_EXISTING</code> and a task was marked <code>LOST</code> +but not yet GCed Aurora will go ahead and create your new task without +attempting to kill the old one (outside the GC interval).</p> + +<h3 id="timezone-configuration">Timezone Configuration</h3> + +<p>Cron timezone is configured indepdendently of JVM timezone with the <code>-cron_timezone</code> flag and +defaults to UTC.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.19.1/features/custom-executors/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/features/custom-executors/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/features/custom-executors/index.html (added) +++ aurora/site/publish/documentation/0.19.1/features/custom-executors/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,391 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/features/custom-executors/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="custom-executors">Custom Executors</h1> + +<p>If the need arises to use a Mesos executor other than the Thermos executor, the scheduler can be +configured to utilize a custom executor by specifying the <code>-custom_executor_config</code> flag. +The flag must be set to the path of a valid executor configuration file.</p> + +<p>The configuration file must be a valid <strong>JSON array</strong> and contain, at minimum, +one executor configuration including the name, command and resources fields and +must be pointed to by the <code>-custom_executor_config</code> flag when the scheduler is +started.</p> + +<h3 id="array-entry">Array Entry</h3> + +<table><thead> +<tr> +<th>Property</th> +<th>Description</th> +</tr> +</thead><tbody> +<tr> +<td>executor (required)</td> +<td>Description of executor.</td> +</tr> +<tr> +<td>task_prefix (required) )</td> +<td>Prefix given to tasks launched with this executor’s configuration.</td> +</tr> +<tr> +<td>volume_mounts (optional)</td> +<td>Volumes to be mounted in container running executor.</td> +</tr> +</tbody></table> + +<h4 id="executor">executor</h4> + +<table><thead> +<tr> +<th>Property</th> +<th>Description</th> +</tr> +</thead><tbody> +<tr> +<td>name (required)</td> +<td>Name of the executor.</td> +</tr> +<tr> +<td>command (required)</td> +<td>How to run the executor.</td> +</tr> +<tr> +<td>resources (required)</td> +<td>Overhead to use for each executor instance.</td> +</tr> +</tbody></table> + +<h4 id="command">command</h4> + +<table><thead> +<tr> +<th>Property</th> +<th>Description</th> +</tr> +</thead><tbody> +<tr> +<td>value (required)</td> +<td>The command to execute.</td> +</tr> +<tr> +<td>arguments (optional)</td> +<td>A list of arguments to pass to the command.</td> +</tr> +<tr> +<td>uris (optional)</td> +<td>List of resources to download into the task sandbox.</td> +</tr> +<tr> +<td>shell (optional)</td> +<td>Run executor via shell.</td> +</tr> +</tbody></table> + +<p>A note on the command property (from <a href="https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto";>mesos.proto</a>):</p> +<pre class="highlight plaintext"><code>1) If 'shell == true', the command will be launched via shell + (i.e., /bin/sh -c 'value'). The 'value' specified will be + treated as the shell command. The 'arguments' will be ignored. +2) If 'shell == false', the command will be launched by passing + arguments to an executable. The 'value' specified will be + treated as the filename of the executable. The 'arguments' + will be treated as the arguments to the executable. This is + similar to how POSIX exec families launch processes (i.e., + execlp(value, arguments(0), arguments(1), ...)). +</code></pre> + +<h5 id="uris-list">uris (list)</h5> + +<ul> +<li>Follows the <a href="http://mesos.apache.org/documentation/latest/fetcher/";>Mesos Fetcher schema</a></li> +</ul> + +<table><thead> +<tr> +<th>Property</th> +<th>Description</th> +</tr> +</thead><tbody> +<tr> +<td>value (required)</td> +<td>Path to the resource needed in the sandbox.</td> +</tr> +<tr> +<td>executable (optional)</td> +<td>Change resource to be executable via chmod.</td> +</tr> +<tr> +<td>extract (optional)</td> +<td>Extract files from packed or compressed archives into the sandbox.</td> +</tr> +<tr> +<td>cache (optional)</td> +<td>Use caching mechanism provided by Mesos for resources.</td> +</tr> +</tbody></table> + +<h4 id="resources-list">resources (list)</h4> + +<table><thead> +<tr> +<th>Property</th> +<th>Description</th> +</tr> +</thead><tbody> +<tr> +<td>name (required)</td> +<td>Name of the resource: cpus or mem.</td> +</tr> +<tr> +<td>type (required)</td> +<td>Type of resource. Should always be SCALAR.</td> +</tr> +<tr> +<td>scalar (required)</td> +<td>Value in float for cpus or int for mem (in MBs)</td> +</tr> +</tbody></table> + +<h3 id="volume_mounts-list">volume_mounts (list)</h3> + +<table><thead> +<tr> +<th>Property</th> +<th>Description</th> +</tr> +</thead><tbody> +<tr> +<td>host_path (required)</td> +<td>Host path to mount inside the container.</td> +</tr> +<tr> +<td>container_path (required)</td> +<td>Path inside the container where <code>host_path</code> will be mounted.</td> +</tr> +<tr> +<td>mode (required)</td> +<td>Mode in which to mount the volume, Read-Write (RW) or Read-Only (RO).</td> +</tr> +</tbody></table> + +<p>A sample configuration is as follows:</p> +<pre class="highlight json"><code><span style="background-color: #f8f8f8">[</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">{</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"executor"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="background-color: #f8f8f8">{</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"name"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"myExecutor"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"command"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="background-color: #f8f8f8">{</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"value"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"myExecutor.a"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"shell"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"false"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"arguments"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="background-color: #f8f8f8">[</span><span style="color: #bbbbbb"> + </span><span style="color: #d14">"localhost:2181"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #d14">"-verbose"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #d14">"-config myConfiguration.config"</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">],</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"uris"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="background-color: #f8f8f8">[</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">{</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"value"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"/dist/myExecutor.a"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"executable"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #000000;font-weight: bold">true</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"extract"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #000000;font-weight: bold">false</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"cache"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #000000;font-weight: bold">true</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">},</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">{</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"value"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"/home/user/myConfiguration.config"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"executable"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #000000;font-weight: bold">false</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"extract"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #000000;font-weight: bold">false</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"cache"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #000000;font-weight: bold">false</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">}</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">]</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">},</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"resources"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="background-color: #f8f8f8">[</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">{</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"name"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"cpus"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"type"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"SCALAR"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"scalar"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="background-color: #f8f8f8">{</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"value"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #009999">1.00</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">}</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">},</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">{</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"name"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"mem"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"type"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"SCALAR"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"scalar"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="background-color: #f8f8f8">{</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"value"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #009999">512</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">}</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">}</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">]</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">},</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"volume_mounts"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="background-color: #f8f8f8">[</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">{</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"mode"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"RO"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"container_path"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"/path/on/container"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"host_path"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"/path/to/host/directory"</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">},</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">{</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"mode"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"RW"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"container_path"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"/container"</span><span style="background-color: #f8f8f8">,</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"host_path"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"/host"</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">}</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">],</span><span style="color: #bbbbbb"> + </span><span style="color: #000080">"task_prefix"</span><span style="background-color: #f8f8f8">:</span><span style="color: #bbbbbb"> </span><span style="color: #d14">"my-executor-"</span><span style="color: #bbbbbb"> + </span><span style="background-color: #f8f8f8">}</span><span style="color: #bbbbbb"> +</span><span style="background-color: #f8f8f8">]</span><span style="color: #bbbbbb"> +</span></code></pre> + +<p>It should be noted that if you do not use Thermos or a Thermos based executor, links in the scheduler’s +Web UI for tasks will not work (at least for the time being). +Some information about launched tasks can still be accessed via the Mesos Web UI or via the Aurora Client.</p> + +<h3 id="using-a-custom-executor">Using a custom executor</h3> + +<p>At this time, it is not currently possible create a job that runs on a custom executor using the default +Aurora client. To allow the scheduler to pick the correct executor, the <code>JobConfiguration.TaskConfig.ExecutorConfig.name</code> +field must be set to match the name used in the custom executor configuration blob. (e.g. to run a job using myExecutor, +<code>JobConfiguration.TaskConfig.ExecutorConfig.name</code> must be set to <code>myExecutor</code>). While support for modifying +this field in Pystachio created, the easiest way to launch jobs with custom executors is to use +an existing custom Client such as <a href="https://github.com/rdelval/gorealis";>gorealis</a>.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.19.1/features/job-updates/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/features/job-updates/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/features/job-updates/index.html (added) +++ aurora/site/publish/documentation/0.19.1/features/job-updates/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,255 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/features/job-updates/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="aurora-job-updates">Aurora Job Updates</h1> + +<p><code>Job</code> configurations can be updated at any point in their lifecycle. +Usually updates are done incrementally using a process called a <em>rolling +upgrade</em>, in which Tasks are upgraded in small groups, one group at a +time. Updates are done using various Aurora Client commands.</p> + +<h2 id="rolling-job-updates">Rolling Job Updates</h2> + +<p>There are several sub-commands to manage job updates:</p> +<pre class="highlight plaintext"><code>aurora update start <job key> <configuration file> +aurora update info <job key> +aurora update pause <job key> +aurora update resume <job key> +aurora update abort <job key> +aurora update list <cluster> +</code></pre> + +<p>When you <code>start</code> a job update, the command will return once it has sent the +instructions to the scheduler. At that point, you may view detailed +progress for the update with the <code>info</code> subcommand, in addition to viewing +graphical progress in the web browser. You may also get a full listing of +in-progress updates in a cluster with <code>list</code>.</p> + +<p>Once an update has been started, you can <code>pause</code> to keep the update but halt +progress. This can be useful for doing things like debug a partially-updated +job to determine whether you would like to proceed. You can <code>resume</code> to +proceed.</p> + +<p>You may <code>abort</code> a job update regardless of the state it is in. This will +instruct the scheduler to completely abandon the job update and leave the job +in the current (possibly partially-updated) state.</p> + +<p>For a configuration update, the Aurora Scheduler calculates required changes +by examining the current job config state and the new desired job config. +It then starts a <em>rolling batched update process</em> by going through every batch +and performing these operations, in order:</p> + +<ul> +<li>If an instance is not present in the scheduler but is present in +the new config, then the instance is created.</li> +<li>If an instance is present in both the scheduler and the new config, then +the scheduler diffs both task configs. If it detects any changes, it +performs an instance update by killing the old config instance and adds +the new config instance.</li> +<li>If an instance is present in the scheduler but isn’t in the new config, +then that instance is killed.</li> +</ul> + +<p>The Aurora Scheduler continues through the instance list until all tasks are +updated and in <code>RUNNING</code>. If the scheduler determines the update is not going +well (based on the criteria specified in the UpdateConfig), it cancels the update.</p> + +<p>Update cancellation runs a procedure similar to the described above +update sequence, but in reverse order. New instance configs are swapped +with old instance configs and batch updates proceed backwards +from the point where the update failed. E.g.; (0,1,2) (3,4,5) (6,7, +8-FAIL) results in a rollback in order (8,7,6) (5,4,3) (2,1,0).</p> + +<p>For details on how to control a job update, please see the +<a href="../../reference/configuration/#updateconfig-objects">UpdateConfig</a> configuration object.</p> + +<h2 id="coordinated-job-updates">Coordinated Job Updates</h2> + +<p>Some Aurora services may benefit from having more control over updates by explicitly +acknowledging (“heartbeating”) job update progress. This may be helpful for mission-critical +service updates where explicit job health monitoring is vital during the entire job update +lifecycle. Such job updates would rely on an external service (or a custom client) periodically +pulsing an active coordinated job update via a +<a href="https://github.com/apache/aurora/blob/rel/0.19.1/api/src/main/thrift/org/apache/aurora/gen/api.thrift";>pulseJobUpdate RPC</a>.</p> + +<p>A coordinated update is defined by setting a positive +<a href="../../reference/configuration/#updateconfig-objects">pulse<em>interval</em>secs</a> value in job configuration +file. If no pulses are received within specified interval the update will be blocked. A blocked +update is unable to continue rolling forward (or rolling back) but retains its active status. +It may only be unblocked by a fresh <code>pulseJobUpdate</code> call.</p> + +<p>NOTE: A coordinated update starts in <code>ROLL_FORWARD_AWAITING_PULSE</code> state and will not make any +progress until the first pulse arrives. However, a paused update (<code>ROLL_FORWARD_PAUSED</code> or +<code>ROLL_BACK_PAUSED</code>) is still considered active and upon resuming will immediately make progress +provided the pulse interval has not expired.</p> + +<h2 id="canary-deployments">Canary Deployments</h2> + +<p>Canary deployments are a pattern for rolling out updates to a subset of job instances, +in order to test different code versions alongside the actual production job. +It is a risk-mitigation strategy for job owners and commonly used in a form where +job instance 0 runs with a different configuration than the instances 1-N.</p> + +<p>For example, consider a job with 4 instances that each +request 1 core of cpu, 1 GB of RAM, and 1 GB of disk space as specified +in the configuration file <code>hello_world.aurora</code>. If you want to +update it so it requests 2 GB of RAM instead of 1. You can create a new +configuration file to do that called <code>new_hello_world.aurora</code> and +issue</p> +<pre class="highlight plaintext"><code>aurora update start <job_key_value>/0-1 new_hello_world.aurora +</code></pre> + +<p>This results in instances 0 and 1 having 1 cpu, 2 GB of RAM, and 1 GB of disk space, +while instances 2 and 3 have 1 cpu, 1 GB of RAM, and 1 GB of disk space. If instance 3 +dies and restarts, it restarts with 1 cpu, 1 GB RAM, and 1 GB disk space.</p> + +<p>So that means there are two simultaneous task configurations for the same job +at the same time, just valid for different ranges of instances. While this isn’t a recommended +pattern, it is valid and supported by the Aurora scheduler.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.19.1/features/mesos-fetcher/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/features/mesos-fetcher/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/features/mesos-fetcher/index.html (added) +++ aurora/site/publish/documentation/0.19.1/features/mesos-fetcher/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,209 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/features/mesos-fetcher/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="mesos-fetcher">Mesos Fetcher</h1> + +<p>Mesos has support for downloading resources into the sandbox through the +use of the <a href="http://mesos.apache.org/documentation/latest/fetcher/";>Mesos Fetcher</a></p> + +<p>Aurora supports passing URIs to the Mesos Fetcher dynamically by including +a list of URIs in job submissions.</p> + +<h2 id="how-to-use">How to use</h2> + +<p>The scheduler flag <code>-enable_mesos_fetcher</code> must be set to true.</p> + +<p>Currently only the scheduler side of this feature has been implemented +so a modification to the existing client, or a custom Thrift client are required +to make use of this feature.</p> + +<p>If using a custom Thrift client, the list of URIs must be included in TaskConfig +as the <code>mesosFetcherUris</code> field.</p> + +<p>Each Mesos Fetcher URI has the following data members:</p> + +<table><thead> +<tr> +<th>Property</th> +<th>Description</th> +</tr> +</thead><tbody> +<tr> +<td>value (required)</td> +<td>Path to the resource needed in the sandbox.</td> +</tr> +<tr> +<td>extract (optional)</td> +<td>Extract files from packed or compressed archives into the sandbox.</td> +</tr> +<tr> +<td>cache (optional)</td> +<td>Use caching mechanism provided by Mesos for resources.</td> +</tr> +</tbody></table> + +<p>Note that this structure is very similar to the one provided for downloading +resources needed for a <a href="../../operations/configuration/">custom executor</a>.</p> + +<p>This is because both features use the Mesos fetcher to retrieve resources into +the sandbox. However, one, the custom executor feature, has a static set of URIs +set in the server side, and the other, the Mesos Fetcher feature, is a dynamic set +of URIs set at the time of job submission.</p> + +<h2 id="security-implications">Security Implications</h2> + +<p>There are security implications that must be taken into account when enabling this feature. +<strong>Enabling this feature may potentially enable any job submitting user to perform a privilege escalation.</strong></p> + +<p>Until a more through solution is created, one step that has been taken to mitigate this issue +is to statically mark every user submitted URI as non-executable. This is in contrast to the set of URIs +set in the custom executor feature which may mark any URI as executable.</p> + +<p>If the need arises to mark a downloaded URI as executable, please consider using the custom executor feature.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html>
