Added: aurora/site/publish/documentation/0.18.1/development/design/command-hooks/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.18.1/development/design/command-hooks/index.html?rev=1813982&view=auto ============================================================================== --- aurora/site/publish/documentation/0.18.1/development/design/command-hooks/index.html (added) +++ aurora/site/publish/documentation/0.18.1/development/design/command-hooks/index.html Wed Nov 1 18:39:52 2017 @@ -0,0 +1,241 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/design/command-hooks/'" + value="0.18.1"> + <option value="0.18.1" + selected="selected"> + 0.18.1 + (latest) + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="command-hooks-for-the-aurora-client">Command Hooks for the Aurora Client</h1> + +<h2 id="introduction-motivation">Introduction/Motivation</h2> + +<p>We’ve got hooks in the client that surround API calls. These are +pretty awkward, because they don’t correlate with user actions. For +example, suppose we wanted a policy that said users weren’t allowed to +kill all instances of a production job at once.</p> + +<p>Right now, all that we could hook would be the “killJob” api call. But +kill (at least in newer versions of the client) normally runs in +batches. If a user called killall, what we would see on the API level +is a series of “killJob” calls, each of which specified a batch of +instances. We woudn’t be able to distinguish between really killing +all instances of a job (which is forbidden under this policy), and +carefully killing in batches (which is permitted.) In each case, the +hook would just see a series of API calls, and couldn’t find out what +the actual command being executed was!</p> + +<p>For most policy enforcement, what we really want to be able to do is +look at and vet the commands that a user is performing, not the API +calls that the client uses to implement those commands.</p> + +<p>So I propose that we add a new kind of hooks, which surround noun/verb +commands. A hook will register itself to handle a collection of (noun, +verb) pairs. Whenever any of those noun/verb commands are invoked, the +hooks methods will be called around the execution of the verb. A +pre-hook will have the ability to reject a command, preventing the +verb from being executed.</p> + +<h2 id="registering-hooks">Registering Hooks</h2> + +<p>These hooks will be registered via configuration plugins. A configuration plugin +can register hooks using an API. Hooks registered this way are, effectively, +hardwired into the client executable.</p> + +<p>The order of execution of hooks is unspecified: they may be called in +any order. There is no way to guarantee that one hook will execute +before some other hook.</p> + +<h3 id="global-hooks">Global Hooks</h3> + +<p>Commands registered by the python call are called <em>global</em> hooks, +because they will run for all configurations, whether or not they +specify any hooks in the configuration file.</p> + +<p>In the implementation, hooks are registered in the module +<code>apache.aurora.client.cli.command_hooks</code>, using the class +<code>GlobalCommandHookRegistry</code>. A global hook can be registered by calling +<code>GlobalCommandHookRegistry.register_command_hook</code> in a configuration plugin.</p> + +<h3 id="the-api">The API</h3> +<pre class="highlight objective_c"><code><span style="background-color: #f8f8f8">class</span> <span style="background-color: #f8f8f8">CommandHook</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">object</span><span style="background-color: #f8f8f8">)</span> + <span style="color: #000000;font-weight: bold">@property</span> + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">name</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #d14">"""Returns a name for the hook."</span> + + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">get_nouns</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #d14">"""Return the nouns that have verbs that should invoke this hook."""</span> + + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">get_verbs</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">noun</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #d14">"""Return the verbs for a particular noun that should invoke his hook."""</span> + + <span style="color: #a61717;background-color: #e3d2d2">@abstractmethod</span> + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">pre_command</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">noun</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">verb</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">context</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">commandline</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #d14">"""Execute a hook before invoking a verb.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * noun: the noun being invoked.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * verb: the verb being invoked.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * context: the context object that will be used to invoke the verb.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> The options object will be initialized before calling the hook</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * commandline: the original argv collection used to invoke the client.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> Returns: True if the command should be allowed to proceed; False if the command</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> should be rejected.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> """</span> + + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">post_command</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">noun</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">verb</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">context</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">commandline</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">result</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #d14">"""Execute a hook after invoking a verb.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * noun: the noun being invoked.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * verb: the verb being invoked.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * context: the context object that will be used to invoke the verb.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> The options object will be initialized before calling the hook</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * commandline: the original argv collection used to invoke the client.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * result: the result code returned by the verb.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> Returns: nothing</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> """</span> + +<span style="background-color: #f8f8f8">class</span> <span style="background-color: #f8f8f8">GlobalCommandHookRegistry</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">object</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #a61717;background-color: #e3d2d2">@classmethod</span> + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">register_command_hook</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">hook</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="background-color: #f8f8f8">pass</span> +</code></pre> + +<h3 id="skipping-hooks">Skipping Hooks</h3> + +<p>To skip a hook, a user uses a command-line option, <code>--skip-hooks</code>. The option can either +specify specific hooks to skip, or “all”:</p> + +<ul> +<li><code>aurora --skip-hooks=all job create east/bozo/devel/myjob</code> will create a job +without running any hooks.</li> +<li><code>aurora --skip-hooks=test,iq create east/bozo/devel/myjob</code> will create a job, +and will skip only the hooks named “test” and “iq”.</li> +</ul> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html>
Added: aurora/site/publish/documentation/0.18.1/development/scheduler/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.18.1/development/scheduler/index.html?rev=1813982&view=auto ============================================================================== --- aurora/site/publish/documentation/0.18.1/development/scheduler/index.html (added) +++ aurora/site/publish/documentation/0.18.1/development/scheduler/index.html Wed Nov 1 18:39:52 2017 @@ -0,0 +1,246 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/scheduler/'" + value="0.18.1"> + <option value="0.18.1" + selected="selected"> + 0.18.1 + (latest) + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="developing-the-aurora-scheduler">Developing the Aurora Scheduler</h1> + +<p>The Aurora scheduler is written in Java code and built with <a href="http://gradle.org";>Gradle</a>.</p> + +<h1 id="prerequisite">Prerequisite</h1> + +<p>When using Apache Aurora checked out from the source repository or the binary +distribution, the Gradle wrapper and JavaScript dependencies are provided. +However, you need to manually install them when using the source release +downloads:</p> + +<ol> +<li>Install Gradle following the instructions on the <a href="http://gradle.org";>Gradle web site</a></li> +<li><p>From the root directory of the Apache Aurora project generate the Gradle +wrapper by running:</p> + +<p>gradle wrapper</p></li> +</ol> + +<h1 id="getting-started">Getting Started</h1> + +<p>You will need Java 8 installed and on your <code>PATH</code> or unzipped somewhere with <code>JAVA_HOME</code> set. Then</p> +<pre class="highlight plaintext"><code>./gradlew tasks +</code></pre> + +<p>will bootstrap the build system and show available tasks. This can take a while the first time you +run it but subsequent runs will be much faster due to cached artifacts.</p> + +<h2 id="running-the-tests">Running the Tests</h2> + +<p>Aurora has a comprehensive unit test suite. To run the tests use</p> +<pre class="highlight plaintext"><code>./gradlew build +</code></pre> + +<p>Gradle will only re-run tests when dependencies of them have changed. To force a re-run of all +tests use</p> +<pre class="highlight plaintext"><code>./gradlew clean build +</code></pre> + +<h2 id="running-the-build-with-code-quality-checks">Running the build with code quality checks</h2> + +<p>To speed up development iteration, the plain gradle commands will not run static analysis tools. +However, you should run these before posting a review diff, and <strong>always</strong> run this before pushing a +commit to origin/master.</p> +<pre class="highlight plaintext"><code>./gradlew build -Pq +</code></pre> + +<h2 id="running-integration-tests">Running integration tests</h2> + +<p>To run the same tests that are run in the Apache Aurora continuous integration +environment:</p> +<pre class="highlight plaintext"><code>./build-support/jenkins/build.sh +</code></pre> + +<p>In addition, there is an end-to-end test that runs a suite of aurora commands +using a virtual cluster:</p> +<pre class="highlight plaintext"><code>./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh +</code></pre> + +<h2 id="creating-a-bundle-for-deployment">Creating a bundle for deployment</h2> + +<p>Gradle can create a zip file containing Aurora, all of its dependencies, and a launch script with</p> +<pre class="highlight plaintext"><code>./gradlew distZip +</code></pre> + +<p>or a tar file containing the same files with</p> +<pre class="highlight plaintext"><code>./gradlew distTar +</code></pre> + +<p>The output file will be written to <code>dist/distributions/aurora-scheduler.zip</code> or +<code>dist/distributions/aurora-scheduler.tar</code>.</p> + +<h1 id="developing-aurora-java-code">Developing Aurora Java code</h1> + +<h2 id="setting-up-an-ide">Setting up an IDE</h2> + +<p>Gradle can generate project files for your IDE. To generate an IntelliJ IDEA project run</p> +<pre class="highlight plaintext"><code>./gradlew idea +</code></pre> + +<p>and import the generated <code>aurora.ipr</code> file.</p> + +<h2 id="adding-or-upgrading-a-dependency">Adding or Upgrading a Dependency</h2> + +<p>New dependencies can be added from Maven central by adding a <code>compile</code> dependency to <code>build.gradle</code>. +For example, to add a dependency on <code>com.example</code>’s <code>example-lib</code> 1.0 add this block:</p> +<pre class="highlight plaintext"><code>compile 'com.example:example-lib:1.0' +</code></pre> + +<p>NOTE: Anyone thinking about adding a new dependency should first familiarize themselves with the +Apache Foundation’s third-party licensing +<a href="http://www.apache.org/legal/resolved.html#category-x";>policy</a>.</p> + +<h1 id="developing-the-aurora-build-system">Developing the Aurora Build System</h1> + +<h2 id="bootstrapping-gradle">Bootstrapping Gradle</h2> + +<p>The following files were autogenerated by <code>gradle wrapper</code> using gradle’s +<a href="http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.wrapper.Wrapper.html";>Wrapper</a> plugin and +should not be modified directly:</p> +<pre class="highlight plaintext"><code>./gradlew +./gradlew.bat +./gradle/wrapper/gradle-wrapper.jar +./gradle/wrapper/gradle-wrapper.properties +</code></pre> + +<p>To upgrade Gradle unpack the new version somewhere, run <code>/path/to/new/gradle wrapper</code> in the +repository root and commit the changed files.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.18.1/development/thermos/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.18.1/development/thermos/index.html?rev=1813982&view=auto ============================================================================== --- aurora/site/publish/documentation/0.18.1/development/thermos/index.html (added) +++ aurora/site/publish/documentation/0.18.1/development/thermos/index.html Wed Nov 1 18:39:52 2017 @@ -0,0 +1,257 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/thermos/'" + value="0.18.1"> + <option value="0.18.1" + selected="selected"> + 0.18.1 + (latest) + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<p>The Python components of Aurora are built using <a href="https://pantsbuild.github.io";>Pants</a>.</p> + +<h1 id="python-build-conventions">Python Build Conventions</h1> + +<p>The Python code is laid out according to the following conventions:</p> + +<ol> +<li><p>1 <code>BUILD</code> per 3rd level directory. For a list of current top-level packages run:</p> +<pre class="highlight plaintext"><code>% find src/main/python -maxdepth 3 -mindepth 3 -type d |\ +while read dname; do echo $dname |\ + sed 's@src/main/python/\(.*\)/\(.*\)/\(.*\).*@\1.\2.\3@'; done +</code></pre></li> +<li><p>Each <code>BUILD</code> file exports 1 +<a href="https://pantsbuild.github.io/build_dictionary.html#bdict_python_library";><code>python_library</code></a> +that provides a +<a href="https://pantsbuild.github.io/build_dictionary.html#setup_py";><code>setup_py</code></a> +containing each +<a href="https://pantsbuild.github.io/build_dictionary.html#python_binary";><code>python_binary</code></a> +in the <code>BUILD</code> file, named the same as the directory it’s in so that it can be referenced +without a ’:’ character. The <code>sources</code> field in the <code>python_library</code> will almost always be +<code>rglobs('*.py')</code>.</p></li> +<li><p>Other BUILD files may only depend on this single public <code>python_library</code> +target. Any other target is considered a private implementation detail and +should be prefixed with an <code>_</code>.</p></li> +<li><p><code>python_binary</code> targets are always named the same as the exported console script.</p></li> +<li><p><code>python_binary</code> targets must have identical <code>dependencies</code> to the <code>python_library</code> exported +by the package and must use <code>entry_point</code>.</p> + +<p>The means a PEX file generated by pants will contain exactly the same files that will be +available on the <code>PYTHONPATH</code> in the case of <code>pip install</code> of the corresponding library +target. This will help our migration off of Pants in the future.</p></li> +</ol> + +<h2 id="annotated-example-apache-thermos-runner">Annotated example - apache.thermos.runner</h2> +<pre class="highlight plaintext"><code>% find src/main/python/apache/thermos/runner +src/main/python/apache/thermos/runner +src/main/python/apache/thermos/runner/__init__.py +src/main/python/apache/thermos/runner/thermos_runner.py +src/main/python/apache/thermos/runner/BUILD +% cat src/main/python/apache/thermos/runner/BUILD +# License boilerplate omitted +import os + + +# Private target so that a setup_py can exist without a circular dependency. Only targets within +# this file should depend on this. +python_library( + name = '_runner', + # The target covers every python file under this directory and subdirectories. + sources = rglobs('*.py'), + dependencies = [ + '3rdparty/python:twitter.common.app', + '3rdparty/python:twitter.common.log', + # Source dependencies are always referenced without a ':'. + 'src/main/python/apache/thermos/common', + 'src/main/python/apache/thermos/config', + 'src/main/python/apache/thermos/core', + ], +) + +# Binary target for thermos_runner.pex. Nothing should depend on this - it's only used as an +# argument to ./pants binary. +python_binary( + name = 'thermos_runner', + # Use entry_point, not source so the files used here are the same ones tests see. + entry_point = 'apache.thermos.bin.thermos_runner', + dependencies = [ + # Notice that we depend only on the single private target from this BUILD file here. + ':_runner', + ], +) + +# The public library that everyone importing the runner symbols uses. +# The test targets and any other dependent source code should depend on this. +python_library( + name = 'runner', + dependencies = [ + # Again, notice that we depend only on the single private target from this BUILD file here. + ':_runner', + ], + # We always provide a setup_py. This will cause any dependee libraries to automatically + # reference this library in their requirements.txt rather than copy the source files into their + # sdist. + provides = setup_py( + # Conventionally named and versioned. + name = 'apache.thermos.runner', + version = open(os.path.join(get_buildroot(), '.auroraversion')).read().strip().upper(), + ).with_binaries({ + # Every binary in this file should also be repeated here. + # Always use the dict-form of .with_binaries so that commands with dashes in their names are + # supported. + # The console script name is always the same as the PEX with .pex stripped. + 'thermos_runner': ':thermos_runner', + }), +) +</code></pre> + +<h1 id="thermos-test-resources">Thermos Test resources</h1> + +<p>The Aurora source repository and distributions contain several +<a href="../../src/test/resources/org/apache/thermos/root/checkpoints">binary files</a> to +qualify the backwards-compatibility of thermos with checkpoint data. Since +thermos persists state to disk, to be read by the thermos observer), it is important that we have +tests that prevent regressions affecting the ability to parse previously-written data.</p> + +<p>The files included represent persisted checkpoints that exercise different +features of thermos. The existing files should not be modified unless +we are accepting backwards incompatibility, such as with a major release.</p> + +<p>It is not practical to write source code to generate these files on the fly, +as source would be vulnerable to drift (e.g. due to refactoring) in ways +that would undermine the goal of ensuring backwards compatibility.</p> + +<p>The most common reason to add a new checkpoint file would be to provide +coverage for new thermos features that alter the data format. This is +accomplished by writing and running a +<a href="../../reference/configuration/">job configuration</a> that exercises the feature, and +copying the checkpoint file from the sandbox directory, by default this is +<code>/var/run/thermos/checkpoints/<aurora task id></code>.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.18.1/development/thrift/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.18.1/development/thrift/index.html?rev=1813982&view=auto ============================================================================== --- aurora/site/publish/documentation/0.18.1/development/thrift/index.html (added) +++ aurora/site/publish/documentation/0.18.1/development/thrift/index.html Wed Nov 1 18:39:52 2017 @@ -0,0 +1,192 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/thrift/'" + value="0.18.1"> + <option value="0.18.1" + selected="selected"> + 0.18.1 + (latest) + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="thrift">Thrift</h1> + +<p>Aurora uses <a href="https://thrift.apache.org/";>Apache Thrift</a> for representing structured data in +client/server RPC protocol as well as for internal data storage. While Thrift is capable of +correctly handling additions and renames of the existing members, field removals must be done +carefully to ensure backwards compatibility and provide predictable deprecation cycle. This +document describes general guidelines for making Thrift schema changes to the existing fields in +<a href="https://github.com/apache/aurora/blob/rel/0.18.1/api/src/main/thrift/org/apache/aurora/gen/api.thrift";>api.thrift</a>.</p> + +<p>It is highly recommended to go through the +<a href="http://diwakergupta.github.io/thrift-missing-guide/";>Thrift: The Missing Guide</a> first to refresh on +basic Thrift schema concepts.</p> + +<h2 id="checklist">Checklist</h2> + +<p>Every existing Thrift schema modification is unique in its requirements and must be analyzed +carefully to identify its scope and expected consequences. The following checklist may help in that +analysis: +* Is this a new field/struct? If yes, go ahead +* Is this a pure field/struct rename without any type/structure change? If yes, go ahead and rename +* Anything else, read further to make sure your change is properly planned</p> + +<h2 id="deprecation-cycle">Deprecation cycle</h2> + +<p>Any time a breaking change (e.g.: field replacement or removal) is required, the following cycle +must be followed:</p> + +<h3 id="vcurrent">vCurrent</h3> + +<p>Change is applied in a way that does not break scheduler/client with this version to +communicate with scheduler/client from vCurrent-1. +* Do not remove or rename the old field +* Add a new field as an eventual replacement of the old one and implement a dual read/write +anywhere the old field is used. If a thrift struct is mapped in the DB store make sure both columns +are marked as <code>NOT NULL</code> +* Check <a href="https://github.com/apache/aurora/blob/rel/0.18.1/api/src/main/thrift/org/apache/aurora/gen/storage.thrift";>storage.thrift</a> to see if +the affected struct is stored in Aurora scheduler storage. If so, it’s almost certainly also +necessary to perform a <a href="../db-migration/">DB migration</a>. +* Add a deprecation jira ticket into the vCurrent+1 release candidate +* Add a TODO for the deprecated field mentioning the jira ticket</p> + +<h3 id="vcurrent-1">vCurrent+1</h3> + +<p>Finalize the change by removing the deprecated fields from the Thrift schema. +* Drop any dual read/write routines added in the previous version +* Remove thrift backfilling in scheduler +* Remove the deprecated Thrift field</p> + +<h2 id="testing">Testing</h2> + +<p>It’s always advisable to test your changes in the local vagrant environment to build more +confidence that you change is backwards compatible. It’s easy to simulate different +client/scheduler versions by playing with <code>aurorabuild</code> command. See <a href="../../getting-started/vagrant/">this document</a> +for more.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.18.1/development/ui/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.18.1/development/ui/index.html?rev=1813982&view=auto ============================================================================== --- aurora/site/publish/documentation/0.18.1/development/ui/index.html (added) +++ aurora/site/publish/documentation/0.18.1/development/ui/index.html Wed Nov 1 18:39:52 2017 @@ -0,0 +1,182 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/ui/'" + value="0.18.1"> + <option value="0.18.1" + selected="selected"> + 0.18.1 + (latest) + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="developing-the-aurora-scheduler-ui">Developing the Aurora Scheduler UI</h1> + +<h2 id="installing-bower-optional">Installing bower (optional)</h2> + +<p>Third party JS libraries used in Aurora (located at 3rdparty/javascript/bower_components) are +managed by bower, a JS dependency manager. Bower is only required if you plan to add, remove or +update JS libraries. Bower can be installed using the following command:</p> +<pre class="highlight plaintext"><code>npm install -g bower +</code></pre> + +<p>Bower depends on node.js and npm. The easiest way to install node on a mac is via brew:</p> +<pre class="highlight plaintext"><code>brew install node +</code></pre> + +<p>For more node.js installation options refer to <a href="https://github.com/joyent/node/wiki/Installation";>https://github.com/joyent/node/wiki/Installation</a>.</p> + +<p>More info on installing and using bower can be found at: <a href="http://bower.io/";>http://bower.io/</a>. Once installed, you can +use the following commands to view and modify the bower repo at +3rdparty/javascript/bower_components</p> +<pre class="highlight plaintext"><code>bower list +bower install <library name> +bower remove <library name> +bower update <library name> +bower help +</code></pre> + +<h2 id="faster-iteration-in-vagrant">Faster Iteration in Vagrant</h2> + +<p>The scheduler serves UI assets from the classpath. For production deployments this means the assets +are served from within a jar. However, for faster development iteration, the vagrant image is +configured to add the <code>scheduler</code> subtree of <code>/vagrant/dist/resources/main</code> to the head of +<code>CLASSPATH</code>. This path is configured as a shared filesystem to the path on the host system where +your Aurora repository lives. This means that any updates under <code>dist/resources/main/scheduler</code> in +your checkout will be reflected immediately in the UI served from within the vagrant image.</p> + +<p>The one caveat to this is that this path is under <code>dist</code> not <code>src</code>. This is because the assets must +be processed by gradle before they can be served. So, unfortunately, you cannot just save your local +changes and see them reflected in the UI, you must first run <code>./gradlew processResources</code>. This is +less than ideal, but better than having to restart the scheduler after every change. Additionally, +gradle makes this process somewhat easier with the use of the <code>--continuous</code> flag. If you run: +<code>./gradlew processResources --continuous</code> gradle will monitor the filesystem for changes and run the +task automatically as necessary. This doesn’t quite provide hot-reload capabilities, but it does +allow for <5s from save to changes being visibile in the UI with no further action required on the +part of the developer.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.18.1/features/constraints/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.18.1/features/constraints/index.html?rev=1813982&view=auto ============================================================================== --- aurora/site/publish/documentation/0.18.1/features/constraints/index.html (added) +++ aurora/site/publish/documentation/0.18.1/features/constraints/index.html Wed Nov 1 18:39:52 2017 @@ -0,0 +1,252 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/features/constraints/'" + value="0.18.1"> + <option value="0.18.1" + selected="selected"> + 0.18.1 + (latest) + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="scheduling-constraints">Scheduling Constraints</h1> + +<p>By default, Aurora will pick any random agent with sufficient resources +in order to schedule a task. This scheduling choice can be further +restricted with the help of constraints.</p> + +<h2 id="mesos-attributes">Mesos Attributes</h2> + +<p>Data centers are often organized with hierarchical failure domains. Common failure domains +include hosts, racks, rows, and PDUs. If you have this information available, it is wise to tag +the Mesos agent with them as +<a href="https://mesos.apache.org/documentation/attributes-resources/";>attributes</a>.</p> + +<p>The Mesos agent <code>--attributes</code> command line argument can be used to mark agents with +static key/value pairs, so called attributes (not to be confused with <code>--resources</code>, which are +dynamic and accounted).</p> + +<p>For example, consider the host <code>cluster1-aaa-03-sr2</code> and its following attributes (given in +key:value format): <code>host:cluster1-aaa-03-sr2</code> and <code>rack:aaa</code>.</p> + +<p>Aurora makes these attributes available for matching with scheduling constraints.</p> + +<h2 id="limit-constraints">Limit Constraints</h2> + +<p>Limit constraints allow to control machine diversity using constraints. The below +constraint ensures that no more than two instances of your job may run on a single host. +Think of this as a “group by” limit.</p> +<pre class="highlight plaintext"><code>Service( + name = 'webservice', + role = 'www-data', + constraints = { + 'host': 'limit:2', + } + ... +) +</code></pre> + +<p>Likewise, you can use constraints to control rack diversity, e.g. at +most one task per rack:</p> +<pre class="highlight plaintext"><code>constraints = { + 'rack': 'limit:1', +} +</code></pre> + +<p>Use these constraints sparingly as they can dramatically reduce Tasks’ schedulability. +Further details are available in the reference documentation on +<a href="../../reference/configuration/#specifying-scheduling-constraints">Scheduling Constraints</a>.</p> + +<h2 id="value-constraints">Value Constraints</h2> + +<p>Value constraints can be used to express that a certain attribute with a certain value +should be present on a Mesos agent. For example, the following job would only be +scheduled on nodes that claim to have an <code>SSD</code> as their disk.</p> +<pre class="highlight plaintext"><code>Service( + name = 'webservice', + role = 'www-data', + constraints = { + 'disk': 'SSD', + } + ... +) +</code></pre> + +<p>Further details are available in the reference documentation on +<a href="../../reference/configuration/#specifying-scheduling-constraints">Scheduling Constraints</a>.</p> + +<h2 id="running-stateful-services">Running stateful services</h2> + +<p>Aurora is best suited to run stateless applications, but it also accommodates for stateful services +like databases, or services that otherwise need to always run on the same machines.</p> + +<h3 id="dedicated-attribute">Dedicated attribute</h3> + +<p>Most of the Mesos attributes arbitrary and available for custom use. There is one exception, +though: the <code>dedicated</code> attribute. Aurora treats this specially, and only allows matching jobs to +run on these machines, and will only schedule matching jobs on these machines.</p> + +<h4 id="syntax">Syntax</h4> + +<p>The dedicated attribute has semantic meaning. The format is <code>$role(/.*)?</code>. When a job is created, +the scheduler requires that the <code>$role</code> component matches the <code>role</code> field in the job +configuration, and will reject the job creation otherwise. The remainder of the attribute is +free-form. We’ve developed the idiom of formatting this attribute as <code>$role/$job</code>, but do not +enforce this. For example: a job <code>devcluster/www-data/prod/hello</code> with a dedicated constraint set as +<code>www-data/web.multi</code> will have its tasks scheduled only on Mesos agents configured with: +<code>--attributes=dedicated:www-data/web.multi</code>.</p> + +<p>A wildcard (<code>*</code>) may be used for the role portion of the dedicated attribute, which will allow any +owner to elect for a job to run on the host(s). For example: tasks from both +<code>devcluster/www-data/prod/hello</code> and <code>devcluster/vagrant/test/hello</code> with a dedicated constraint +formatted as <code>*/web.multi</code> will be scheduled only on Mesos agents configured with +<code>--attributes=dedicated:*/web.multi</code>. This may be useful when assembling a virtual cluster of +machines sharing the same set of traits or requirements.</p> + +<h5 id="example">Example</h5> + +<p>Consider the following agent command line:</p> +<pre class="highlight plaintext"><code>mesos-slave --attributes="dedicated:db_team/redis" ... +</code></pre> + +<p>And this job configuration:</p> +<pre class="highlight plaintext"><code>Service( + name = 'redis', + role = 'db_team', + constraints = { + 'dedicated': 'db_team/redis' + } + ... +) +</code></pre> + +<p>The job configuration is indicating that it should only be scheduled on agents with the attribute +<code>dedicated:db_team/redis</code>. Additionally, Aurora will prevent any tasks that do <em>not</em> have that +constraint from running on those agents.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.18.1/features/containers/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.18.1/features/containers/index.html?rev=1813982&view=auto ============================================================================== --- aurora/site/publish/documentation/0.18.1/features/containers/index.html (added) +++ aurora/site/publish/documentation/0.18.1/features/containers/index.html Wed Nov 1 18:39:52 2017 @@ -0,0 +1,264 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/features/containers/'" + value="0.18.1"> + <option value="0.18.1" + selected="selected"> + 0.18.1 + (latest) + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="containers">Containers</h1> + +<p>Aurora supports several containerizers, notably the Mesos containerizer and the Docker +containerizer. The Mesos containerizer uses native OS features directly to provide isolation between +containers, while the Docker containerizer delegates container management to the Docker engine.</p> + +<p>The support for launching container images via both containerizers has to be +<a href="../../operations/configuration/#containers">enabled by a cluster operator</a>.</p> + +<h2 id="mesos-containerizer">Mesos Containerizer</h2> + +<p>The Mesos containerizer is the native Mesos containerizer solution. It allows tasks to be +run with an array of <a href="../resource-isolation/">pluggable isolators</a> and can launch tasks using +<a href="https://github.com/docker/docker/blob/master/image/spec/v1.md";>Docker</a> images, +<a href="https://github.com/appc/spec/blob/master/SPEC.md";>AppC</a> images, or directly on the agent host +filesystem.</p> + +<p>The following example (available in our <a href="../../getting-started/vagrant/">Vagrant environment</a>) +launches a hello world example within a <code>debian/jessie</code> Docker image:</p> +<pre class="highlight plaintext"><code>$ cat /vagrant/examples/jobs/hello_docker_image.aurora +hello_loop = Process( + name = 'hello', + cmdline = """ + while true; do + echo hello world + sleep 10 + done + """) + +task = Task( + processes = [hello_loop], + resources = Resources(cpu=1, ram=1*MB, disk=8*MB) +) + +jobs = [ + Service( + cluster = 'devcluster', + environment = 'devel', + role = 'www-data', + name = 'hello_docker_image', + task = task, + container = Mesos(image=DockerImage(name='debian', tag='jessie')) + ) +] +</code></pre> + +<p>Docker and Appc images are designated using an appropriate <code>image</code> property of the <code>Mesos</code> +configuration object. If either <code>container</code> or <code>image</code> is left unspecified, the host filesystem +will be used. Further details of how to specify images can be found in the +<a href="../../reference/configuration/#mesos-object">Reference Documentation</a>.</p> + +<p>By default, Aurora launches processes as the Linux user named like the used role (e.g. <code>www-data</code> +in the example above). This user has to exist on the host filesystem. If it does not exist within +the container image, it will be created automatically. Otherwise, this user and its primary group +has to exist in the image with matching uid/gid.</p> + +<p>For more information on the Mesos containerizer filesystem, namespace, and isolator features, visit +<a href="http://mesos.apache.org/documentation/latest/mesos-containerizer/";>Mesos Containerizer</a> and +<a href="http://mesos.apache.org/documentation/latest/container-image/";>Mesos Container Images</a>.</p> + +<h2 id="docker-containerizer">Docker Containerizer</h2> + +<p>The Docker containerizer launches container images using the Docker engine. It may often provide +more advanced features than the native Mesos containerizer, but has to be installed separately to +Mesos on each agent host.</p> + +<p>Starting with the 0.17.0 release, <code>image</code> can be specified with a <code>{{docker.image[name][tag]}}</code> binder so that +the tag can be resolved to a concrete image digest. This ensures that the job always uses the same image +across restarts, even if the version identified by the tag has been updated, guaranteeing that only job +updates can mutate configuration.</p> + +<p>Example (available in the <a href="../../getting-started/vagrant/">Vagrant environment</a>):</p> +<pre class="highlight plaintext"><code>$ cat /vagrant/examples/jobs/hello_docker_engine.aurora +hello_loop = Process( + name = 'hello', + cmdline = """ + while true; do + echo hello world + sleep 10 + done + """) + +task = Task( + processes = [hello_loop], + resources = Resources(cpu=1, ram=1*MB, disk=8*MB) +) + +jobs = [ + Service( + cluster = 'devcluster', + environment = 'devel', + role = 'www-data', + name = 'hello_docker', + task = task, + container = Docker(image = 'python:2.7') + ), Service( + cluster = 'devcluster', + environment = 'devel', + role = 'www-data', + name = 'hello_docker_engine_binding', + task = task, + container = Docker(image = '{{docker.image[library/python][2.7]}}') + ) +] +</code></pre> + +<p>Note, this feature requires a v2 Docker registry. If using a private Docker registry its url +must be specified in the <code>clusters.json</code> configuration file under the key <code>docker_registry</code>. +If not specified <code>docker_registry</code> defaults to <code>https://registry-1.docker.io</code> (Docker Hub).</p> + +<p>Example: + # clusters.json + [{ + “name”: “devcluster”, + … + “docker_registry”: “<a href="https://registry.example.com";>https://registry.example.com</a>” + }]</p> + +<p>Details of how to use Docker via the Docker engine can be found in the +<a href="../../reference/configuration/#docker-object">Reference Documentation</a>. Please note that in order to +correctly execute processes inside a job, the Docker container must have Python 2.7 and potentitally +further Mesos dependencies installed. This limitation does not hold for Docker containers used via +the Mesos containerizer.</p> + +<p>For more information on launching Docker containers through the Docker containerizer, visit +<a href="http://mesos.apache.org/documentation/latest/docker-containerizer/";>Docker Containerizer</a></p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.18.1/features/cron-jobs/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.18.1/features/cron-jobs/index.html?rev=1813982&view=auto ============================================================================== --- aurora/site/publish/documentation/0.18.1/features/cron-jobs/index.html (added) +++ aurora/site/publish/documentation/0.18.1/features/cron-jobs/index.html Wed Nov 1 18:39:52 2017 @@ -0,0 +1,277 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/features/cron-jobs/'" + value="0.18.1"> + <option value="0.18.1" + selected="selected"> + 0.18.1 + (latest) + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="cron-jobs">Cron Jobs</h1> + +<p>Aurora supports execution of scheduled jobs on a Mesos cluster using cron-style syntax.</p> + +<ul> +<li><a href="#overview">Overview</a></li> +<li><a href="#collision-policies">Collision Policies</a></li> +<li><a href="#failure-recovery">Failure recovery</a></li> +<li><a href="#interacting-with-cron-jobs-via-the-aurora-cli">Interacting with cron jobs via the Aurora CLI</a> + +<ul> +<li><a href="#cron-schedule">cron schedule</a></li> +<li><a href="#cron-deschedule">cron deschedule</a></li> +<li><a href="#cron-start">cron start</a></li> +<li><a href="#job-killall-job-restart-job-kill">job killall, job restart, job kill</a></li> +</ul></li> +<li><a href="#technical-note-about-syntax">Technical Note About Syntax</a></li> +<li><a href="#caveats">Caveats</a> + +<ul> +<li><a href="#failovers">Failovers</a></li> +<li><a href="#collision-policy-is-best-effort">Collision policy is best-effort</a></li> +<li><a href="#timezone-configuration">Timezone Configuration</a></li> +</ul></li> +</ul> + +<h2 id="overview">Overview</h2> + +<p>A job is identified as a cron job by the presence of a +<code>cron_schedule</code> attribute containing a cron-style schedule in the +<a href="../../reference/configuration/#job-objects"><code>Job</code></a> object. Examples of cron schedules +include “every 5 minutes” (<code>*/5 * * * *</code>), “Fridays at 17:00” (<code>* 17 * * FRI</code>), and +“the 1st and 15th day of the month at 03:00” (<code>0 3 1,15 *</code>).</p> + +<p>Example (available in the <a href="../../getting-started/vagrant/">Vagrant environment</a>):</p> +<pre class="highlight plaintext"><code>$ cat /vagrant/examples/jobs/cron_hello_world.aurora +# A cron job that runs every 5 minutes. +jobs = [ + Job( + cluster = 'devcluster', + role = 'www-data', + environment = 'test', + name = 'cron_hello_world', + cron_schedule = '*/5 * * * *', + task = SimpleTask( + 'cron_hello_world', + 'echo "Hello world from cron, the time is now $(date --rfc-822)"'), + ), +] +</code></pre> + +<h2 id="collision-policies">Collision Policies</h2> + +<p>The <code>cron_collision_policy</code> field specifies the scheduler’s behavior when a new cron job is +triggered while an older run hasn’t finished. The scheduler has two policies available:</p> + +<ul> +<li><code>KILL_EXISTING</code>: The default policy - on a collision the old instances are killed and a instances with the current +configuration are started.</li> +<li><code>CANCEL_NEW</code>: On a collision the new run is cancelled.</li> +</ul> + +<p>Note that the use of <code>CANCEL_NEW</code> is likely a code smell - interrupted cron jobs should be able +to recover their progress on a subsequent invocation, otherwise they risk having their work queue +grow faster than they can process it.</p> + +<h2 id="failure-recovery">Failure recovery</h2> + +<p>Unlike with services, which aurora will always re-execute regardless of exit status, instances of +cron jobs retry according to the <code>max_task_failures</code> attribute of the +<a href="../../reference/configuration/#task-object">Task</a> object. To get “run-until-success” semantics, +set <code>max_task_failures</code> to <code>-1</code>.</p> + +<h2 id="interacting-with-cron-jobs-via-the-aurora-cli">Interacting with cron jobs via the Aurora CLI</h2> + +<p>Most interaction with cron jobs takes place using the <code>cron</code> subcommand. See <code>aurora cron -h</code> +for up-to-date usage instructions.</p> + +<h3 id="cron-schedule">cron schedule</h3> + +<p>Schedules a new cron job on the Aurora cluster for later runs or replaces the existing cron template +with a new one. Only future runs will be affected, any existing active tasks are left intact.</p> +<pre class="highlight plaintext"><code>$ aurora cron schedule devcluster/www-data/test/cron_hello_world /vagrant/examples/jobs/cron_hello_world.aurora +</code></pre> + +<h3 id="cron-deschedule">cron deschedule</h3> + +<p>Deschedules a cron job, preventing future runs but allowing current runs to complete.</p> +<pre class="highlight plaintext"><code>$ aurora cron deschedule devcluster/www-data/test/cron_hello_world +</code></pre> + +<h3 id="cron-start">cron start</h3> + +<p>Start a cron job immediately, outside of its normal cron schedule.</p> +<pre class="highlight plaintext"><code>$ aurora cron start devcluster/www-data/test/cron_hello_world +</code></pre> + +<h3 id="job-killall-job-restart-job-kill">job killall, job restart, job kill</h3> + +<p>Cron jobs create instances running on the cluster that you can interact with like normal Aurora +tasks with <code>job kill</code> and <code>job restart</code>.</p> + +<h2 id="technical-note-about-syntax">Technical Note About Syntax</h2> + +<p><code>cron_schedule</code> uses a restricted subset of BSD crontab syntax. While the +execution engine currently uses Quartz, the schedule parsing is custom, a subset of FreeBSD +<a href="http://www.freebsd.org/cgi/man.cgi?crontab(5)">crontab(5)</a> syntax. See +<a href="https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/cron/CrontabEntry.java#L106-L124";>the source</a> +for details.</p> + +<h2 id="caveats">Caveats</h2> + +<h3 id="failovers">Failovers</h3> + +<p>No failover recovery. Aurora does not record the latest minute it fired +triggers for across failovers. Therefore it’s possible to miss triggers +on failover. Note that this behavior may change in the future.</p> + +<p>It’s necessary to sync time between schedulers with something like <code>ntpd</code>. +Clock skew could cause double or missed triggers in the case of a failover.</p> + +<h3 id="collision-policy-is-best-effort">Collision policy is best-effort</h3> + +<p>Aurora aims to always have <em>at least one copy</em> of a given instance running at a time - it’s +an AP system, meaning it chooses Availability and Partition Tolerance at the expense of +Consistency.</p> + +<p>If your collision policy was <code>CANCEL_NEW</code> and a task has terminated but +Aurora has not noticed this Aurora will go ahead and create your new +task.</p> + +<p>If your collision policy was <code>KILL_EXISTING</code> and a task was marked <code>LOST</code> +but not yet GCed Aurora will go ahead and create your new task without +attempting to kill the old one (outside the GC interval).</p> + +<h3 id="timezone-configuration">Timezone Configuration</h3> + +<p>Cron timezone is configured indepdendently of JVM timezone with the <code>-cron_timezone</code> flag and +defaults to UTC.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html>
