Added: aurora/site/publish/documentation/0.19.1/development/client/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/development/client/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/development/client/index.html (added) +++ aurora/site/publish/documentation/0.19.1/development/client/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,296 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/client/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="developing-the-aurora-client">Developing the Aurora Client</h1> + +<p>The client is written in Python, and uses the +<a href="http://pantsbuild.github.io/python-readme.html";>Pants</a> build tool.</p> + +<h2 id="building-and-testing">Building and Testing</h2> + +<p>Building and testing the client code are both done using Pants. The relevant targets to know about +are:</p> + +<ul> +<li>Build a client executable: <code>./pants binary src/main/python/apache/aurora/client:aurora</code></li> +<li>Test client code: <code>./pants test src/test/python/apache/aurora/client/cli:cli</code></li> +</ul> + +<p>If you want to build a source distribution of the client, you need to run <code>./build-support/release/make-python-sdists</code>.</p> + +<h2 id="creating-custom-builds">Creating Custom Builds</h2> + +<p>There are situations where you may want to plug in custom logic to the Client that may not be +applicable to the open source codebase. Rather than create a whole CLI from scratch, you can +easily create your own custom, drop-in replacement aurora.pex using the pants build tool.</p> + +<p>First, create an AuroraCommandLine implementation as an entry-point for registering customizations:</p> +<pre class="highlight objective_c"><code><span style="background-color: #f8f8f8">from</span> <span style="background-color: #f8f8f8">apache</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">aurora</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">client</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">cli</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">client</span> <span style="background-color: #f8f8f8">import</span> <span style="background-color: #f8f8f8">AuroraCommandLine</span> + +<span style="background-color: #f8f8f8">class</span> <span style="background-color: #f8f8f8">CustomAuroraCommandLine</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">AuroraCommandLine</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> +<span style="color: #d14">"""Custom AuroraCommandLine for your needs"""</span> + +<span style="color: #000000;font-weight: bold">@property</span> +<span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">name</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #000000;font-weight: bold">return</span> <span style="color: #d14">"your-company-aurora"</span> + +<span style="color: #a61717;background-color: #e3d2d2">@classmethod</span> +<span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">get_description</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">cls</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #000000;font-weight: bold">return</span> <span style="color: #a61717;background-color: #e3d2d2">'</span><span style="background-color: #f8f8f8">Your</span> <span style="background-color: #f8f8f8">Company</span> <span style="background-color: #f8f8f8">internal</span> <span style="background-color: #f8f8f8">Aurora</span> <span style="background-color: #f8f8f8">client</span> <span style="background-color: #f8f8f8">command</span> <span style="background-color: #f8f8f8">line</span><span style="color: #a61717;background-color: #e3d2d2">'</span> + +<span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">__init__</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="background-color: #f8f8f8">super</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">CustomAuroraCommandLine</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">).</span><span style="background-color: #f8f8f8">__init__</span><span style="background-color: #f8f8f8">()</span> + <span style="color: #999999;font-weight: bold"># Add custom plugins.. +</span> <span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">register_plugin</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">YourCustomPlugin</span><span style="background-color: #f8f8f8">())</span> + +<span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">register_nouns</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="background-color: #f8f8f8">super</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">CustomAuroraCommandLine</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">).</span><span style="background-color: #f8f8f8">register_nouns</span><span style="background-color: #f8f8f8">()</span> + <span style="color: #999999;font-weight: bold"># You can even add new commands / sub-commands! +</span> <span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">register_noun</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">YourStartUpdateProxy</span><span style="background-color: #f8f8f8">())</span> + <span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">.</span><span style="background-color: #f8f8f8">register_noun</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">YourDeployWorkflowCommand</span><span style="background-color: #f8f8f8">())</span> +</code></pre> + +<p>Secondly, create a main entry point:</p> +<pre class="highlight plaintext"><code>def proxy_main(): + client = CustomAuroraCommandLine() + if len(sys.argv) == 1: + sys.argv.append("-h") + sys.exit(client.execute(sys.argv[1:])) +</code></pre> + +<p>Finally, you can wire everything up with a pants BUILD file in your project directory:</p> +<pre class="highlight plaintext"><code>python_binary( + name='aurora', + entry_point='your_company.aurora.client:proxy_main', + dependencies=[ + ':client_lib' + ] +) + +python_library( + name='client_lib', + sources = [ + 'client.py', + 'custom_plugin.py', + 'custom_command.py', + ], + dependencies = [ + # The Apache Aurora client + # Any other dependencies for your custom code + ], +) +</code></pre> + +<p>Using the same commands to build the client as above (but obviously pointing to this BUILD file +instead), you will have a drop-in replacement aurora.pex file with your customizations.</p> + +<h2 id="running-debugging">Running/Debugging</h2> + +<p>For manually testing client changes against a cluster, we use <a href="https://www.vagrantup.com/";>Vagrant</a>. +To start a virtual cluster, you need to install Vagrant, and then run <code>vagrant up</code> for the root of +the aurora workspace. This will create a vagrant host named “devcluster”, with a Mesos master, a set +of Mesos agents, and an Aurora scheduler.</p> + +<p>If you have a change you would like to test in your local cluster, you’ll rebuild the client:</p> +<pre class="highlight plaintext"><code>vagrant ssh -c 'aurorabuild client' +</code></pre> + +<p>Once this completes, the <code>aurora</code> command will reflect your changes.</p> + +<h2 id="running-debugging-in-pycharm">Running/Debugging in PyCharm</h2> + +<p>It’s possible to use PyCharm to run and debug both the client and client tests in an IDE. In order +to do this, first run:</p> +<pre class="highlight plaintext"><code>build-support/python/make-pycharm-virtualenv +</code></pre> + +<p>This script will configure a virtualenv with all of our Python requirements. Once the script +completes it will emit instructions for configuring PyCharm:</p> +<pre class="highlight plaintext"><code>Your PyCharm environment is now set up. You can open the project root +directory with PyCharm. + +Once the project is loaded: + - open project settings + - click 'Project Interpreter' + - click the cog in the upper-right corner + - click 'Add Local' + - select 'build-support/python/pycharm.venv/bin/python' + - click 'OK' +</code></pre> + +<h3 id="running-debugging-tests">Running/Debugging Tests</h3> + +<p>After following these instructions, you should now be able to run/debug tests directly from the IDE +by right-clicking on a test (or test class) and choosing to run or debug:</p> + +<p><a href="/documentation/latest/images/debug-client-test.png"><img alt="Debug Client Test" src="../../images/debug-client-test.png" /></a></p> + +<p>If you’ve set a breakpoint, you can see the run will now stop and let you debug:</p> + +<p><a href="/documentation/latest/images/debugging-client-test.png"><img alt="Debugging Client Test" src="../../images/debugging-client-test.png" /></a></p> + +<h3 id="running-debugging-the-client">Running/Debugging the Client</h3> + +<p>Actually running and debugging the client is unfortunately a bit more complex. You’ll need to create +a Run configuration:</p> + +<ul> +<li>Go to Run â Edit Configurations</li> +<li>Click the + icon to add a new configuration.</li> +<li>Choose python and name the configuration ‘client’.</li> +<li>Set the script path to <code>/your/path/to/aurora/src/main/python/apache/aurora/client/cli/client.py</code></li> +<li>Set the script parameters to the command you want to run (e.g. <code>job status <job key></code>)</li> +<li>Expand the Environment section and click the ellipsis to add a new environment variable</li> +<li>Click the + at the bottom to add a new variable named AURORA<em>CONFIG</em>ROOT whose value is the +path where the your cluster configuration can be found. For example, to talk to the scheduler +running in the vagrant image, it would be set to <code>/your/path/to/aurora/examples/vagrant</code> (this +is the directory where our example clusters.json is found).</li> +<li>You should now be able to run and debug this configuration!</li> +</ul> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html>
Added: aurora/site/publish/documentation/0.19.1/development/committers-guide/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/development/committers-guide/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/development/committers-guide/index.html (added) +++ aurora/site/publish/documentation/0.19.1/development/committers-guide/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,251 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/committers-guide/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="committer-39-s-guide">Committer’s Guide</h1> + +<p>Information for official Apache Aurora committers.</p> + +<h2 id="setting-up-your-email-account">Setting up your email account</h2> + +<p>Once your Apache ID has been set up you can configure your account and add ssh keys and setup an +email forwarding address at</p> +<pre class="highlight plaintext"><code>http://id.apache.org +</code></pre> + +<p>Additional instructions for setting up your new committer email can be found at</p> +<pre class="highlight plaintext"><code>http://www.apache.org/dev/user-email.html +</code></pre> + +<p>The recommended setup is to configure all services (mailing lists, JIRA, ReviewBoard) to send +emails to your @apache.org email address.</p> + +<h2 id="creating-a-gpg-key-for-releases">Creating a gpg key for releases</h2> + +<p>In order to create a release candidate you will need a gpg key published to an external key server +and that key will need to be added to our KEYS file as well.</p> + +<ol> +<li><p>Create a key:</p> +<pre class="highlight plaintext"><code> gpg --gen-key +</code></pre></li> +<li><p>Add your gpg key to the Apache Aurora KEYS file:</p> +<pre class="highlight plaintext"><code> git clone https://git-wip-us.apache.org/repos/asf/aurora.git + (gpg --list-sigs <KEY ID> && gpg --armor --export <KEY ID>) >> KEYS + git add KEYS && git commit -m "Adding gpg key for <APACHE ID>" + ./rbt post -o -g +</code></pre></li> +<li><p>Publish the key to an external key server:</p> +<pre class="highlight plaintext"><code> gpg --keyserver pgp.mit.edu --send-keys <KEY ID> +</code></pre></li> +<li><p>Update the changes to the KEYS file to the Apache Aurora svn dist locations listed below:</p> +<pre class="highlight plaintext"><code> https://dist.apache.org/repos/dist/dev/aurora/KEYS + https://dist.apache.org/repos/dist/release/aurora/KEYS +</code></pre></li> +<li><p>Add your key to git config for use with the release scripts:</p> +<pre class="highlight plaintext"><code> git config --global user.signingkey <KEY ID> +</code></pre></li> +</ol> + +<h2 id="creating-a-release">Creating a release</h2> + +<p>The following will guide you through the steps to create a release candidate, vote, and finally an +official Apache Aurora release. Before starting your gpg key should be in the KEYS file and you +must have access to commit to the dist.a.o repositories.</p> + +<ol> +<li>Ensure that all issues resolved for this release candidate are tagged with the correct Fix +Version in JIRA, the changelog script will use this to generate the CHANGELOG in step #2. +To assign the fix version:</li> +</ol> +<pre class="highlight plaintext"><code>* Look up the [previous release date](https://issues.apache.org/jira/browse/aurora/?selectedTab=com.atlassian.jira.jira-projects-plugin:versions-panel). +* Query all issues resolved after that release date: `project = AURORA AND status in (resolved, Closed) and fixVersion is empty and resolutiondate >= "YYYY/MM/DD"` +* In the upper right corner of the query result, select Tools > Bulk Edit. +* Select all issues > edit issue > set 'Change Fix Version/s' to the release version. +* Make sure to uncheck 'Send mail for this update' at the bottom. +</code></pre> + +<ol> +<li><p>Prepare RELEASE-NOTES.md for the release. This just boils down to removing the “(Not yet +released)” suffix from the impending release.</p></li> +<li><p>Create a release candidate. This will automatically update the CHANGELOG and commit it, create a +branch and update the current version within the trunk. To create a minor version update and publish +it run</p> +<pre class="highlight plaintext"><code> ./build-support/release/release-candidate -l m -p +</code></pre></li> +<li><p>Update, if necessary, the draft email created from the <code>release-candidate</code> script in step #2 and +send the [VOTE] email to the dev@ mailing list. You can verify the release signature and checksums +by running</p> +<pre class="highlight plaintext"><code> ./build-support/release/verify-release-candidate +</code></pre></li> +<li><p>Wait for the vote to complete. If the vote fails close the vote by replying to the initial [VOTE] +email sent in step #3 by editing the subject to [RESULT][VOTE] … and noting the failure reason +(example <a href="http://markmail.org/message/d4d6xtvj7vgwi76f";>here</a>). You’ll also need to manually revert +the commits generated by the release candidate script that incremented the snapshot version and +updated the changelog. Once that is done, now address any issues and go back to step #1 and run +again, this time you will use the -r flag to increment the release candidate version. This will +automatically clean up the release candidate rc0 branch and source distribution.</p> +<pre class="highlight plaintext"><code> ./build-support/release/release-candidate -l m -r 1 -p +</code></pre></li> +<li><p>Once the vote has successfully passed create the release</p></li> +</ol> + +<p><strong>IMPORTANT: make sure to use the correct release at this final step (e.g.: <code>-r 1</code> if rc1 candidate +has been voted for). Once the release tag is pushed it will be very hard to undo due to remote +git pre-receive hook explicitly forbidding release tag manipulations.</strong></p> +<pre class="highlight plaintext"><code> ./build-support/release/release +</code></pre> + +<ol> +<li><p>Update the draft email created fom the <code>release</code> script in step #5 to include the Apache ID’s for +all binding votes and send the [RESULT][VOTE] email to the dev@ mailing list.</p></li> +<li><p>Update the <a href="http://aurora.apache.org/";>Aurora Website</a> by following the +<a href="https://svn.apache.org/repos/asf/aurora/site/README.md";>instructions</a> on the ASF Aurora SVN repo. +Remember to add a blog post under source/blog and regenerate the site before committing.</p></li> +</ol> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.19.1/development/db-migration/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/development/db-migration/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/development/db-migration/index.html (added) +++ aurora/site/publish/documentation/0.19.1/development/db-migration/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,183 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/db-migration/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="db-migrations">DB Migrations</h1> + +<p>Changes to the DB schema should be made in the form of migrations. This ensures that all changes +are applied correctly after a DB dump from a previous version is restored.</p> + +<p>DB migrations are managed through a system built on top of +<a href="http://www.mybatis.org/migrations/";>MyBatis Migrations</a>. The migrations are run automatically when +a snapshot is restored, no manual interaction is required by cluster operators.</p> + +<h2 id="upgrades">Upgrades</h2> + +<p>When adding or altering tables or changing data, in addition to making to change in +<a href="../../src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql">schema.sql</a>, a new +migration class should be created under the org.apache.aurora.scheduler.storage.db.migration +package. The class should implement the <a href="https://github.com/mybatis/migrations/blob/master/src/main/java/org/apache/ibatis/migration/MigrationScript.java";>MigrationScript</a> +interface (see <a href="https://github.com/apache/aurora/blob/rel/0.19.1/src/test/java/org/apache/aurora/scheduler/storage/db/testmigration/V001_TestMigration.java";>V001_TestMigration</a> +as an example). The upgrade and downgrade scripts are defined in this class. When restoring a +snapshot the list of migrations on the classpath is compared to the list of applied changes in the +DB. Any changes that have not yet been applied are executed and their downgrade script is stored +alongside the changelog entry in the database to faciliate downgrades in the event of a rollback.</p> + +<h2 id="downgrades">Downgrades</h2> + +<p>If, while running migrations, a rollback is detected, i.e. a change exists in the DB changelog that +does not exist on the classpath, the downgrade script associated with each affected change is +applied.</p> + +<h2 id="baselines">Baselines</h2> + +<p>After enough time has passed (at least 1 official release), it should be safe to baseline migrations +if desired. This can be accomplished by ensuring the changes from migrations have been applied to +<a href="../../src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql">schema.sql</a> and then +removing the corresponding migration classes and adding a migration to remove the changelog entries.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.19.1/development/design-documents/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/development/design-documents/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/development/design-documents/index.html (added) +++ aurora/site/publish/documentation/0.19.1/development/design-documents/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,174 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/design-documents/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="design-documents">Design Documents</h1> + +<p>Since its inception as an Apache project, larger feature additions to the +Aurora code base are discussed in form of design documents. Design documents +are living documents until a consensus has been reached to implement a feature +in the proposed form.</p> + +<p>Current and past documents:</p> + +<ul> +<li><a href="../design/command-hooks/">Command Hooks for the Aurora Client</a></li> +<li><a href="https://docs.google.com/document/d/19gV8Po6DIHO14tOC7Qouk8RnboY8UCfRTninwn_5-7c/edit";>Dynamic Reservations</a></li> +<li><a href="https://docs.google.com/document/d/1J9SIswRMpVKQpnlvJAMAJtKfPP7ZARFknuyXl-2aZ-M/edit";>GPU Resources in Aurora</a></li> +<li><a href="https://docs.google.com/document/d/1KOO0LC046k75TqQqJ4c0FQcVGbxvrn71E10wAjMorVY/edit";>Health Checks for Updates</a></li> +<li><a href="https://docs.google.com/document/d/1Fc_YhhV7fc4D9Xv6gJzpfooxbK4YWZcvzw6Bd3qVTL8/edit";>JobUpdateDiff thrift API</a></li> +<li><a href="https://docs.google.com/document/d/11_lAsYIRlD5ETRzF2eSd3oa8LXAHYFD8rSetspYXaf4/edit";>REST API RFC</a></li> +<li><a href="https://docs.google.com/document/d/1r1WCHgmPJp5wbrqSZLsgtxPNj3sULfHrSFmxp2GyPTo/edit";>Revocable Mesos offers in Aurora</a></li> +<li><a href="https://docs.google.com/document/d/111T09NBF2zjjl7HE95xglsDpRdKoZqhCRM5hHmOfTLA/edit?usp=sharing";>Supporting the Mesos Universal Containerizer</a></li> +<li><a href="https://docs.google.com/document/d/1erszT-HsWf1zCIfhbqHlsotHxWUvDyI2xUwNQQQxLgs/edit?usp=sharing";>Tier Management In Apache Aurora</a></li> +<li><a href="https://docs.google.com/document/d/12hr6GnUZU3mc7xsWRzMi3nQILGB-3vyUxvbG-6YmvdE/edit";>Ubiquitous Jobs</a></li> +<li><a href="https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit";>Pluggable Scheduling</a></li> +</ul> + +<p>Design documents can be found in the Aurora issue tracker via the query <a href="https://issues.apache.org/jira/browse/AURORA-1528?jql=project%20%3D%20AURORA%20AND%20text%20~%20%22docs.google.com%22%20ORDER%20BY%20created";><code>project = AURORA AND text ~ "docs.google.com" ORDER BY created</code></a>.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.19.1/development/design/command-hooks/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/development/design/command-hooks/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/development/design/command-hooks/index.html (added) +++ aurora/site/publish/documentation/0.19.1/development/design/command-hooks/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,253 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/design/command-hooks/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="command-hooks-for-the-aurora-client">Command Hooks for the Aurora Client</h1> + +<h2 id="introduction-motivation">Introduction/Motivation</h2> + +<p>We’ve got hooks in the client that surround API calls. These are +pretty awkward, because they don’t correlate with user actions. For +example, suppose we wanted a policy that said users weren’t allowed to +kill all instances of a production job at once.</p> + +<p>Right now, all that we could hook would be the “killJob” api call. But +kill (at least in newer versions of the client) normally runs in +batches. If a user called killall, what we would see on the API level +is a series of “killJob” calls, each of which specified a batch of +instances. We woudn’t be able to distinguish between really killing +all instances of a job (which is forbidden under this policy), and +carefully killing in batches (which is permitted.) In each case, the +hook would just see a series of API calls, and couldn’t find out what +the actual command being executed was!</p> + +<p>For most policy enforcement, what we really want to be able to do is +look at and vet the commands that a user is performing, not the API +calls that the client uses to implement those commands.</p> + +<p>So I propose that we add a new kind of hooks, which surround noun/verb +commands. A hook will register itself to handle a collection of (noun, +verb) pairs. Whenever any of those noun/verb commands are invoked, the +hooks methods will be called around the execution of the verb. A +pre-hook will have the ability to reject a command, preventing the +verb from being executed.</p> + +<h2 id="registering-hooks">Registering Hooks</h2> + +<p>These hooks will be registered via configuration plugins. A configuration plugin +can register hooks using an API. Hooks registered this way are, effectively, +hardwired into the client executable.</p> + +<p>The order of execution of hooks is unspecified: they may be called in +any order. There is no way to guarantee that one hook will execute +before some other hook.</p> + +<h3 id="global-hooks">Global Hooks</h3> + +<p>Commands registered by the python call are called <em>global</em> hooks, +because they will run for all configurations, whether or not they +specify any hooks in the configuration file.</p> + +<p>In the implementation, hooks are registered in the module +<code>apache.aurora.client.cli.command_hooks</code>, using the class +<code>GlobalCommandHookRegistry</code>. A global hook can be registered by calling +<code>GlobalCommandHookRegistry.register_command_hook</code> in a configuration plugin.</p> + +<h3 id="the-api">The API</h3> +<pre class="highlight objective_c"><code><span style="background-color: #f8f8f8">class</span> <span style="background-color: #f8f8f8">CommandHook</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">object</span><span style="background-color: #f8f8f8">)</span> + <span style="color: #000000;font-weight: bold">@property</span> + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">name</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #d14">"""Returns a name for the hook."</span> + + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">get_nouns</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #d14">"""Return the nouns that have verbs that should invoke this hook."""</span> + + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">get_verbs</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">noun</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #d14">"""Return the verbs for a particular noun that should invoke his hook."""</span> + + <span style="color: #a61717;background-color: #e3d2d2">@abstractmethod</span> + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">pre_command</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">noun</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">verb</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">context</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">commandline</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #d14">"""Execute a hook before invoking a verb.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * noun: the noun being invoked.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * verb: the verb being invoked.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * context: the context object that will be used to invoke the verb.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> The options object will be initialized before calling the hook</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * commandline: the original argv collection used to invoke the client.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> Returns: True if the command should be allowed to proceed; False if the command</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> should be rejected.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> """</span> + + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">post_command</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">noun</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">verb</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">context</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">commandline</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">result</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #d14">"""Execute a hook after invoking a verb.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * noun: the noun being invoked.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * verb: the verb being invoked.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * context: the context object that will be used to invoke the verb.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> The options object will be initialized before calling the hook</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * commandline: the original argv collection used to invoke the client.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> * result: the result code returned by the verb.</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> Returns: nothing</span><span style="color: #a61717;background-color: #e3d2d2"> +</span><span style="color: #d14"> """</span> + +<span style="background-color: #f8f8f8">class</span> <span style="background-color: #f8f8f8">GlobalCommandHookRegistry</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">object</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="color: #a61717;background-color: #e3d2d2">@classmethod</span> + <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">register_command_hook</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">hook</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span> + <span style="background-color: #f8f8f8">pass</span> +</code></pre> + +<h3 id="skipping-hooks">Skipping Hooks</h3> + +<p>To skip a hook, a user uses a command-line option, <code>--skip-hooks</code>. The option can either +specify specific hooks to skip, or “all”:</p> + +<ul> +<li><code>aurora --skip-hooks=all job create east/bozo/devel/myjob</code> will create a job +without running any hooks.</li> +<li><code>aurora --skip-hooks=test,iq create east/bozo/devel/myjob</code> will create a job, +and will skip only the hooks named “test” and “iq”.</li> +</ul> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.19.1/development/scheduler/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/development/scheduler/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/development/scheduler/index.html (added) +++ aurora/site/publish/documentation/0.19.1/development/scheduler/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,258 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/scheduler/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="developing-the-aurora-scheduler">Developing the Aurora Scheduler</h1> + +<p>The Aurora scheduler is written in Java code and built with <a href="http://gradle.org";>Gradle</a>.</p> + +<h1 id="prerequisite">Prerequisite</h1> + +<p>When using Apache Aurora checked out from the source repository or the binary +distribution, the Gradle wrapper and JavaScript dependencies are provided. +However, you need to manually install them when using the source release +downloads:</p> + +<ol> +<li>Install Gradle following the instructions on the <a href="http://gradle.org";>Gradle web site</a></li> +<li><p>From the root directory of the Apache Aurora project generate the Gradle +wrapper by running:</p> + +<p>gradle wrapper</p></li> +</ol> + +<h1 id="getting-started">Getting Started</h1> + +<p>You will need Java 8 installed and on your <code>PATH</code> or unzipped somewhere with <code>JAVA_HOME</code> set. Then</p> +<pre class="highlight plaintext"><code>./gradlew tasks +</code></pre> + +<p>will bootstrap the build system and show available tasks. This can take a while the first time you +run it but subsequent runs will be much faster due to cached artifacts.</p> + +<h2 id="running-the-tests">Running the Tests</h2> + +<p>Aurora has a comprehensive unit test suite. To run the tests use</p> +<pre class="highlight plaintext"><code>./gradlew build +</code></pre> + +<p>Gradle will only re-run tests when dependencies of them have changed. To force a re-run of all +tests use</p> +<pre class="highlight plaintext"><code>./gradlew clean build +</code></pre> + +<h2 id="running-the-build-with-code-quality-checks">Running the build with code quality checks</h2> + +<p>To speed up development iteration, the plain gradle commands will not run static analysis tools. +However, you should run these before posting a review diff, and <strong>always</strong> run this before pushing a +commit to origin/master.</p> +<pre class="highlight plaintext"><code>./gradlew build -Pq +</code></pre> + +<h2 id="running-integration-tests">Running integration tests</h2> + +<p>To run the same tests that are run in the Apache Aurora continuous integration +environment:</p> +<pre class="highlight plaintext"><code>./build-support/jenkins/build.sh +</code></pre> + +<p>In addition, there is an end-to-end test that runs a suite of aurora commands +using a virtual cluster:</p> +<pre class="highlight plaintext"><code>./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh +</code></pre> + +<h2 id="creating-a-bundle-for-deployment">Creating a bundle for deployment</h2> + +<p>Gradle can create a zip file containing Aurora, all of its dependencies, and a launch script with</p> +<pre class="highlight plaintext"><code>./gradlew distZip +</code></pre> + +<p>or a tar file containing the same files with</p> +<pre class="highlight plaintext"><code>./gradlew distTar +</code></pre> + +<p>The output file will be written to <code>dist/distributions/aurora-scheduler.zip</code> or +<code>dist/distributions/aurora-scheduler.tar</code>.</p> + +<h1 id="developing-aurora-java-code">Developing Aurora Java code</h1> + +<h2 id="setting-up-an-ide">Setting up an IDE</h2> + +<p>Gradle can generate project files for your IDE. To generate an IntelliJ IDEA project run</p> +<pre class="highlight plaintext"><code>./gradlew idea +</code></pre> + +<p>and import the generated <code>aurora.ipr</code> file.</p> + +<h2 id="adding-or-upgrading-a-dependency">Adding or Upgrading a Dependency</h2> + +<p>New dependencies can be added from Maven central by adding a <code>compile</code> dependency to <code>build.gradle</code>. +For example, to add a dependency on <code>com.example</code>’s <code>example-lib</code> 1.0 add this block:</p> +<pre class="highlight plaintext"><code>compile 'com.example:example-lib:1.0' +</code></pre> + +<p>NOTE: Anyone thinking about adding a new dependency should first familiarize themselves with the +Apache Foundation’s third-party licensing +<a href="http://www.apache.org/legal/resolved.html#category-x";>policy</a>.</p> + +<h1 id="developing-the-aurora-build-system">Developing the Aurora Build System</h1> + +<h2 id="bootstrapping-gradle">Bootstrapping Gradle</h2> + +<p>The following files were autogenerated by <code>gradle wrapper</code> using gradle’s +<a href="http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.wrapper.Wrapper.html";>Wrapper</a> plugin and +should not be modified directly:</p> +<pre class="highlight plaintext"><code>./gradlew +./gradlew.bat +./gradle/wrapper/gradle-wrapper.jar +./gradle/wrapper/gradle-wrapper.properties +</code></pre> + +<p>To upgrade Gradle unpack the new version somewhere, run <code>/path/to/new/gradle wrapper</code> in the +repository root and commit the changed files.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.19.1/development/thermos/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/development/thermos/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/development/thermos/index.html (added) +++ aurora/site/publish/documentation/0.19.1/development/thermos/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,269 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/thermos/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<p>The Python components of Aurora are built using <a href="https://pantsbuild.github.io";>Pants</a>.</p> + +<h1 id="python-build-conventions">Python Build Conventions</h1> + +<p>The Python code is laid out according to the following conventions:</p> + +<ol> +<li><p>1 <code>BUILD</code> per 3rd level directory. For a list of current top-level packages run:</p> +<pre class="highlight plaintext"><code>% find src/main/python -maxdepth 3 -mindepth 3 -type d |\ +while read dname; do echo $dname |\ + sed 's@src/main/python/\(.*\)/\(.*\)/\(.*\).*@\1.\2.\3@'; done +</code></pre></li> +<li><p>Each <code>BUILD</code> file exports 1 +<a href="https://pantsbuild.github.io/build_dictionary.html#bdict_python_library";><code>python_library</code></a> +that provides a +<a href="https://pantsbuild.github.io/build_dictionary.html#setup_py";><code>setup_py</code></a> +containing each +<a href="https://pantsbuild.github.io/build_dictionary.html#python_binary";><code>python_binary</code></a> +in the <code>BUILD</code> file, named the same as the directory it’s in so that it can be referenced +without a ’:’ character. The <code>sources</code> field in the <code>python_library</code> will almost always be +<code>rglobs('*.py')</code>.</p></li> +<li><p>Other BUILD files may only depend on this single public <code>python_library</code> +target. Any other target is considered a private implementation detail and +should be prefixed with an <code>_</code>.</p></li> +<li><p><code>python_binary</code> targets are always named the same as the exported console script.</p></li> +<li><p><code>python_binary</code> targets must have identical <code>dependencies</code> to the <code>python_library</code> exported +by the package and must use <code>entry_point</code>.</p> + +<p>The means a PEX file generated by pants will contain exactly the same files that will be +available on the <code>PYTHONPATH</code> in the case of <code>pip install</code> of the corresponding library +target. This will help our migration off of Pants in the future.</p></li> +</ol> + +<h2 id="annotated-example-apache-thermos-runner">Annotated example - apache.thermos.runner</h2> +<pre class="highlight plaintext"><code>% find src/main/python/apache/thermos/runner +src/main/python/apache/thermos/runner +src/main/python/apache/thermos/runner/__init__.py +src/main/python/apache/thermos/runner/thermos_runner.py +src/main/python/apache/thermos/runner/BUILD +% cat src/main/python/apache/thermos/runner/BUILD +# License boilerplate omitted +import os + + +# Private target so that a setup_py can exist without a circular dependency. Only targets within +# this file should depend on this. +python_library( + name = '_runner', + # The target covers every python file under this directory and subdirectories. + sources = rglobs('*.py'), + dependencies = [ + '3rdparty/python:twitter.common.app', + '3rdparty/python:twitter.common.log', + # Source dependencies are always referenced without a ':'. + 'src/main/python/apache/thermos/common', + 'src/main/python/apache/thermos/config', + 'src/main/python/apache/thermos/core', + ], +) + +# Binary target for thermos_runner.pex. Nothing should depend on this - it's only used as an +# argument to ./pants binary. +python_binary( + name = 'thermos_runner', + # Use entry_point, not source so the files used here are the same ones tests see. + entry_point = 'apache.thermos.bin.thermos_runner', + dependencies = [ + # Notice that we depend only on the single private target from this BUILD file here. + ':_runner', + ], +) + +# The public library that everyone importing the runner symbols uses. +# The test targets and any other dependent source code should depend on this. +python_library( + name = 'runner', + dependencies = [ + # Again, notice that we depend only on the single private target from this BUILD file here. + ':_runner', + ], + # We always provide a setup_py. This will cause any dependee libraries to automatically + # reference this library in their requirements.txt rather than copy the source files into their + # sdist. + provides = setup_py( + # Conventionally named and versioned. + name = 'apache.thermos.runner', + version = open(os.path.join(get_buildroot(), '.auroraversion')).read().strip().upper(), + ).with_binaries({ + # Every binary in this file should also be repeated here. + # Always use the dict-form of .with_binaries so that commands with dashes in their names are + # supported. + # The console script name is always the same as the PEX with .pex stripped. + 'thermos_runner': ':thermos_runner', + }), +) +</code></pre> + +<h1 id="thermos-test-resources">Thermos Test resources</h1> + +<p>The Aurora source repository and distributions contain several +<a href="../../src/test/resources/org/apache/thermos/root/checkpoints">binary files</a> to +qualify the backwards-compatibility of thermos with checkpoint data. Since +thermos persists state to disk, to be read by the thermos observer), it is important that we have +tests that prevent regressions affecting the ability to parse previously-written data.</p> + +<p>The files included represent persisted checkpoints that exercise different +features of thermos. The existing files should not be modified unless +we are accepting backwards incompatibility, such as with a major release.</p> + +<p>It is not practical to write source code to generate these files on the fly, +as source would be vulnerable to drift (e.g. due to refactoring) in ways +that would undermine the goal of ensuring backwards compatibility.</p> + +<p>The most common reason to add a new checkpoint file would be to provide +coverage for new thermos features that alter the data format. This is +accomplished by writing and running a +<a href="../../reference/configuration/">job configuration</a> that exercises the feature, and +copying the checkpoint file from the sandbox directory, by default this is +<code>/var/run/thermos/checkpoints/<aurora task id></code>.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.19.1/development/thrift/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.19.1/development/thrift/index.html?rev=1828294&view=auto ============================================================================== --- aurora/site/publish/documentation/0.19.1/development/thrift/index.html (added) +++ aurora/site/publish/documentation/0.19.1/development/thrift/index.html Tue Apr 3 23:54:44 2018 @@ -0,0 +1,204 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/development/thrift/'" + value="0.19.1"> + <option value="0.20.0" + > + 0.20.0 + (latest) + </option> + <option value="0.19.1" + selected="selected"> + 0.19.1 + </option> + <option value="0.19.0" + > + 0.19.0 + </option> + <option value="0.18.1" + > + 0.18.1 + </option> + <option value="0.18.0" + > + 0.18.0 + </option> + <option value="0.17.0" + > + 0.17.0 + </option> + <option value="0.16.0" + > + 0.16.0 + </option> + <option value="0.15.0" + > + 0.15.0 + </option> + <option value="0.14.0" + > + 0.14.0 + </option> + <option value="0.13.0" + > + 0.13.0 + </option> + <option value="0.12.0" + > + 0.12.0 + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="thrift">Thrift</h1> + +<p>Aurora uses <a href="https://thrift.apache.org/";>Apache Thrift</a> for representing structured data in +client/server RPC protocol as well as for internal data storage. While Thrift is capable of +correctly handling additions and renames of the existing members, field removals must be done +carefully to ensure backwards compatibility and provide predictable deprecation cycle. This +document describes general guidelines for making Thrift schema changes to the existing fields in +<a href="https://github.com/apache/aurora/blob/rel/0.19.1/api/src/main/thrift/org/apache/aurora/gen/api.thrift";>api.thrift</a>.</p> + +<p>It is highly recommended to go through the +<a href="http://diwakergupta.github.io/thrift-missing-guide/";>Thrift: The Missing Guide</a> first to refresh on +basic Thrift schema concepts.</p> + +<h2 id="checklist">Checklist</h2> + +<p>Every existing Thrift schema modification is unique in its requirements and must be analyzed +carefully to identify its scope and expected consequences. The following checklist may help in that +analysis: +* Is this a new field/struct? If yes, go ahead +* Is this a pure field/struct rename without any type/structure change? If yes, go ahead and rename +* Anything else, read further to make sure your change is properly planned</p> + +<h2 id="deprecation-cycle">Deprecation cycle</h2> + +<p>Any time a breaking change (e.g.: field replacement or removal) is required, the following cycle +must be followed:</p> + +<h3 id="vcurrent">vCurrent</h3> + +<p>Change is applied in a way that does not break scheduler/client with this version to +communicate with scheduler/client from vCurrent-1. +* Do not remove or rename the old field +* Add a new field as an eventual replacement of the old one and implement a dual read/write +anywhere the old field is used. If a thrift struct is mapped in the DB store make sure both columns +are marked as <code>NOT NULL</code> +* Check <a href="https://github.com/apache/aurora/blob/rel/0.19.1/api/src/main/thrift/org/apache/aurora/gen/storage.thrift";>storage.thrift</a> to see if +the affected struct is stored in Aurora scheduler storage. If so, it’s almost certainly also +necessary to perform a <a href="../db-migration/">DB migration</a>. +* Add a deprecation jira ticket into the vCurrent+1 release candidate +* Add a TODO for the deprecated field mentioning the jira ticket</p> + +<h3 id="vcurrent-1">vCurrent+1</h3> + +<p>Finalize the change by removing the deprecated fields from the Thrift schema. +* Drop any dual read/write routines added in the previous version +* Remove thrift backfilling in scheduler +* Remove the deprecated Thrift field</p> + +<h2 id="testing">Testing</h2> + +<p>It’s always advisable to test your changes in the local vagrant environment to build more +confidence that you change is backwards compatible. It’s easy to simulate different +client/scheduler versions by playing with <code>aurorabuild</code> command. See <a href="../../getting-started/vagrant/">this document</a> +for more.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html>
