Added: aurora/site/publish/documentation/0.12.0/sla/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.12.0/sla/index.html?rev=1733548&view=auto ============================================================================== --- aurora/site/publish/documentation/0.12.0/sla/index.html (added) +++ aurora/site/publish/documentation/0.12.0/sla/index.html Fri Mar 4 02:43:01 2016 @@ -0,0 +1,306 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/sla/'" + value="0.12.0"> + <option value="0.12.0" + selected="selected"> + 0.12.0 + (latest) + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h2 id="aurora-sla-measurement">Aurora SLA Measurement</h2> + +<ul> +<li><a href="#overview">Overview</a></li> +<li><a href="#metric-details">Metric Details</a> + +<ul> +<li><a href="#platform-uptime">Platform Uptime</a></li> +<li><a href="#job-uptime">Job Uptime</a></li> +<li><a href="#median-time-to-assigned-(mtta)">Median Time To Assigned (MTTA)</a></li> +<li><a href="#median-time-to-running-(mttr)">Median Time To Running (MTTR)</a></li> +</ul></li> +<li><a href="#limitations">Limitations</a></li> +</ul> + +<h2 id="overview">Overview</h2> + +<p>The primary goal of the feature is collection and monitoring of Aurora job SLA (Service Level +Agreements) metrics that defining a contractual relationship between the Aurora/Mesos platform +and hosted services.</p> + +<p>The Aurora SLA feature is by default only enabled for service (non-cron) +production jobs (<code>"production = True"</code> in your <code>.aurora</code> config). It can be enabled for +non-production services via the scheduler command line flag <code>-sla_non_prod_metrics</code>.</p> + +<p>Counters that track SLA measurements are computed periodically within the scheduler. +The individual instance metrics are refreshed every minute (configurable via +<code>sla_stat_refresh_interval</code>). The instance counters are subsequently aggregated by +relevant grouping types before exporting to scheduler <code>/vars</code> endpoint (when using <code>vagrant</code> +that would be <code>http://192.168.33.7:8081/vars</code>)</p> + +<h2 id="metric-details">Metric Details</h2> + +<h3 id="platform-uptime">Platform Uptime</h3> + +<p><em>Aggregate amount of time a job spends in a non-runnable state due to platform unavailability +or scheduling delays. This metric tracks Aurora/Mesos uptime performance and reflects on any +system-caused downtime events (tasks LOST or DRAINED). Any user-initiated task kills/restarts +will not degrade this metric.</em></p> + +<p><strong>Collection scope:</strong></p> + +<ul> +<li>Per job - <code>sla_<job_key>_platform_uptime_percent</code></li> +<li>Per cluster - <code>sla_cluster_platform_uptime_percent</code></li> +</ul> + +<p><strong>Units:</strong> percent</p> + +<p>A fault in the task environment may cause the Aurora/Mesos to have different views on the task state +or lose track of the task existence. In such cases, the service task is marked as LOST and +rescheduled by Aurora. For example, this may happen when the task stays in ASSIGNED or STARTING +for too long or the Mesos slave becomes unhealthy (or disappears completely). The time between +task entering LOST and its replacement reaching RUNNING state is counted towards platform downtime.</p> + +<p>Another example of a platform downtime event is the administrator-requested task rescheduling. This +happens during planned Mesos slave maintenance when all slave tasks are marked as DRAINED and +rescheduled elsewhere.</p> + +<p>To accurately calculate Platform Uptime, we must separate platform incurred downtime from user +actions that put a service instance in a non-operational state. It is simpler to isolate +user-incurred downtime and treat all other downtime as platform incurred.</p> + +<p>Currently, a user can cause a healthy service (task) downtime in only two ways: via <code>killTasks</code> +or <code>restartShards</code> RPCs. For both, their affected tasks leave an audit state transition trail +relevant to uptime calculations. By applying a special “SLA meaning” to exposed task state +transition records, we can build a deterministic downtime trace for every given service instance.</p> + +<p>A task going through a state transition carries one of three possible SLA meanings +(see <a href="https://github.com/apache/aurora/blob/#{git_tag}/src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java";>SlaAlgorithm.java</a>) for +sla-to-task-state mapping):</p> + +<ul> +<li><p>Task is UP: starts a period where the task is considered to be up and running from the Aurora +platform standpoint.</p></li> +<li><p>Task is DOWN: starts a period where the task cannot reach the UP state for some +non-user-related reason. Counts towards instance downtime.</p></li> +<li><p>Task is REMOVED from SLA: starts a period where the task is not expected to be UP due to +user initiated action or failure. We ignore this period for the uptime calculation purposes.</p></li> +</ul> + +<p>This metric is recalculated over the last sampling period (last minute) to account for +any UP/DOWN/REMOVED events. It ignores any UP/DOWN events not immediately adjacent to the +sampling interval as well as adjacent REMOVED events.</p> + +<h3 id="job-uptime">Job Uptime</h3> + +<p><em>Percentage of the job instances considered to be in RUNNING state for the specified duration +relative to request time. This is a purely application side metric that is considering aggregate +uptime of all RUNNING instances. Any user- or platform initiated restarts directly affect +this metric.</em></p> + +<p><strong>Collection scope:</strong> We currently expose job uptime values at 5 pre-defined +percentiles (50th,75th,90th,95th and 99th):</p> + +<ul> +<li><code>sla_<job_key>_job_uptime_50_00_sec</code></li> +<li><code>sla_<job_key>_job_uptime_75_00_sec</code></li> +<li><code>sla_<job_key>_job_uptime_90_00_sec</code></li> +<li><code>sla_<job_key>_job_uptime_95_00_sec</code></li> +<li><code>sla_<job_key>_job_uptime_99_00_sec</code></li> +</ul> + +<p><strong>Units:</strong> seconds +You can also get customized real-time stats from aurora client. See <code>aurora sla -h</code> for +more details.</p> + +<h3 id="median-time-to-assigned-mtta">Median Time To Assigned (MTTA)</h3> + +<p><em>Median time a job spends waiting for its tasks to be assigned to a host. This is a combined +metric that helps track the dependency of scheduling performance on the requested resources +(user scope) as well as the internal scheduler bin-packing algorithm efficiency (platform scope).</em></p> + +<p><strong>Collection scope:</strong></p> + +<ul> +<li>Per job - <code>sla_<job_key>_mtta_ms</code></li> +<li>Per cluster - <code>sla_cluster_mtta_ms</code></li> +<li>Per instance size (small, medium, large, x-large, xx-large). Size are defined in: +<a href="https://github.com/apache/aurora/blob/#{git_tag}/src/main/java/org/apache/aurora/scheduler/base/ResourceAggregates.java";>ResourceAggregates.java</a>) + +<ul> +<li>By CPU:</li> +<li><code>sla_cpu_small_mtta_ms</code></li> +<li><code>sla_cpu_medium_mtta_ms</code></li> +<li><code>sla_cpu_large_mtta_ms</code></li> +<li><code>sla_cpu_xlarge_mtta_ms</code></li> +<li><code>sla_cpu_xxlarge_mtta_ms</code></li> +<li>By RAM:</li> +<li><code>sla_ram_small_mtta_ms</code></li> +<li><code>sla_ram_medium_mtta_ms</code></li> +<li><code>sla_ram_large_mtta_ms</code></li> +<li><code>sla_ram_xlarge_mtta_ms</code></li> +<li><code>sla_ram_xxlarge_mtta_ms</code></li> +<li>By DISK:</li> +<li><code>sla_disk_small_mtta_ms</code></li> +<li><code>sla_disk_medium_mtta_ms</code></li> +<li><code>sla_disk_large_mtta_ms</code></li> +<li><code>sla_disk_xlarge_mtta_ms</code></li> +<li><code>sla_disk_xxlarge_mtta_ms</code></li> +</ul></li> +</ul> + +<p><strong>Units:</strong> milliseconds</p> + +<p>MTTA only considers instances that have already reached ASSIGNED state and ignores those +that are still PENDING. This ensures straggler instances (e.g. with unreasonable resource +constraints) do not affect metric curves.</p> + +<h3 id="median-time-to-running-mttr">Median Time To Running (MTTR)</h3> + +<p><em>Median time a job waits for its tasks to reach RUNNING state. This is a comprehensive metric +reflecting on the overall time it takes for the Aurora/Mesos to start executing user content.</em></p> + +<p><strong>Collection scope:</strong></p> + +<ul> +<li>Per job - <code>sla_<job_key>_mttr_ms</code></li> +<li>Per cluster - <code>sla_cluster_mttr_ms</code></li> +<li>Per instance size (small, medium, large, x-large, xx-large). Size are defined in: +<a href="https://github.com/apache/aurora/blob/#{git_tag}/src/main/java/org/apache/aurora/scheduler/base/ResourceAggregates.java";>ResourceAggregates.java</a>) + +<ul> +<li>By CPU:</li> +<li><code>sla_cpu_small_mttr_ms</code></li> +<li><code>sla_cpu_medium_mttr_ms</code></li> +<li><code>sla_cpu_large_mttr_ms</code></li> +<li><code>sla_cpu_xlarge_mttr_ms</code></li> +<li><code>sla_cpu_xxlarge_mttr_ms</code></li> +<li>By RAM:</li> +<li><code>sla_ram_small_mttr_ms</code></li> +<li><code>sla_ram_medium_mttr_ms</code></li> +<li><code>sla_ram_large_mttr_ms</code></li> +<li><code>sla_ram_xlarge_mttr_ms</code></li> +<li><code>sla_ram_xxlarge_mttr_ms</code></li> +<li>By DISK:</li> +<li><code>sla_disk_small_mttr_ms</code></li> +<li><code>sla_disk_medium_mttr_ms</code></li> +<li><code>sla_disk_large_mttr_ms</code></li> +<li><code>sla_disk_xlarge_mttr_ms</code></li> +<li><code>sla_disk_xxlarge_mttr_ms</code></li> +</ul></li> +</ul> + +<p><strong>Units:</strong> milliseconds</p> + +<p>MTTR only considers instances in RUNNING state. This ensures straggler instances (e.g. with +unreasonable resource constraints) do not affect metric curves.</p> + +<h2 id="limitations">Limitations</h2> + +<ul> +<li><p>The availability of Aurora SLA metrics is bound by the scheduler availability.</p></li> +<li><p>All metrics are calculated at a pre-defined interval (currently set at 1 minute). +Scheduler restarts may result in missed collections.</p></li> +</ul> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html>
Added: aurora/site/publish/documentation/0.12.0/storage-config/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.12.0/storage-config/index.html?rev=1733548&view=auto ============================================================================== --- aurora/site/publish/documentation/0.12.0/storage-config/index.html (added) +++ aurora/site/publish/documentation/0.12.0/storage-config/index.html Fri Mar 4 02:43:01 2016 @@ -0,0 +1,274 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/storage-config/'" + value="0.12.0"> + <option value="0.12.0" + selected="selected"> + 0.12.0 + (latest) + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="storage-configuration-and-maintenance">Storage Configuration And Maintenance</h1> + +<ul> +<li><a href="#overview">Overview</a></li> +<li><a href="#scheduler-storage-configuration-flags">Scheduler storage configuration flags</a> + +<ul> +<li><a href="#mesos-replicated-log-configuration-flags">Mesos replicated log configuration flags</a></li> +<li><a href="#-native_log_quorum_size">-native<em>log</em>quorum_size</a></li> +<li><a href="#-native_log_file_path">-native<em>log</em>file_path</a></li> +<li><a href="#-native_log_zk_group_path">-native<em>log</em>zk<em>group</em>path</a></li> +<li><a href="#backup-configuration-flags">Backup configuration flags</a></li> +<li><a href="#-backup_interval">-backup_interval</a></li> +<li><a href="#-backup_dir">-backup_dir</a></li> +<li><a href="#-max_saved_backups">-max<em>saved</em>backups</a></li> +</ul></li> +<li><a href="#recovering-from-a-scheduler-backup">Recovering from a scheduler backup</a> + +<ul> +<li><a href="#summary">Summary</a></li> +<li><a href="#preparation">Preparation</a></li> +<li><a href="#cleanup-and-re-initialize-mesos-replicated-log">Cleanup and re-initialize Mesos replicated log</a></li> +<li><a href="#restore-from-backup">Restore from backup</a></li> +<li><a href="#cleanup">Cleanup</a></li> +</ul></li> +</ul> + +<h2 id="overview">Overview</h2> + +<p>This document summarizes Aurora storage configuration and maintenance details and is +intended for use by anyone deploying and/or maintaining Aurora.</p> + +<p>For a high level overview of the Aurora storage architecture refer to <a href="/documentation/0.12.0/storage/">this document</a>.</p> + +<h2 id="scheduler-storage-configuration-flags">Scheduler storage configuration flags</h2> + +<p>Below is a summary of scheduler storage configuration flags that either don’t have default values +or require attention before deploying in a production environment.</p> + +<h3 id="mesos-replicated-log-configuration-flags">Mesos replicated log configuration flags</h3> + +<h4 id="nativelogquorum_size">-native<em>log</em>quorum_size</h4> + +<p>Defines the Mesos replicated log quorum size. See +<a href="/documentation/0.12.0/deploying-aurora-scheduler/#replicated-log-configuration">the replicated log configuration document</a> +on how to choose the right value.</p> + +<h4 id="nativelogfile_path">-native<em>log</em>file_path</h4> + +<p>Location of the Mesos replicated log files. Consider allocating a dedicated disk (preferably SSD) +for Mesos replicated log files to ensure optimal storage performance.</p> + +<h4 id="nativelogzkgrouppath">-native<em>log</em>zk<em>group</em>path</h4> + +<p>ZooKeeper path used for Mesos replicated log quorum discovery.</p> + +<p>See <a href="https://github.com/apache/aurora/blob/#{git_tag}/src/main/java/org/apache/aurora/scheduler/log/mesos/MesosLogStreamModule.java";>code</a>) for +other available Mesos replicated log configuration options and default values.</p> + +<h3 id="backup-configuration-flags">Backup configuration flags</h3> + +<p>Configuration options for the Aurora scheduler backup manager.</p> + +<h4 id="backup_interval">-backup_interval</h4> + +<p>The interval on which the scheduler writes local storage backups. The default is every hour.</p> + +<h4 id="backup_dir">-backup_dir</h4> + +<p>Directory to write backups to.</p> + +<h4 id="maxsavedbackups">-max<em>saved</em>backups</h4> + +<p>Maximum number of backups to retain before deleting the oldest backup(s).</p> + +<h2 id="recovering-from-a-scheduler-backup">Recovering from a scheduler backup</h2> + +<ul> +<li><a href="#overview">Overview</a></li> +<li><a href="#preparation">Preparation</a></li> +<li><a href="#assess-mesos-replicated-log-damage">Assess Mesos replicated log damage</a></li> +<li><a href="#restore-from-backup">Restore from backup</a></li> +<li><a href="#cleanup">Cleanup</a></li> +</ul> + +<p><strong>Be sure to read the entire page before attempting to restore from a backup, as it may have +unintended consequences.</strong></p> + +<h3 id="summary">Summary</h3> + +<p>The restoration procedure replaces the existing (possibly corrupted) Mesos replicated log with an +earlier, backed up, version and requires all schedulers to be taken down temporarily while +restoring. Once completed, the scheduler state resets to what it was when the backup was created. +This means any jobs/tasks created or updated after the backup are unknown to the scheduler and will +be killed shortly after the cluster restarts. All other tasks continue operating as normal.</p> + +<p>Usually, it is a bad idea to restore a backup that is not extremely recent (i.e. older than a few +hours). This is because the scheduler will expect the cluster to look exactly as the backup does, +so any tasks that have been rescheduled since the backup was taken will be killed.</p> + +<h3 id="preparation">Preparation</h3> + +<p>Follow these steps to prepare the cluster for restoring from a backup:</p> + +<ul> +<li><p>Stop all scheduler instances</p></li> +<li><p>Consider blocking external traffic on a port defined in <code>-http_port</code> for all schedulers to +prevent users from interacting with the scheduler during the restoration process. This will help +troubleshooting by reducing the scheduler log noise and prevent users from making changes that will +be erased after the backup snapshot is restored</p></li> +<li><p>Next steps are required to put scheduler into a partially disabled state where it would still be +able to accept storage recovery requests but unable to schedule or change task states. This may be +accomplished by updating the following scheduler configuration options:</p> + +<ul> +<li>Set <code>-mesos_master_address</code> to a non-existent zk address. This will prevent scheduler from +registering with Mesos. E.g.: <code>-mesos_master_address=zk://localhost:2181</code></li> +<li><code>-max_registration_delay</code> - set to sufficiently long interval to prevent registration timeout +and as a result scheduler suicide. E.g: <code>-max_registration_delay=360mins</code></li> +<li>Make sure <code>-reconciliation_initial_delay</code> option is set high enough (e.g.: <code>365days</code>) to +prevent accidental task GC. This is important as scheduler will attempt to reconcile the cluster +state and will kill all tasks when restarted with an empty Mesos replicated log.</li> +</ul></li> +<li><p>Restart all schedulers</p></li> +</ul> + +<h3 id="cleanup-and-re-initialize-mesos-replicated-log">Cleanup and re-initialize Mesos replicated log</h3> + +<p>Get rid of the corrupted files and re-initialize Mesos replicate log:</p> + +<ul> +<li>Stop schedulers</li> +<li>Delete all files under <code>-native_log_file_path</code> on all schedulers</li> +<li>Initialize Mesos replica’s log file: <code>mesos-log initialize --path=<-native_log_file_path></code></li> +<li>Restart schedulers</li> +</ul> + +<h3 id="restore-from-backup">Restore from backup</h3> + +<p>At this point the scheduler is ready to rehydrate from the backup:</p> + +<ul> +<li><p>Identify the leading scheduler by:</p> + +<ul> +<li>running <code>aurora_admin get_scheduler <cluster></code> - if scheduler is responsive</li> +<li>examining scheduler logs</li> +<li>or examining Zookeeper registration under the path defined by <code>-zk_endpoints</code> +and <code>-serverset_path</code></li> +</ul></li> +<li><p>Locate the desired backup file, copy it to the leading scheduler and stage recovery by running +the following command on a leader +<code>aurora_admin scheduler_stage_recovery <cluster> scheduler-backup-<yyyy-MM-dd-HH-mm></code></p></li> +<li><p>At this point, the recovery snapshot is staged and available for manual verification/modification +via <code>aurora_admin scheduler_print_recovery_tasks</code> and <code>scheduler_delete_recovery_tasks</code> commands. +See <code>aurora_admin help <command></code> for usage details.</p></li> +<li><p>Commit recovery. This instructs the scheduler to overwrite the existing Mesosreplicated log with +the provided backup snapshot and initiate a mandatory failover +<code>aurora_admin scheduler_commit_recovery <cluster></code></p></li> +</ul> + +<h3 id="cleanup">Cleanup</h3> + +<p>Undo any modification done during <a href="#preparation">Preparation</a> sequence.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.12.0/storage/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.12.0/storage/index.html?rev=1733548&view=auto ============================================================================== --- aurora/site/publish/documentation/0.12.0/storage/index.html (added) +++ aurora/site/publish/documentation/0.12.0/storage/index.html Fri Mar 4 02:43:01 2016 @@ -0,0 +1,206 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/storage/'" + value="0.12.0"> + <option value="0.12.0" + selected="selected"> + 0.12.0 + (latest) + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="aurora-scheduler-storage">Aurora Scheduler Storage</h1> + +<ul> +<li><a href="#overview">Overview</a></li> +<li><a href="#reads-writes-modifications">Reads, writes, modifications</a> + +<ul> +<li><a href="#read-lifecycle">Read lifecycle</a></li> +<li><a href="#write-lifecycle">Write lifecycle</a></li> +</ul></li> +<li><a href="#atomicity-consistency-and-isolation">Atomicity, consistency and isolation</a></li> +<li><a href="#population-on-restart">Population on restart</a></li> +</ul> + +<h2 id="overview">Overview</h2> + +<p>Aurora scheduler maintains data that need to be persisted to survive failovers and restarts. +For example:</p> + +<ul> +<li>Task configurations and scheduled task instances</li> +<li>Job update configurations and update progress</li> +<li>Production resource quotas</li> +<li>Mesos resource offer host attributes</li> +</ul> + +<p>Aurora solves its persistence needs by leveraging the Mesos implementation of a Paxos replicated +log <a href="https://ramcloud.stanford.edu/~ongaro/userstudy/paxos.pdf";>[1]</a> +<a href="http://en.wikipedia.org/wiki/State_machine_replication";>[2]</a> with a key-value +<a href="https://github.com/google/leveldb";>LevelDB</a> storage as persistence media.</p> + +<p>Conceptually, it can be represented by the following major components:</p> + +<ul> +<li>Volatile storage: in-memory cache of all available data. Implemented via in-memory +<a href="http://www.h2database.com/html/main.html";>H2 Database</a> and accessed via +<a href="http://mybatis.github.io/mybatis-3/";>MyBatis</a>.</li> +<li>Log manager: interface between Aurora storage and Mesos replicated log. The default schema format +is <a href="https://github.com/apache/thrift";>thrift</a>. Data is stored in serialized binary form.</li> +<li>Snapshot manager: all data is periodically persisted in Mesos replicated log in a single snapshot. +This helps establishing periodic recovery checkpoints and speeds up volatile storage recovery on +restart.</li> +<li>Backup manager: as a precaution, snapshots are periodically written out into backup files. +This solves a <a href="/documentation/0.12.0/storage-config/#recovering-from-a-scheduler-backup">disaster recovery problem</a> +in case of a complete loss or corruption of Mesos log files.</li> +</ul> + +<p><img alt="Storage hierarchy" src="../images/storage_hierarchy.png" /></p> + +<h2 id="reads-writes-modifications">Reads, writes, modifications</h2> + +<p>All services in Aurora access data via a set of predefined store interfaces (aka stores) logically +grouped by the type of data they serve. Every interface defines a specific set of operations allowed +on the data thus abstracting out the storage access and the actual persistence implementation. The +latter is especially important in view of a general immutability of persisted data. With the Mesos +replicated log as the underlying persistence solution, data can be read and written easily but not +modified. All modifications are simulated by saving new versions of modified objects. This feature +and general performance considerations justify the existence of the volatile in-memory store.</p> + +<h3 id="read-lifecycle">Read lifecycle</h3> + +<p>There are two types of reads available in Aurora: consistent and weakly-consistent. The difference +is explained <a href="#atomicity-and-isolation">below</a>.</p> + +<p>All reads are served from the volatile storage making reads generally cheap storage operations +from the performance standpoint. The majority of the volatile stores are represented by the +in-memory H2 database. This allows for rich schema definitions, queries and relationships that +key-value storage is unable to match.</p> + +<h3 id="write-lifecycle">Write lifecycle</h3> + +<p>Writes are more involved operations since in addition to updating the volatile store data has to be +appended to the replicated log. Data is not available for reads until fully ack-ed by both +replicated log and volatile storage.</p> + +<h2 id="atomicity-consistency-and-isolation">Atomicity, consistency and isolation</h2> + +<p>Aurora uses <a href="http://en.wikipedia.org/wiki/Write-ahead_logging";>write-ahead logging</a> to ensure +consistency between replicated and volatile storage. In Aurora, data is first written into the +replicated log and only then updated in the volatile store.</p> + +<p>Aurora storage uses read-write locks to serialize data mutations and provide consistent view of the +available data. The available <code>Storage</code> interface exposes 3 major types of operations: +* <code>consistentRead</code> - access is locked using reader’s lock and provides consistent view on read +* <code>weaklyConsistentRead</code> - access is lock-less. Delivers best contention performance but may result +in stale reads +* <code>write</code> - access is fully serialized by using writer’s lock. Operation success requires both +volatile and replicated writes to succeed.</p> + +<p>The consistency of the volatile store is enforced via H2 transactional isolation.</p> + +<h2 id="population-on-restart">Population on restart</h2> + +<p>Any time a scheduler restarts, it restores its volatile state from the most recent position recorded +in the replicated log by restoring the snapshot and replaying individual log entries on top to fully +recover the state up to the last write.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.12.0/test-resource-generation/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.12.0/test-resource-generation/index.html?rev=1733548&view=auto ============================================================================== --- aurora/site/publish/documentation/0.12.0/test-resource-generation/index.html (added) +++ aurora/site/publish/documentation/0.12.0/test-resource-generation/index.html Fri Mar 4 02:43:01 2016 @@ -0,0 +1,136 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/test-resource-generation/'" + value="0.12.0"> + <option value="0.12.0" + selected="selected"> + 0.12.0 + (latest) + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="generating-test-resources">Generating test resources</h1> + +<h2 id="background">Background</h2> + +<p>The Aurora source repository and distributions contain several +<a href="https://github.com/apache/aurora/blob/#{git_tag}/src/test/resources/org/apache/thermos/root/checkpoints";>binary files</a>) to +qualify the backwards-compatibility of thermos with checkpoint data. Since +thermos persists state to disk, to be read by the thermos observer), it is important that we have +tests that prevent regressions affecting the ability to parse previously-written data.</p> + +<h2 id="generating-test-files">Generating test files</h2> + +<p>The files included represent persisted checkpoints that exercise different +features of thermos. The existing files should not be modified unless +we are accepting backwards incompatibility, such as with a major release.</p> + +<p>It is not practical to write source code to generate these files on the fly, +as source would be vulnerable to drift (e.g. due to refactoring) in ways +that would undermine the goal of ensuring backwards compatibility.</p> + +<p>The most common reason to add a new checkpoint file would be to provide +coverage for new thermos features that alter the data format. This is +accomplished by writing and running a +<a href="/documentation/0.12.0/configuration-reference/">job configuration</a> that exercises the feature, and +copying the checkpoint file from the sandbox directory, by default this is +<code>/var/run/thermos/checkpoints/<aurora task id></code>.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.12.0/thrift-deprecation/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.12.0/thrift-deprecation/index.html?rev=1733548&view=auto ============================================================================== --- aurora/site/publish/documentation/0.12.0/thrift-deprecation/index.html (added) +++ aurora/site/publish/documentation/0.12.0/thrift-deprecation/index.html Fri Mar 4 02:43:01 2016 @@ -0,0 +1,165 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/thrift-deprecation/'" + value="0.12.0"> + <option value="0.12.0" + selected="selected"> + 0.12.0 + (latest) + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="thrift-api-changes">Thrift API Changes</h1> + +<h2 id="overview">Overview</h2> + +<p>Aurora uses <a href="https://thrift.apache.org/";>Apache Thrift</a> for representing structured data in +client/server RPC protocol as well as for internal data storage. While Thrift is capable of +correctly handling additions and renames of the existing members, field removals must be done +carefully to ensure backwards compatibility and provide predictable deprecation cycle. This +document describes general guidelines for making Thrift schema changes to the existing fields in +<a href="https://github.com/apache/aurora/blob/#{git_tag}/api/src/main/thrift/org/apache/aurora/gen/api.thrift";>api.thrift</a>).</p> + +<p>It is highly recommended to go through the +<a href="http://diwakergupta.github.io/thrift-missing-guide/";>Thrift: The Missing Guide</a> first to refresh on +basic Thrift schema concepts.</p> + +<h2 id="checklist">Checklist</h2> + +<p>Every existing Thrift schema modification is unique in its requirements and must be analyzed +carefully to identify its scope and expected consequences. The following checklist may help in that +analysis: +* Is this a new field/struct? If yes, go ahead +* Is this a pure field/struct rename without any type/structure change? If yes, go ahead and rename +* Anything else, read further to make sure your change is properly planned</p> + +<h2 id="deprecation-cycle">Deprecation cycle</h2> + +<p>Any time a breaking change (e.g.: field replacement or removal) is required, the following cycle +must be followed:</p> + +<h3 id="vcurrent">vCurrent</h3> + +<p>Change is applied in a way that does not break scheduler/client with this version to +communicate with scheduler/client from vCurrent-1. +* Do not remove or rename the old field +* Add a new field as an eventual replacement of the old one and implement a dual read/write +anywhere the old field is used +* Check <a href="https://github.com/apache/aurora/blob/#{git_tag}/api/src/main/thrift/org/apache/aurora/gen/storage.thrift";>storage.thrift</a>) to see if the +affected struct is stored in Aurora scheduler storage. If so, you most likely need to backfill +existing data to ensure both fields are populated eagerly on startup +See <a href="https://github.com/apache/aurora/blob/#{git_tag}/src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java";>StorageBackfill.java</a>) +* Add a deprecation jira ticket into the vCurrent+1 release candidate +* Add a TODO for the deprecated field mentioning the jira ticket</p> + +<h3 id="vcurrent-1">vCurrent+1</h3> + +<p>Finalize the change by removing the deprecated fields from the Thrift schema. +* Drop any dual read/write routines added in the previous version +* Remove the deprecated Thrift field</p> + +<h2 id="testing">Testing</h2> + +<p>It’s always advisable to test your changes in the local vagrant environment to build more +confidence that you change is backwards compatible. It’s easy to simulate different +client/scheduler versions by playing with <code>aurorabuild</code> command. See <a href="/documentation/0.12.0/vagrant/">this document</a> +for more.</p> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.12.0/tools/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.12.0/tools/index.html?rev=1733548&view=auto ============================================================================== --- aurora/site/publish/documentation/0.12.0/tools/index.html (added) +++ aurora/site/publish/documentation/0.12.0/tools/index.html Fri Mar 4 02:43:01 2016 @@ -0,0 +1,135 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/tools/'" + value="0.12.0"> + <option value="0.12.0" + selected="selected"> + 0.12.0 + (latest) + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="tools">Tools</h1> + +<p>Various tools integrate with Aurora. Is there a tool missing? Let us know, or submit a patch to add it!</p> + +<ul> +<li><p>Load-balacing technology used to direct traffic to services running on Aurora</p> + +<ul> +<li><a href="https://github.com/airbnb/synapse";>synapse</a> based on HAProxy</li> +<li><a href="https://github.com/tellapart/aurproxy";>aurproxy</a> based on nginx</li> +<li><a href="https://github.com/benley/aurora-jobhopper";>jobhopper</a> performing HTTP redirects for easy developers and administor access</li> +</ul></li> +<li><p>Monitoring</p> + +<ul> +<li><a href="https://github.com/zircote/collectd-aurora";>collectd-aurora</a> for cluster monitoring using collectd</li> +<li><a href="https://github.com/tommyulfsparre/aurora_exporter";>Prometheus Aurora exporter</a> for cluster monitoring using Prometheus</li> +<li><a href="http://prometheus.io/docs/operating/configuration/#zookeeper-serverset-sd-configurations-serverset_sd_config";>Prometheus service discovery integration</a> for discovering and monitoring services running on Aurora</li> +</ul></li> +<li><p>Packaging and deployment</p> + +<ul> +<li><a href="https://github.com/apache/aurora-packaging";>aurora-packaging</a>, the source of the official Aurora packaes</li> +</ul></li> +</ul> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html> Added: aurora/site/publish/documentation/0.12.0/tutorial/index.html URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.12.0/tutorial/index.html?rev=1733548&view=auto ============================================================================== --- aurora/site/publish/documentation/0.12.0/tutorial/index.html (added) +++ aurora/site/publish/documentation/0.12.0/tutorial/index.html Fri Mar 4 02:43:01 2016 @@ -0,0 +1,369 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Apache Aurora</title> + <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css";> + <link href="/assets/css/main.css" rel="stylesheet"> + <!-- Analytics --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-45879646-1']); + _gaq.push(['_setDomainName', 'apache.org']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + </head> + <body> + <div class="container-fluid section-header"> + <div class="container"> + <div class="nav nav-bar"> + <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a> + <ul class="nav navbar-nav navbar-right"> + <li><a href="/documentation/latest/">Documentation</a></li> + <li><a href="/community/">Community</a></li> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/blog/">Blog</a></li> + </ul> + </div> + </div> +</div> + + <div class="container-fluid"> + <div class="container content"> + <div class="col-md-12 documentation"> +<h5 class="page-header text-uppercase">Documentation +<select onChange="window.location.href='/documentation/' + this.value + '/tutorial/'" + value="0.12.0"> + <option value="0.12.0" + selected="selected"> + 0.12.0 + (latest) + </option> + <option value="0.11.0" + > + 0.11.0 + </option> + <option value="0.10.0" + > + 0.10.0 + </option> + <option value="0.9.0" + > + 0.9.0 + </option> + <option value="0.8.0" + > + 0.8.0 + </option> + <option value="0.7.0-incubating" + > + 0.7.0-incubating + </option> + <option value="0.6.0-incubating" + > + 0.6.0-incubating + </option> + <option value="0.5.0-incubating" + > + 0.5.0-incubating + </option> +</select> +</h5> +<h1 id="aurora-tutorial">Aurora Tutorial</h1> + +<p>This tutorial shows how to use the Aurora scheduler to run (and “<code>printf-debug</code>”) +a hello world program on Mesos. This is the recommended document for new Aurora users +to start getting up to speed on the system.</p> + +<ul> +<li><a href="#setup-install-aurora">Prerequisite</a></li> +<li><a href="#the-script">The Script</a></li> +<li><a href="#aurora-configuration">Aurora Configuration</a></li> +<li><a href="#creating-the-job">Creating the Job</a></li> +<li><a href="#watching-the-job-run">Watching the Job Run</a></li> +<li><a href="#cleanup">Cleanup</a></li> +<li><a href="#next-steps">Next Steps</a></li> +</ul> + +<h2 id="prerequisite">Prerequisite</h2> + +<p>This tutorial assumes you are running <a href="/documentation/0.12.0/vagrant/">Aurora locally using Vagrant</a>. +However, in general the instructions are also applicable to any other +<a href="/documentation/0.12.0/installing/">Aurora installation</a>.</p> + +<p>Unless otherwise stated, all commands are to be run from the root of the aurora +repository clone.</p> + +<h2 id="the-script">The Script</h2> + +<p>Our “hello world” application is a simple Python script that loops +forever, displaying the time every few seconds. Copy the code below and +put it in a file named <code>hello_world.py</code> in the root of your Aurora repository clone +(Note: this directory is the same as <code>/vagrant</code> inside the Vagrant VMs).</p> + +<p>The script has an intentional bug, which we will explain later on.</p> + +<!-- NOTE: If you are changing this file, be sure to also update examples/vagrant/test_tutorial.sh. +--> +<pre class="highlight python"><code><span style="color: #000000;font-weight: bold">import</span> <span style="color: #555555">time</span> + +<span style="color: #000000;font-weight: bold">def</span> <span style="color: #990000;font-weight: bold">main</span><span style="background-color: #f8f8f8">():</span> + <span style="background-color: #f8f8f8">SLEEP_DELAY</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #009999">10</span> + <span style="color: #999988;font-style: italic"># Python ninjas - ignore this blatant bug.</span> + <span style="color: #000000;font-weight: bold">for</span> <span style="background-color: #f8f8f8">i</span> <span style="color: #000000;font-weight: bold">in</span> <span style="background-color: #f8f8f8">xrang</span><span style="background-color: #f8f8f8">(</span><span style="color: #009999">100</span><span style="background-color: #f8f8f8">):</span> + <span style="color: #000000;font-weight: bold">print</span><span style="background-color: #f8f8f8">(</span><span style="color: #d14">"Hello world! The time is now: </span><span style="color: #d14">%</span><span style="color: #d14">s. Sleeping for </span><span style="color: #d14">%</span><span style="color: #d14">d secs"</span> <span style="color: #000000;font-weight: bold">%</span> <span style="background-color: #f8f8f8">(</span> + <span style="background-color: #f8f8f8">time</span><span style="color: #000000;font-weight: bold">.</span><span style="background-color: #f8f8f8">asctime</span><span style="background-color: #f8f8f8">(),</span> <span style="background-color: #f8f8f8">SLEEP_DELAY</span><span style="background-color: #f8f8f8">))</span> + <span style="background-color: #f8f8f8">time</span><span style="color: #000000;font-weight: bold">.</span><span style="background-color: #f8f8f8">sleep</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">SLEEP_DELAY</span><span style="background-color: #f8f8f8">)</span> + +<span style="color: #000000;font-weight: bold">if</span> <span style="background-color: #f8f8f8">__name__</span> <span style="color: #000000;font-weight: bold">==</span> <span style="color: #d14">"__main__"</span><span style="background-color: #f8f8f8">:</span> + <span style="background-color: #f8f8f8">main</span><span style="background-color: #f8f8f8">()</span> +</code></pre> + +<h2 id="aurora-configuration">Aurora Configuration</h2> + +<p>Once we have our script/program, we need to create a <em>configuration +file</em> that tells Aurora how to manage and launch our Job. Save the below +code in the file <code>hello_world.aurora</code>.</p> + +<!-- NOTE: If you are changing this file, be sure to also update examples/vagrant/test_tutorial.sh. +--> +<pre class="highlight python"><code><span style="background-color: #f8f8f8">pkg_path</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #d14">'/vagrant/hello_world.py'</span> + +<span style="color: #999988;font-style: italic"># we use a trick here to make the configuration change with</span> +<span style="color: #999988;font-style: italic"># the contents of the file, for simplicity. in a normal setting, packages would be</span> +<span style="color: #999988;font-style: italic"># versioned, and the version number would be changed in the configuration.</span> +<span style="color: #000000;font-weight: bold">import</span> <span style="color: #555555">hashlib</span> +<span style="color: #000000;font-weight: bold">with</span> <span style="color: #0086B3">open</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">pkg_path</span><span style="background-color: #f8f8f8">,</span> <span style="color: #d14">'rb'</span><span style="background-color: #f8f8f8">)</span> <span style="color: #000000;font-weight: bold">as</span> <span style="background-color: #f8f8f8">f</span><span style="background-color: #f8f8f8">:</span> + <span style="background-color: #f8f8f8">pkg_checksum</span> <span style="color: #000000;font-weight: bold">=</span> <span style="background-color: #f8f8f8">hashlib</span><span style="color: #000000;font-weight: bold">.</span><span style="background-color: #f8f8f8">md5</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">f</span><span style="color: #000000;font-weight: bold">.</span><span style="background-color: #f8f8f8">read</span><span style="background-color: #f8f8f8">())</span><span style="color: #000000;font-weight: bold">.</span><span style="background-color: #f8f8f8">hexdigest</span><span style="background-color: #f8f8f8">()</span> + +<span style="color: #999988;font-style: italic"># copy hello_world.py into the local sandbox</span> +<span style="background-color: #f8f8f8">install</span> <span style="color: #000000;font-weight: bold">=</span> <span style="background-color: #f8f8f8">Process</span><span style="background-color: #f8f8f8">(</span> + <span style="background-color: #f8f8f8">name</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #d14">'fetch_package'</span><span style="background-color: #f8f8f8">,</span> + <span style="background-color: #f8f8f8">cmdline</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #d14">'cp </span><span style="color: #d14">%</span><span style="color: #d14">s . && echo </span><span style="color: #d14">%</span><span style="color: #d14">s && chmod +x hello_world.py'</span> <span style="color: #000000;font-weight: bold">%</span> <span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">pkg_path</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">pkg_checksum</span><span style="background-color: #f8f8f8">))</span> + +<span style="color: #999988;font-style: italic"># run the script</span> +<span style="background-color: #f8f8f8">hello_world</span> <span style="color: #000000;font-weight: bold">=</span> <span style="background-color: #f8f8f8">Process</span><span style="background-color: #f8f8f8">(</span> + <span style="background-color: #f8f8f8">name</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #d14">'hello_world'</span><span style="background-color: #f8f8f8">,</span> + <span style="background-color: #f8f8f8">cmdline</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #d14">'python -u hello_world.py'</span><span style="background-color: #f8f8f8">)</span> + +<span style="color: #999988;font-style: italic"># describe the task</span> +<span style="background-color: #f8f8f8">hello_world_task</span> <span style="color: #000000;font-weight: bold">=</span> <span style="background-color: #f8f8f8">SequentialTask</span><span style="background-color: #f8f8f8">(</span> + <span style="background-color: #f8f8f8">processes</span> <span style="color: #000000;font-weight: bold">=</span> <span style="background-color: #f8f8f8">[</span><span style="background-color: #f8f8f8">install</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">hello_world</span><span style="background-color: #f8f8f8">],</span> + <span style="background-color: #f8f8f8">resources</span> <span style="color: #000000;font-weight: bold">=</span> <span style="background-color: #f8f8f8">Resources</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">cpu</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #009999">1</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">ram</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #009999">1</span><span style="color: #000000;font-weight: bold">*</span><span style="background-color: #f8f8f8">MB</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">disk</span><span style="color: #000000;font-weight: bold">=</span><span style="color: #009999">8</span><span style="color: #000000;font-weight: bold">*</span><span style="background-color: #f8f8f8">MB</span><span style="background-color: #f8f8f8">) )</span> + +<span style="background-color: #f8f8f8">jobs</span> <span style="color: #000000;font-weight: bold">=</span> <span style="background-color: #f8f8f8">[</span> + <span style="background-color: #f8f8f8">Service</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">cluster</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #d14">'devcluster'</span><span style="background-color: #f8f8f8">,</span> + <span style="background-color: #f8f8f8">environment</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #d14">'devel'</span><span style="background-color: #f8f8f8">,</span> + <span style="background-color: #f8f8f8">role</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #d14">'www-data'</span><span style="background-color: #f8f8f8">,</span> + <span style="background-color: #f8f8f8">name</span> <span style="color: #000000;font-weight: bold">=</span> <span style="color: #d14">'hello_world'</span><span style="background-color: #f8f8f8">,</span> + <span style="background-color: #f8f8f8">task</span> <span style="color: #000000;font-weight: bold">=</span> <span style="background-color: #f8f8f8">hello_world_task</span><span style="background-color: #f8f8f8">)</span> +<span style="background-color: #f8f8f8">]</span> +</code></pre> + +<p>There is a lot going on in that configuration file:</p> + +<ol> +<li><p>From a “big picture” viewpoint, it first defines two +Processes. Then it defines a Task that runs the two Processes in the +order specified in the Task definition, as well as specifying what +computational and memory resources are available for them. Finally, +it defines a Job that will schedule the Task on available and suitable +machines. This Job is the sole member of a list of Jobs; you can +specify more than one Job in a config file.</p></li> +<li><p>At the Process level, it specifies how to get your code into the +local sandbox in which it will run. It then specifies how the code is +actually run once the second Process starts.</p></li> +</ol> + +<p>For more about Aurora configuration files, see the <a href="/documentation/0.12.0/configuration-tutorial/">Configuration +Tutorial</a> and the <a href="/documentation/0.12.0/configuration-reference/">Aurora + Thermos +Reference</a> (preferably after finishing this +tutorial).</p> + +<h2 id="creating-the-job">Creating the Job</h2> + +<p>We’re ready to launch our job! To do so, we use the Aurora Client to +issue a Job creation request to the Aurora scheduler.</p> + +<p>Many Aurora Client commands take a <em>job key</em> argument, which uniquely +identifies a Job. A job key consists of four parts, each separated by a +“/”. The four parts are <code><cluster>/<role>/<environment>/<jobname></code> +in that order:</p> + +<ul> +<li>Cluster refers to the name of a particular Aurora installation.</li> +<li>Role names are user accounts existing on the slave machines. If you +don’t know what accounts are available, contact your sysadmin.</li> +<li>Environment names are namespaces; you can count on <code>test</code>, <code>devel</code>, +<code>staging</code> and <code>prod</code> existing.</li> +<li>Jobname is the custom name of your job.</li> +</ul> + +<p>When comparing two job keys, if any of the four parts is different from +its counterpart in the other key, then the two job keys identify two separate +jobs. If all four values are identical, the job keys identify the same job.</p> + +<p>The <code>clusters.json</code> <a href="/documentation/0.12.0/client-cluster-configuration/">client configuration</a> +for the Aurora scheduler defines the available cluster names. +For Vagrant, from the top-level of your Aurora repository clone, do:</p> +<pre class="highlight plaintext"><code>$ vagrant ssh +</code></pre> + +<p>Followed by:</p> +<pre class="highlight plaintext"><code>vagrant@aurora:~$ cat /etc/aurora/clusters.json +</code></pre> + +<p>You’ll see something like the following. The <code>name</code> value shown here, corresponds to a job key’s cluster value.</p> +<pre class="highlight javascript"><code><span style="background-color: #f8f8f8">[{</span> + <span style="color: #d14">"name"</span><span style="background-color: #f8f8f8">:</span> <span style="color: #d14">"devcluster"</span><span style="background-color: #f8f8f8">,</span> + <span style="color: #d14">"zk"</span><span style="background-color: #f8f8f8">:</span> <span style="color: #d14">"192.168.33.7"</span><span style="background-color: #f8f8f8">,</span> + <span style="color: #d14">"scheduler_zk_path"</span><span style="background-color: #f8f8f8">:</span> <span style="color: #d14">"/aurora/scheduler"</span><span style="background-color: #f8f8f8">,</span> + <span style="color: #d14">"auth_mechanism"</span><span style="background-color: #f8f8f8">:</span> <span style="color: #d14">"UNAUTHENTICATED"</span><span style="background-color: #f8f8f8">,</span> + <span style="color: #d14">"slave_run_directory"</span><span style="background-color: #f8f8f8">:</span> <span style="color: #d14">"latest"</span><span style="background-color: #f8f8f8">,</span> + <span style="color: #d14">"slave_root"</span><span style="background-color: #f8f8f8">:</span> <span style="color: #d14">"/var/lib/mesos"</span> +<span style="background-color: #f8f8f8">}]</span> +</code></pre> + +<p>The Aurora Client command that actually runs our Job is <code>aurora job create</code>. It creates a Job as +specified by its job key and configuration file arguments and runs it.</p> +<pre class="highlight plaintext"><code>aurora job create <cluster>/<role>/<environment>/<jobname> <config_file> +</code></pre> + +<p>Or for our example:</p> +<pre class="highlight plaintext"><code>aurora job create devcluster/www-data/devel/hello_world /vagrant/hello_world.aurora +</code></pre> + +<p>After entering our virtual machine using <code>vagrant ssh</code>, this returns:</p> +<pre class="highlight plaintext"><code>vagrant@aurora:~$ aurora job create devcluster/www-data/devel/hello_world /vagrant/hello_world.aurora + INFO] Creating job hello_world + INFO] Checking status of devcluster/www-data/devel/hello_world +Job create succeeded: job url=http://aurora.local:8081/scheduler/www-data/devel/hello_world +</code></pre> + +<h2 id="watching-the-job-run">Watching the Job Run</h2> + +<p>Now that our job is running, let’s see what it’s doing. Access the +scheduler web interface at <code>http://$scheduler_hostname:$scheduler_port/scheduler</code> +Or when using <code>vagrant</code>, <code>http://192.168.33.7:8081/scheduler</code> +First we see what Jobs are scheduled:</p> + +<p><img alt="Scheduled Jobs" src="../images/ScheduledJobs.png" /></p> + +<p>Click on your user name, which in this case was <code>www-data</code>, and we see the Jobs associated +with that role:</p> + +<p><img alt="Role Jobs" src="../images/RoleJobs.png" /></p> + +<p>If you click on your <code>hello_world</code> Job, you’ll see:</p> + +<p><img alt="hello_world Job" src="../images/HelloWorldJob.png" /></p> + +<p>Oops, looks like our first job didn’t quite work! The task is temporarily throttled for +having failed on every attempt of the Aurora scheduler to run it. We have to figure out +what is going wrong.</p> + +<p>On the Completed tasks tab, we see all past attempts of the Aurora scheduler to run our job.</p> + +<p><img alt="Completed tasks tab" src="../images/CompletedTasks.png" /></p> + +<p>We can navigate to the Task page of a failed run by clicking on the host link.</p> + +<p><img alt="Task page" src="../images/TaskBreakdown.png" /></p> + +<p>Once there, we see that the <code>hello_world</code> process failed. The Task page +captures the standard error and standard output streams and makes them available. +Clicking through to <code>stderr</code> on the failed <code>hello_world</code> process, we see what happened.</p> + +<p><img alt="stderr page" src="../images/stderr.png" /></p> + +<p>It looks like we made a typo in our Python script. We wanted <code>xrange</code>, +not <code>xrang</code>. Edit the <code>hello_world.py</code> script to use the correct function +and save it as <code>hello_world_v2.py</code>. Then update the <code>hello_world.aurora</code> +configuration to the newest version.</p> + +<p>In order to try again, we can now instruct the scheduler to update our job:</p> +<pre class="highlight plaintext"><code>vagrant@aurora:~$ aurora update start devcluster/www-data/devel/hello_world /vagrant/hello_world.aurora + INFO] Starting update for: hello_world +Job update has started. View your update progress at http://aurora.local:8081/scheduler/www-data/devel/hello_world/update/8ef38017-e60f-400d-a2f2-b5a8b724e95b +</code></pre> + +<p>This time, the task comes up.</p> + +<p><img alt="Running Job" src="../images/RunningJob.png" /></p> + +<p>By again clicking on the host, we inspect the Task page, and see that the +<code>hello_world</code> process is running.</p> + +<p><img alt="Running Task page" src="../images/runningtask.png" /></p> + +<p>We then inspect the output by clicking on <code>stdout</code> and see our process’ +output:</p> + +<p><img alt="stdout page" src="../images/stdout.png" /></p> + +<h2 id="cleanup">Cleanup</h2> + +<p>Now that we’re done, we kill the job using the Aurora client:</p> +<pre class="highlight plaintext"><code>vagrant@aurora:~$ aurora job killall devcluster/www-data/devel/hello_world + INFO] Killing tasks for job: devcluster/www-data/devel/hello_world + INFO] Instances to be killed: [0] +Successfully killed instances [0] +Job killall succeeded +</code></pre> + +<p>The job page now shows the <code>hello_world</code> tasks as completed.</p> + +<p><img alt="Killed Task page" src="../images/killedtask.png" /></p> + +<h2 id="next-steps">Next Steps</h2> + +<p>Now that you’ve finished this Tutorial, you should read or do the following:</p> + +<ul> +<li><a href="/documentation/0.12.0/configuration-tutorial/">The Aurora Configuration Tutorial</a>, which provides more examples +and best practices for writing Aurora configurations. You should also look at +the <a href="/documentation/0.12.0/configuration-reference/">Aurora + Thermos Configuration Reference</a>.</li> +<li>The <a href="/documentation/0.12.0/user-guide/">Aurora User Guide</a> provides an overview of how Aurora, Mesos, and +Thermos work “under the hood”.</li> +<li>Explore the Aurora Client - use <code>aurora -h</code>, and read the +<a href="/documentation/0.12.0/client-commands/">Aurora Client Commands</a> document.</li> +</ul> + +</div> + + </div> + </div> + <div class="container-fluid section-footer buffer"> + <div class="container"> + <div class="row"> + <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3> + <ul> + <li><a href="/downloads/">Downloads</a></li> + <li><a href="/community/">Mailing Lists</a></li> + <li><a href="http://issues.apache.org/jira/browse/AURORA";>Issue Tracking</a></li> + <li><a href="/documentation/latest/contributing/">How To Contribute</a></li> + </ul> + </div> + <div class="col-md-2"><h3>The ASF</h3> + <ul> + <li><a href="http://www.apache.org/licenses/";>License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + <li><a href="http://www.apache.org/security/";>Security</a></li> + </ul> + </div> + <div class="col-md-6"> + <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/";>Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/";>Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + + </body> +</html>
