Modified: aurora/site/source/documentation/latest/tutorial.md URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/tutorial.md?rev=1733548&r1=1733547&r2=1733548&view=diff ============================================================================== --- aurora/site/source/documentation/latest/tutorial.md (original) +++ aurora/site/source/documentation/latest/tutorial.md Fri Mar 4 02:43:01 2016 @@ -1,65 +1,52 @@ -Aurora Tutorial ---------------- +# Aurora Tutorial -- [Introduction](#introduction) -- [Setup: Install Aurora](#setup-install-aurora) +This tutorial shows how to use the Aurora scheduler to run (and "`printf-debug`") +a hello world program on Mesos. This is the recommended document for new Aurora users +to start getting up to speed on the system. + +- [Prerequisite](#setup-install-aurora) - [The Script](#the-script) - [Aurora Configuration](#aurora-configuration) -- [What's Going On In That Configuration File?](#whats-going-on-in-that-configuration-file) - [Creating the Job](#creating-the-job) - [Watching the Job Run](#watching-the-job-run) - [Cleanup](#cleanup) - [Next Steps](#next-steps) -## Introduction - -This tutorial shows how to use the Aurora scheduler to run (and -"`printf-debug`") a hello world program on Mesos. The operational -hierarchy is: -- Aurora manages and schedules jobs for Mesos to run. -- Mesos manages the individual tasks that make up a job. -- Thermos manages the individual processes that make up a task. +## Prerequisite -This is the recommended first Aurora users document to read to start -getting up to speed on the system. +This tutorial assumes you are running [Aurora locally using Vagrant](/documentation/latest/vagrant/). +However, in general the instructions are also applicable to any other +[Aurora installation](/documentation/latest/installing/). -To get help, email questions to the Aurora Developer List, -[[email protected]](mailto:[email protected]) +Unless otherwise stated, all commands are to be run from the root of the aurora +repository clone. -## Setup: Install Aurora - -You use the Aurora client and web UI to interact with Aurora jobs. To -install it locally, see [vagrant.md](/documentation/latest/vagrant/). The remainder of this -Tutorial assumes you are running Aurora using Vagrant. Unless otherwise stated, -all commands are to be run from the root of the aurora repository clone. ## The Script Our "hello world" application is a simple Python script that loops forever, displaying the time every few seconds. Copy the code below and -put it in a file named `hello_world.py` in the root of your Aurora repository clone (Note: -this directory is the same as `/vagrant` inside the Vagrant VMs). +put it in a file named `hello_world.py` in the root of your Aurora repository clone +(Note: this directory is the same as `/vagrant` inside the Vagrant VMs). The script has an intentional bug, which we will explain later on. <!-- NOTE: If you are changing this file, be sure to also update examples/vagrant/test_tutorial.sh. --> ```python -import sys import time -def main(argv): +def main(): SLEEP_DELAY = 10 # Python ninjas - ignore this blatant bug. for i in xrang(100): print("Hello world! The time is now: %s. Sleeping for %d secs" % ( time.asctime(), SLEEP_DELAY)) - sys.stdout.flush() time.sleep(SLEEP_DELAY) if __name__ == "__main__": - main(sys.argv) + main() ``` ## Aurora Configuration @@ -88,7 +75,7 @@ install = Process( # run the script hello_world = Process( name = 'hello_world', - cmdline = 'python hello_world.py') + cmdline = 'python -u hello_world.py') # describe the task hello_world_task = SequentialTask( @@ -104,14 +91,7 @@ jobs = [ ] ``` -For more about Aurora configuration files, see the [Configuration -Tutorial](/documentation/latest/configuration-tutorial/) and the [Aurora + Thermos -Reference](/documentation/latest/configuration-reference/) (preferably after finishing this -tutorial). - -## What's Going On In That Configuration File? - -More than you might think. +There is a lot going on in that configuration file: 1. From a "big picture" viewpoint, it first defines two Processes. Then it defines a Task that runs the two Processes in the @@ -125,6 +105,12 @@ specify more than one Job in a config fi local sandbox in which it will run. It then specifies how the code is actually run once the second Process starts. +For more about Aurora configuration files, see the [Configuration +Tutorial](/documentation/latest/configuration-tutorial/) and the [Aurora + Thermos +Reference](/documentation/latest/configuration-reference/) (preferably after finishing this +tutorial). + + ## Creating the Job We're ready to launch our job! To do so, we use the Aurora Client to @@ -133,39 +119,42 @@ issue a Job creation request to the Auro Many Aurora Client commands take a *job key* argument, which uniquely identifies a Job. A job key consists of four parts, each separated by a "/". The four parts are `<cluster>/<role>/<environment>/<jobname>` -in that order. When comparing two job keys, if any of the -four parts is different from its counterpart in the other key, then the -two job keys identify two separate jobs. If all four values are -identical, the job keys identify the same job. - -`/etc/aurora/clusters.json` within the Aurora scheduler has the available -cluster names. For Vagrant, from the top-level of your Aurora repository clone, -do: +in that order: + +* Cluster refers to the name of a particular Aurora installation. +* Role names are user accounts existing on the slave machines. If you +don't know what accounts are available, contact your sysadmin. +* Environment names are namespaces; you can count on `test`, `devel`, +`staging` and `prod` existing. +* Jobname is the custom name of your job. + +When comparing two job keys, if any of the four parts is different from +its counterpart in the other key, then the two job keys identify two separate +jobs. If all four values are identical, the job keys identify the same job. + +The `clusters.json` [client configuration](/documentation/latest/client-cluster-configuration/) +for the Aurora scheduler defines the available cluster names. +For Vagrant, from the top-level of your Aurora repository clone, do: $ vagrant ssh Followed by: - vagrant@precise64:~$ cat /etc/aurora/clusters.json + vagrant@aurora:~$ cat /etc/aurora/clusters.json -You'll see something like: +You'll see something like the following. The `name` value shown here, corresponds to a job key's cluster value. ```javascript [{ "name": "devcluster", "zk": "192.168.33.7", "scheduler_zk_path": "/aurora/scheduler", - "auth_mechanism": "UNAUTHENTICATED" + "auth_mechanism": "UNAUTHENTICATED", + "slave_run_directory": "latest", + "slave_root": "/var/lib/mesos" }] ``` -Use a `name` value for your job key's cluster value. - -Role names are user accounts existing on the slave machines. If you don't know what accounts -are available, contact your sysadmin. - -Environment names are namespaces; you can count on `prod`, `devel` and `test` existing. - The Aurora Client command that actually runs our Job is `aurora job create`. It creates a Job as specified by its job key and configuration file arguments and runs it. @@ -175,20 +164,13 @@ Or for our example: aurora job create devcluster/www-data/devel/hello_world /vagrant/hello_world.aurora -This returns: - - $ vagrant ssh - Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64) +After entering our virtual machine using `vagrant ssh`, this returns: - * Documentation: https://help.ubuntu.com/ - Welcome to your Vagrant-built virtual machine. - Last login: Fri Jan 3 02:18:55 2014 from 10.0.2.2 - vagrant@precise64:~$ aurora job create devcluster/www-data/devel/hello_world \ - /vagrant/hello_world.aurora + vagrant@aurora:~$ aurora job create devcluster/www-data/devel/hello_world /vagrant/hello_world.aurora INFO] Creating job hello_world - INFO] Response from scheduler: OK (message: 1 new tasks pending for job - www-data/devel/hello_world) - INFO] Job url: http://precise64:8081/scheduler/www-data/devel/hello_world + INFO] Checking status of devcluster/www-data/devel/hello_world + Job create succeeded: job url=http://aurora.local:8081/scheduler/www-data/devel/hello_world + ## Watching the Job Run @@ -208,27 +190,40 @@ If you click on your `hello_world` Job,  -Oops, looks like our first job didn't quite work! The task failed, so we have -to figure out what went wrong. +Oops, looks like our first job didn't quite work! The task is temporarily throttled for +having failed on every attempt of the Aurora scheduler to run it. We have to figure out +what is going wrong. + +On the Completed tasks tab, we see all past attempts of the Aurora scheduler to run our job. + + -Access the page for our Task by clicking on its host. +We can navigate to the Task page of a failed run by clicking on the host link.  -Once there, we see that the -`hello_world` process failed. The Task page captures the standard error and -standard output streams and makes them available. Clicking through -to `stderr` on the failed `hello_world` process, we see what happened. +Once there, we see that the `hello_world` process failed. The Task page +captures the standard error and standard output streams and makes them available. +Clicking through to `stderr` on the failed `hello_world` process, we see what happened.  It looks like we made a typo in our Python script. We wanted `xrange`, -not `xrang`. Edit the `hello_world.py` script to use the correct function and -we will try again. +not `xrang`. Edit the `hello_world.py` script to use the correct function +and save it as `hello_world_v2.py`. Then update the `hello_world.aurora` +configuration to the newest version. + +In order to try again, we can now instruct the scheduler to update our job: + + vagrant@aurora:~$ aurora update start devcluster/www-data/devel/hello_world /vagrant/hello_world.aurora + INFO] Starting update for: hello_world + Job update has started. View your update progress at http://aurora.local:8081/scheduler/www-data/devel/hello_world/update/8ef38017-e60f-400d-a2f2-b5a8b724e95b + +This time, the task comes up. - aurora update start devcluster/www-data/devel/hello_world /vagrant/hello_world.aurora + -This time, the task comes up, we inspect the page, and see that the +By again clicking on the host, we inspect the Task page, and see that the `hello_world` process is running.  @@ -242,11 +237,11 @@ output: Now that we're done, we kill the job using the Aurora client: - vagrant@precise64:~$ aurora job killall devcluster/www-data/devel/hello_world + vagrant@aurora:~$ aurora job killall devcluster/www-data/devel/hello_world INFO] Killing tasks for job: devcluster/www-data/devel/hello_world - INFO] Response from scheduler: OK (message: Tasks killed.) - INFO] Job url: http://precise64:8081/scheduler/www-data/devel/hello_world - vagrant@precise64:~$ + INFO] Instances to be killed: [0] + Successfully killed instances [0] + Job killall succeeded The job page now shows the `hello_world` tasks as completed.
Modified: aurora/site/source/documentation/latest/user-guide.md URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/user-guide.md?rev=1733548&r1=1733547&r2=1733548&view=diff ============================================================================== --- aurora/site/source/documentation/latest/user-guide.md (original) +++ aurora/site/source/documentation/latest/user-guide.md Fri Mar 4 02:43:01 2016 @@ -170,7 +170,7 @@ and performing these operations: then that instance is killed. - If an instance is not present in the scheduler but is present in the new config, then the instance is created. -- If an instance is present in both the scheduler the new config, then +- If an instance is present in both the scheduler and the new config, then the client diffs both task configs. If it detects any changes, it performs an instance update by killing the old config instance and adds the new config instance.
