Stephen, I am trying to get started and run aurora without thermos executor (setting it to /dev/null does not help) - on a local linux box for now & planning to containerize/dockerize it later.
Can you please point me to the right documentation (or a pointer to the cli parsing source code) which can help me resolve this? Also, are there any steps steps to import source code into eclipse to browse & analyze code for this. Also, where do i find all the *.pex files? They are not present in the zip file nor anywhere in the built source code. I know I am asking too many queries on a single thread here, & would appreciate the help. I think at the end of this, I will put the steps I followed in a gist/blog so others might find their way around, & not struggle as much. -- κρισhναν On Tue, Oct 20, 2015 at 4:09 PM, Erb, Stephan <[email protected]> wrote: > Hi Krish, > > > you don't have to set framework_authentication_file and > zk_digest_credentials. The scheduler help text is misleading here as > everything will work fine if you leave those empty. > > > In addition, looks like you are misunderstanding the usage of the > thermos_executor_path command line flag of the scheduler. It is supposed > to point to the binary containing the generic Aurora executor > (thermos_executor.pex). You only need the hello_world.aurora once your > scheduler is up an running. It serves as an example input for the aurora > command line client which can be used to scheduler jobs and services on an > Aurora master. > > > Have you tried to use the vagrant box? Just type 'vagrant up`in a checkout > of the Aurora source code. It gives you a running scheduler to play with. > Once you have understood how it works, you can start trying to install it > on your own (by reverse-engineering the vagrant box). > > > Hope this helps a little, > > Stephan > > > > ------------------------------ > *From:* Krish <[email protected]> > *Sent:* Tuesday, October 20, 2015 11:39 AM > *To:* Bill Farner > *Cc:* [email protected]; Erb, Stephan > > *Subject:* Re: Stacktrace when running Apache Aurora > > Bill/Stephen, > I still get a stacktrace when running the aurora scheduler CLI. > > I do not know what to specify for -framework_authentication_file > & -zk_digest_credentials, and they are required arguments. > > I am not using any authentication on Mesos master, do I still need the > framework_authentication_file parameter? > > > rm -rf /db /backup_dir > mesos-log initialize --path="/db" > export JAVA_HOME=/usr/lib/jvm/java-8-oracle/ > JAVA_OPTS="-Xmx1536m -Xms256m" > /usr/local/aurora-scheduler/bin/aurora-scheduler -backup_dir=/backup_dir > -cluster_name=tc -mesos_master_address=zk://localhost:2181/mesos/master > -serverset_path=/scheduler/aurora -zk_endpoints=localhost:2181 > -native_log_quorum_size=1 -vlog=SEVERE -logtostderr=false > -native_log_file_path=/db > -thermos_executor_path=/home/ubuntu/hello_world.aurora > ... > ... > INFO: Binding org.apache.aurora.scheduler.http.Utilization to > GuiceManagedCompon > entProvider with the scope "PerRequest" > Oct 20, 2015 9:27:40 AM org.apache.aurora.scheduler.cron.quartz.CronModule > provi > deTimeZone > WARNING: Cron schedules are configured to fire according to timezone > Greenwich M > ean Time but system timezone is set to Coordinated Universal Time > Oct 20, 2015 9:27:41 AM org.eclipse.jetty.server.AbstractConnector doStart > INFO: Started [email protected]:43843 > E1020 09:27:41.290 THREAD1 > org.apache.aurora.scheduler.SchedulerLifecycle$9.exec > ute: Caught unchecked exception: com.google.inject.ProvisionException: > Guice pro > vision errors: > > 1) Error in custom provider, java.lang.IllegalArgumentException: Path > cannot be null at > org.apache.aurora.scheduler.log.mesos.MesosLogStreamModule.provideLog(Mesos > LogStreamModule.java:117) > at > org.apache.aurora.scheduler.log.mesos.MesosLogStreamModule.provideLog(Mesos > LogStreamModule.java:117) > while locating org.apache.mesos.Log > at > org.apache.aurora.scheduler.log.mesos.MesosLogStreamModule.provideLogInterf > ace(MesosLogStreamModule.java:152) > while locating org.apache.aurora.scheduler.log.mesos.LogInterface > > 1 error > com.google.inject.ProvisionException: Guice provision errors: > > 1) Error in custom provider, java.lang.IllegalArgumentException: Path > cannot be > null > at > org.apache.aurora.scheduler.log.mesos.MesosLogStreamModule.provideLog(Mesos > LogStreamModule.java:117) > at > org.apache.aurora.scheduler.log.mesos.MesosLogStreamModule.provideLog(Mesos > LogStreamModule.java:117) > while locating org.apache.mesos.Log > at > org.apache.aurora.scheduler.log.mesos.MesosLogStreamModule.provideLogInterf > ace(MesosLogStreamModule.java:152) > while locating org.apache.aurora.scheduler.log.mesos.LogInterface > > 1 error > at > com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987) > at > org.apache.aurora.scheduler.log.mesos.MesosLog.open(MesosLog.java:136 > > > > -- > κρισhναν > > On Tue, Oct 20, 2015 at 6:14 AM, Bill Farner <[email protected]> wrote: > >> The typical flow is that you keep your .aurora file checked into git, and >> commit every time you deploy/update. When you change your file, you will >> instruct Aurora to update the live job (have a look at aurora update -h). >> Aurora will perform a rolling upgrade of your job to the new config. >> You'll use this same flow for updating your job's software as well as >> resizing the job. >> >> For (3), you could set up alerting for stats that the scheduler exports. >> Have a look here for monitoring background: >> https://github.com/apache/aurora/blob/master/docs/monitoring.md >> >> You'll find want to look at scheduler stats related to 'pending'. >> >> On Mon, Oct 19, 2015 at 12:16 PM, Krish <[email protected]> >> wrote: >> >>> Thanks for the pointer. Now I notice that the aurora-scheduler script >>> has the --thermos_executor_path as a mandatory requirement. >>> >>> I have a couple of questions on how the thermos_executor/.aurora config >>> file functions: >>> 1. Do we have to statically define the file beforehand? >>> 2. What happens when we want to dynamically change the config, say >>> increasing the number of instances of a service required? Does aurora >>> require a reboot then? >>> 3. How do I get notified about the message mesos sends when it cannot >>> schedule tasks for lack of resources? Should I depend on aurora for this or >>> try to look for a hook into mesos? >>> >>> I think a little bit of context would help here. >>> What I plan to check is to run a very basic job/task inside a docker >>> container with aurora & wait for a 'resource not available' message from >>> mesos, and accordingly call an api to spin up a new node in my cluster. >>> >>> >>> >>> >>> -- >>> κρισhναν >>> >>> On Mon, Oct 19, 2015 at 1:24 PM, Erb, Stephan < >>> [email protected]> wrote: >>> >>>> I believe you are missing the thermos_executor options that have to be >>>> passed to the scheduler command line. >>>> >>>> >>>> See >>>> https://github.com/apache/aurora/blob/33d7e2170a86f54722a02a2dc9cb1e09fb52df25/examples/vagrant/upstart/aurora-scheduler.conf#L39 >>>> for an example >>>> >>>> >>>> Best Regards, >>>> >>>> Stephan >>>> >>>> >>>> ------------------------------ >>>> *From:* Krish <[email protected]> >>>> *Sent:* Monday, October 19, 2015 8:45 AM >>>> *To:* [email protected] >>>> *Subject:* Re: Stacktrace when running Apache Aurora >>>> >>>> Hi, >>>> I am a n00b with apache aurora & trying to experiment some things on my >>>> local machine with zookeeper and mesos-master running locally. They have >>>> initialized properly. When I try to run aurora with the required options, I >>>> get the following error, & googing hasn't helped me much here. >>>> Appreciate any help. Thanks in advance. >>>> >>>> ... >>>> ... >>>> WARNING: Method [public void >>>> org.apache.aurora.scheduler.storage.log.SnapshotStoreImpl.applySnapshot(java.lang.Object)] >>>> is synthetic and is being intercepted by >>>> [com.twitter.common.inject.TimedInterceptor@604c5de8]. This could >>>> indicate a bug. The method >>>> may be intercepted twice, or may not be intercepted at all. >>>> Exception in thread "main" com.google.inject.CreationException: Guice >>>> creation errors: >>>> >>>> 1) An exception was caught and reported. Message: A value may only be >>>> retrieved from a variable that has a default or has been >>>> set. >>>> at >>>> com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:133) >>>> >>>> 2) Could not find a suitable constructor in >>>> org.apache.aurora.scheduler.mesos.ExecutorSettings. Classes must have >>>> either one (a >>>> nd only one) constructor annotated with @Inject or a zero-argument >>>> constructor that is not private. >>>> at >>>> org.apache.aurora.scheduler.mesos.ExecutorSettings.class(ExecutorSettings.java:43) >>>> at >>>> org.apache.aurora.scheduler.app.SchedulerMain$3.configure(SchedulerMain.java:204) >>>> >>>> 2 errors >>>> at >>>> com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:435) >>>> at >>>> com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:154) >>>> at >>>> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:106) >>>> at com.google.inject.Guice.createInjector(Guice.java:95) >>>> at com.google.inject.Guice.createInjector(Guice.java:83) >>>> at >>>> com.twitter.common.application.AppLauncher.configureInjection(AppLauncher.java:120) >>>> at >>>> com.twitter.common.application.AppLauncher.run(AppLauncher.java:87) >>>> at >>>> com.twitter.common.application.AppLauncher.launch(AppLauncher.java:181) >>>> at >>>> com.twitter.common.application.AppLauncher.launch(AppLauncher.java:142) >>>> at >>>> org.apache.aurora.scheduler.app.SchedulerMain.main(SchedulerMain.java:263) >>>> Caused by: java.lang.IllegalStateException: A value may only be >>>> retrieved from a variable that has a default or has been set. >>>> at >>>> com.google.common.base.Preconditions.checkState(Preconditions.java:176) >>>> at com.twitter.common.args.Arg.get(Arg.java:82) >>>> at >>>> org.apache.aurora.scheduler.app.SchedulerMain$3.configure(SchedulerMain.java:206) >>>> at >>>> com.google.inject.AbstractModule.configure(AbstractModule.java:59) >>>> at >>>> com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:223) >>>> at com.google.inject.util.Modules$2.configure(Modules.java:114) >>>> at >>>> com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:223) >>>> at com.google.inject.spi.Elements.getElements(Elements.java:101) >>>> at >>>> com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:133) >>>> at >>>> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:103) >>>> ... 7 more >>>> >>>> Complete logs are present @http://pastebin.com/i72HvbYi. >>>> >>>> >>>> >>>>> -- >>>>> κρισhναν >>>>> >>>> >>>> >>> >> >
