On Sun, Jan 12, 2014 at 9:14 PM, Ankur Chauhan <an...@malloc64.com> wrote:
> Thanks everyone for all the help. > Marathon does seem like a good framework but my use case requires the app > to evaluate it's own health and scale up based on internal load stats (SLA > requirements) and I don't know if marathon supports that. This is the main > reason why i am looking at building out my own scheduler/executor. I will > give another go with Vinod's comments and have a look at the hadoop > scheduler. > > Just a recommendation to any mesos experts out there, it would be super > helpful if there was a complete mock app with annotated code somewhere. > Another good page on the website would a good FAQ page. > > I am still pretty n00b as far as mesos is concerned, so, pardon any stupid > comments/suggestions/questions. > Apologies are not necessary! Questions like these are great for creating a public discussion for others to learn from, especially given the current lack of a FAQ document. Please reach out with further questions if you have any. > -- Ankur > > > On Fri, Dec 27, 2013 at 10:16 AM, Abhishek Parolkar <abhis...@parolkar.com > > wrote: > >> @Ankur, >> In case Marathon looks like direction you want to go with, I have a >> small demo in here if that helps >> http://www.youtube.com/watch?v=2YWVGMuMTrg >> >> -parolkar >> >> >> On Sat, Dec 28, 2013 at 2:10 AM, Vinod Kone <vinodk...@gmail.com> wrote: >> >>> I can't really find an example that is an end-to-end use case. By that I >>>> mean, I would like to know how to put the scheduler and the executor in the >>>> correct places. Right now I have a single jar with can be run from the >>>> command line: java -jar target/collector.jar and that would take care of >>>> everything. >>>> >>>> This collector.jar can act as both scheduler and executor, presumably >>> based on command line flags? If yes, thats certainly doable. Typically the >>> scheduler and executor are split into separate jars. This makes it easy to >>> decouple the upgrade of scheduler and executor. >>> >>> >>>> My current train of thought is that the webapp jar would stay somewhere >>>> on an S3 url and the "CollectorScheduler" would "somehow" tell a mesos >>>> slave to run the "CollectorExecutor" which in turn fetch the jar from S3 >>>> and run it. >>>> >>>> >>> Yes, you are on the right track. Mesos slave can download the jar for >>> you as long as it could be accessed via (http://, https://, ftp://, >>> hdfs:// etc). This is how you do it: >>> >>> When you launch a task from the scheduler via 'launchTaks()' you give it >>> a 'vector<TaskInfo>' as one of the arguments. Since you are using a custom >>> executor you should set 'TaskInfo.ExecutorInfo' (see mesos.proto) to point >>> to your executor. To specify the S3 URL you would set >>> 'TaskInfo.ExecutorInfo.CommandInfo.URI.value'. To tell slave the command to >>> launch the executor after it downloads the the executor, you would set >>> 'TaskInfo.ExecutorInfo.CommandInfo.value'. >>> >>> You can find some examples here: >>> >>> Hadoop >>> scheduler<https://github.com/mesos/hadoop/blob/master/src/main/java/org/apache/hadoop/mapred/ResourcePolicy.java> >>> >>> Example Java scheduler >>> <https://github.com/apache/mesos/blob/master/src/examples/java/TestFramework.java> >>> >>> Hope that helps. Let us know if you have additional questions. >>> >>> Vinod >>> >> >> >
