Hey Ankur, your question is super timely, I've been working on a demo framework that shows exactly what you're trying to do with Jetty. The code is still a little rough and there are some hardcoded paths etc. but since you asked I just published it: https://github.com/guenter/jetty-mesos I'm also the main author of Marathon and auto scaling has been on my mind. The main question is what an interface for reporting load stats would look like. Curious what you think!
On Sun, Jan 12, 2014 at 9:14 PM, Ankur Chauhan <[email protected]> wrote: > Thanks everyone for all the help. > Marathon does seem like a good framework but my use case requires the app > to evaluate it's own health and scale up based on internal load stats (SLA > requirements) and I don't know if marathon supports that. This is the main > reason why i am looking at building out my own scheduler/executor. I will > give another go with Vinod's comments and have a look at the hadoop > scheduler. > > Just a recommendation to any mesos experts out there, it would be super > helpful if there was a complete mock app with annotated code somewhere. > Another good page on the website would a good FAQ page. > > I am still pretty n00b as far as mesos is concerned, so, pardon any stupid > comments/suggestions/questions. > > -- Ankur > > > On Fri, Dec 27, 2013 at 10:16 AM, Abhishek Parolkar <[email protected] > > wrote: > >> @Ankur, >> In case Marathon looks like direction you want to go with, I have a >> small demo in here if that helps >> http://www.youtube.com/watch?v=2YWVGMuMTrg >> >> -parolkar >> >> >> On Sat, Dec 28, 2013 at 2:10 AM, Vinod Kone <[email protected]> wrote: >> >>> I can't really find an example that is an end-to-end use case. By that I >>>> mean, I would like to know how to put the scheduler and the executor in the >>>> correct places. Right now I have a single jar with can be run from the >>>> command line: java -jar target/collector.jar and that would take care of >>>> everything. >>>> >>>> This collector.jar can act as both scheduler and executor, presumably >>> based on command line flags? If yes, thats certainly doable. Typically the >>> scheduler and executor are split into separate jars. This makes it easy to >>> decouple the upgrade of scheduler and executor. >>> >>> >>>> My current train of thought is that the webapp jar would stay somewhere >>>> on an S3 url and the "CollectorScheduler" would "somehow" tell a mesos >>>> slave to run the "CollectorExecutor" which in turn fetch the jar from S3 >>>> and run it. >>>> >>>> >>> Yes, you are on the right track. Mesos slave can download the jar for >>> you as long as it could be accessed via (http://, https://, ftp://, >>> hdfs:// etc). This is how you do it: >>> >>> When you launch a task from the scheduler via 'launchTaks()' you give it >>> a 'vector<TaskInfo>' as one of the arguments. Since you are using a custom >>> executor you should set 'TaskInfo.ExecutorInfo' (see mesos.proto) to point >>> to your executor. To specify the S3 URL you would set >>> 'TaskInfo.ExecutorInfo.CommandInfo.URI.value'. To tell slave the command to >>> launch the executor after it downloads the the executor, you would set >>> 'TaskInfo.ExecutorInfo.CommandInfo.value'. >>> >>> You can find some examples here: >>> >>> Hadoop >>> scheduler<https://github.com/mesos/hadoop/blob/master/src/main/java/org/apache/hadoop/mapred/ResourcePolicy.java> >>> >>> Example Java scheduler >>> <https://github.com/apache/mesos/blob/master/src/examples/java/TestFramework.java> >>> >>> Hope that helps. Let us know if you have additional questions. >>> >>> Vinod >>> >> >> >
