Hi Srinivas, Can you elaborate more about what does maintaining a dynamic count of executors?
You can always write a custom framework that provides the scheduling, similiar to what Marathon or Aurora is doing if it doesn't fit your need. Tim On Fri, Jan 9, 2015 at 1:18 PM, Srinivas Murthy <[email protected]> wrote: > Thanks Vinod. I need to deal with a very conservative management that > needs a lot of selling for each additional open source framework. I have > glossed over Marathon so far. I was hoping to hear there's some way I could > override the Scheduler and work with what I have, but I hear you say that > isn't the route I should be pursuing :-) > > > On Fri, Jan 9, 2015 at 11:43 AM, Vinod Kone <[email protected]> wrote: > >> Have you looked at Aurora or Marathon? They have some (most?) of the >> features you are looking for. >> >> On Fri, Jan 9, 2015 at 10:59 AM, Srinivas Murthy <[email protected]> >> wrote: >> >>> We have a legacy system with home brewn workflows defined in XPDL, >>> running across multiple dozens of nodes. Resources are mapped in XML >>> definition files, and availability of resource to a given task at hand >>> managed by a custom written job scheduler. Jobs communicate status with >>> callback/JMS messages. Job completion decides steps in the workflow. >>> >>> To this eco system now comes some Hadoop/Spark jobs. >>> I am tentatively exploring Mesos to manage this disparate set of >>> clusters. >>> How can I maintain a dynamic count of Executors, how can I provide >>> dynamic workflow orchestration to pull off above architecture in the Mesos >>> world? Sorry for the noob question! >>> >> >> >
