-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi,
@Sharma - Is mantis/fenzo available on github or something, I did find some maven artifacts but the repository netflix/fenzo is a 404. I am interested in learning about the bin packing logic of fenzo. - -- Ankur Chauhan On 04/06/2015 22:35, Sharma Podila wrote: > We Autoscale our Mesos cluster in EC2 from within our framework. > Scaling up can be easy via watching demand Vs supply. However, > scaling down requires bin packing the tasks tightly onto as few > servers as possible. Do you have any specific ideas on how you > would leverage Mantis/Mesos for Spark based jobs? Fenzo, the > scheduler part of Mantis, could be another point of leverage, which > could give a framework the ability to autoscale the cluster among > other benefits. > > > > On Thu, Jun 4, 2015 at 1:06 PM, Dmitry Goldenberg > <[email protected] <mailto:[email protected]>> > wrote: > > Thanks, Vinod. I'm really interested in how we could leverage > something like Mantis and Mesos to achieve autoscaling in a > Spark-based data processing system... > > On Jun 4, 2015, at 3:54 PM, Vinod Kone <[email protected] > <mailto:[email protected]>> wrote: > >> Hey Dmitry. At the current time there is no built-in support for >> Mesos to autoscale nodes in the cluster. I've heard people >> (Netflix?) do it out of band on EC2. >> >> On Thu, Jun 4, 2015 at 9:08 AM, Dmitry Goldenberg >> <[email protected] <mailto:[email protected]>> >> wrote: >> >> A Mesos noob here. Could someone point me at the doc or summary >> for the cluster autoscaling capabilities in Mesos? >> >> Is there a way to feed it events and have it detect the need to >> bring in more machines or decommission machines? Is there a way >> to receive events back that notify you that machines have been >> allocated or decommissioned? >> >> Would this work within a certain set of >> "preallocated"/pre-provisioned/"stand-by" machines or will Mesos >> go and grab machines from the cloud? >> >> What are the integration points of Apache Spark and Mesos? What >> are the true advantages of running Spark on Mesos? >> >> Can Mesos autoscale the cluster based on some signals/events >> coming out of Spark runtime or Spark consumers, then cause the >> consumers to run on the updated cluster, or signal to the >> consumers to restart themselves into an updated cluster? >> >> Thanks. >> >> > -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJVcUoqAAoJEOSJAMhvLp3LjYIIAK9pgU41hU3Dbn5tlVWxTK7y knsVOnVYiuA43DwDUTXgUUFNl67wMR0DAcueSPtUkXRfyWcgGtwDJfsF1R1vdlrN kAiSEVxOSnRb9Gg35HVjAE4Y4uYE5xZnULf6UWi65pIPUEV9nAm3i0K5chjyC/6T VE2QagNg3FurXrzeSMJkMrTuwIW+rWHkOifQMtnJb3HwqmdhidZlErXh7Sz5qiDv 0GMqjcEjpFK0ahrmDK4Nv675HitPOQN0R9V+sYhveKeRXe43CcoIUvk6yTlLN42Q oxl8HFLYxvZ4y+BlHuHO2sfVn6GJyO55sZWyk6k5BGVFT5RSCAjYME9jtCuSk3U= =RIIH -----END PGP SIGNATURE-----
