I was planning to run the resource manager (hopefully two via marathon once ha is done) and the history server via marathon. The three cons listed don't seem too severe. Addressing the cons:
1. In case of multiple Myriad clusters and single Marathon instance services will rely on FIFO within Marathon to get resources Not as bad as you'd think, FIFO is pretty safe for long running resources. You're not going to be hit that hard. Added benefit, the TLS, JHS, RM are running on a Marathon framework with a potentially better role. I do this already for one instance of Marathon for services I want to give high priority to. In general, I'd be inclined to leave a NM to a lower priority role. I want my M/R jobs to eventually run, I want the TLS to be running not eventually running. 2. Some env. variables, settings that Myriad already has like RM host/ports may not be easily obtainable/available from outside of Myriad framework(s). If running the RM in Marathon the relevant configs should be obtainable via a URI already. I'm caveating the on the fact I already rely on a DNS style service discovery (Mesos-DNS and Consul-Mesos both work fine for this). 3. When tearing down Myriad cluster those services will have to be stopped separately. Possibly a bug, I might want to inspect the TLS and JHS after shutting down YARN, or I may just want to update the RM (Myriad is actively being developed). Might be worth reinvestigating at a later date. One pro I could see is by setting the JHS and TLS in Myriad, it may be easier to assign them dynamic ports via Mesos, but this would be a lot of added complexity for updating the configs of running NMs if a JHS or TLS was reassigned. I don't think this is something to address right now as it adds complexity which I think can be avoided. Darin Hello here, As we discussed last meeting I have wrote a small doc on pros/cons and design suggestions for running JHS as MyriadTask. https://docs.google.com/document/d/1TGGU2bn1cVPSmHeriNrtkAEhdvsRKcFtuWRaaGp9bSY/edit?usp=sharing Also commented on: https://github.com/mesos/myriad/issues/60 Comments, suggestions, objections are welcome. I am trying to wrap up design this week, so I could go one way or the other starting next week, as I will be gone for 2 weeks, starting Aug 1st. Thanks,Yuliya
