I guess we are talking about the concept of multi-service scheduler, or one framework for multiple services, in our case multiple YARN services (different versions, and so on).
Javi Roman Twitter: @javiromanrh GitHub: github.com/javiroman Linkedin: es.linkedin.com/in/javiroman Big Data Blog: dataintensive.info Apache Id: javiroman On Tue, Sep 24, 2019 at 5:40 PM Yuliya <yufeld...@yahoo.com.invalid> wrote: > > Hello there, > > Sorry for late reply. > > Frankly speaking I don’t know original motivation , I would probably say that > it started organically as a need. > > What do you mean by starting yarn from myriad? I believe myriad daemons > encompass some functionality of yarn daemons, namely rm and nm, so it’s not > yarn that starts myriad, but myriad daemons that play role of yarn daemons. > Unless I am missing something here. > > Are you proposing to have the same myriad daemons starting different versions > of yarn? > I would consider different set of docker containers built with different > versions of yarn would be better decoupling. I am open for discussion though. > > Thanks, > Yuliya > > > > > On Sep 15, 2019, at 10:35 PM, Javi Roman <jroman.espi...@gmail.com> wrote: > > > > Hi Oscar, > > > > I have to say I don't know the initial motivation of this design. You > > are right the way of starting Myriad, strongly coupled to YARN is a > > little bit weird. > > Because of lack of activity of the initial committers, this is a > > question that probably we never get a clear answer. > > > > By the way, your proposal, according with MYRIAD-295 is, from my > > understanding, the right way to go ahead with the project. > > > > This new design is totally aligned with the further Myriad UI design > > (https://issues.apache.org/jira/browse/MYRIAD-279). > > > > The document design of this new UI here: > > https://docs.google.com/document/d/16gA67RXoPK24OIxDMNNhuYS8ioScI1eOBR-XMMPjWQE/edit?usp=sharing > > -- > > Javi Roman > > > > Twitter: @javiromanrh > > GitHub: github.com/javiroman > > Linkedin: es.linkedin.com/in/javiroman > > Big Data Blog: dataintensive.info > > Apache Id: javiroman > > > >> On Thu, Sep 12, 2019 at 8:55 AM Oscar Fernandez <oscarf...@apache.org> > >> wrote: > >> > >> Hi, > >> > >> I've started working on https://issues.apache.org/jira/browse/MYRIAD-295 - > >> Multiple versions of Apache Hadoop YARN as a Service. > >> > >> In order to implement this, we should avoid starting the Myriad framework > >> from Yarn and instead starting Yarn(s) from Myriad on demand. > >> > >> I wanted to ask the Myriad community if this design was intended for a > >> reason or if you think it's a good idea to decouple the execution of Myriad > >> from the Yarn RM. With the new design, the Myriad Framework would register > >> on Mesos, and then, start on demand the RM and NM that the user wants, > >> allowing several Yarn clusters to run in he same Mesos, even with different > >> versions. > >> > >> Thank you >
