Hello,
Please see inline.
Thanks,Yuliya
On Thursday, September 26, 2019, 12:54:47 AM PDT, Oscar Fernandez
wrote:
Hi,
*1. *Myriad Scheduler would be responsible to register with Mesos and, on
demand, bring up Yarn clusters (RMs and NMs) and manage its resources.
>>> OK
*2. *Yes, the
Hi,
*1. *Myriad Scheduler would be responsible to register with Mesos and, on
demand, bring up Yarn clusters (RMs and NMs) and manage its resources.
*2. *Yes, the idea is that Myriad will control NMs for all YARN clusters
that the user wants to deploy. Obviously the web UI should be updated and
Hello,
Thank you for the diagrams - it helps. Could you also enable comments in your
doc?
Few thoughts:1. Myriad Scheduler is wonderful - but it's yet another scheduler
you need to deal with - or you plan to have current MyriadScheduler that sits
in RM and use it instead?2. Is Myriad
Hi,
I've made a diagram to represent the new proposed design in order to
support Yarn as a service with some of the pros:
https://docs.google.com/document/d/15X0-zSu0G0BDpWyndRhbvAJCXLtAbkNA45wQ_xVKOKQ
Thank you for all your comments and help
On Tue, Sep 24, 2019 at 8:57 PM Javi Roman wrote:
Honestly your opinion is welcome, this kind of discussions are great
in this small traffic dev list ;-)
--
Javi Roman
Twitter: @javiromanrh
GitHub: github.com/javiroman
Linkedin: es.linkedin.com/in/javiroman
Big Data Blog: dataintensive.info
Apache Id: javiroman
On Tue, Sep 24, 2019 at 8:55 PM
I am not saying it's crazy. I was voicing my opinion. Isn't it what was the
purpose of the discussion?
It's definitely great to have UI that manages all the YARN clusters, but it's
not like UI/Web service has to be coupled/collocated with any of the Myriad
particular YARN version daemons.
It's
It's true with one "but" you need a yet another scheduler in your service and
it's not a trivial feat - if you want it to be fulfilling the purpose,
otherwise Marathon as good as it is. As far as I remember Marathon has simple
FIFO one (may be it evolved though).
In any case - it was my
On Tue, Sep 24, 2019 at 7:58 PM yuliya Feldman
wrote:
>
> Hello,
> Again I apologize for the late reply.
> I think I replied to the thread, but will add more direct notes here
> What you are proposing is to have yet another daemon that would start Yarn
> Clusters on demand within Mesos
On Tue, Sep 24, 2019 at 7:48 PM yuliya Feldman
wrote:
>
> >>> I guess we are talking about the concept of multi-service scheduler
Are you proposing to have a multi service scheduler? I thought that's
what Mesos is for?What am I missing here?
Yes is common in Mesos (IMHO): Marathon is a
Hello,
Again I apologize for the late reply.
I think I replied to the thread, but will add more direct notes here
What you are proposing is to have yet another daemon that would start Yarn
Clusters on demand within Mesos framework.
Meaning - it would be another layer of abstraction. In this
>>> I guess we are talking about the concept of multi-service schedulerAre you
>>>proposing to have a multi service scheduler? I thought that's what Mesos is
>>>for?What am I missing here?
Thanks,Yuliya
On Tuesday, September 24, 2019, 10:00:39 AM PDT, Javi Roman
wrote:
I guess we
I guess we are talking about the concept of multi-service scheduler, or one
framework for multiple services, in our case multiple YARN services
(different versions, and so on).
Javi Roman
Twitter: @javiromanrh
GitHub: github.com/javiroman
Linkedin: es.linkedin.com/in/javiroman
Big Data Blog:
Hello there,
Sorry for late reply.
Frankly speaking I don’t know original motivation , I would probably say that
it started organically as a need.
What do you mean by starting yarn from myriad? I believe myriad daemons
encompass some functionality of yarn daemons, namely rm and nm, so it’s
Hi Oscar,
I have to say I don't know the initial motivation of this design. You
are right the way of starting Myriad, strongly coupled to YARN is a
little bit weird.
Because of lack of activity of the initial committers, this is a
question that probably we never get a clear answer.
By the way,
14 matches
Mail list logo
