+1
> On Apr 24, 2018, at 11:41 AM, Brandon Gulla <gulla.bran...@gmail.com> wrote:
>
> Possible, maybe, the right thing to do? Probably not.
>
> There are multiple container orchestration platforms these days, and the
> pecking order is k8s -> mesos -> yarn. DC/OS has already implemented
> support for kubernetes, but purely because it is seeing k8s as a competitor
> and Mesosphere desires to make it as easy as possible to support hybrid
> cloud solutions for their customers. While they may one day add k8s
> scheduling functionality to Marathon (speculative), I highly doubt that
> they would do the same for yarn.
>
> A lot of Hadoop-ecosystems are now adding support for kubernetes directly,
> look at the latest version of spark. You go where the customers are, and
> YARN is still in third place and I would be shocked to see that change any
> time soon.
>
>
>
> On Tue, Apr 24, 2018 at 2:27 PM, Javi Roman <jroman.espi...@gmail.com>
> wrote:
>
>> Probably you are right, however thinking that we have Apache Mesos as
>> coarse-grained scheduling, and generally we run at least one
>> fine-grained "general purpose" or "meta-scheduler" such as Marathon,
>> Aurora or even Kubernetes as framework. Why not apply the same
>> rationale for Myriad, YARN as general purpose framework.
>>
>> I know this is probably crazy, but imagine a kind of DC/OS flavor with
>> Apache Mesos + Apache Myriad + Apache Hadoop running as foundation for
>> this flavor (without Marathon or kubernetes frameworks), you run
>> Mesos, Myriad and Hadoop as system services, and the user can run
>> Hadoop applications, Docker applications and whatever application.
>> When you click the "Services" tab at DC/OS interface you get the
>> Myriad UI for run applications.
>>
>> It's crazy I know, but it would be possible? I really don't know.
>>
>>
>> --
>> Javi Roman
>>
>> Twitter: @javiromanrh
>> GitHub: github.com/javiroman
>> Linkedin: es.linkedin.com/in/javiroman
>> Big Data Blog: dataintensive.info
>>
>>
>> On Tue, Apr 24, 2018 at 8:04 PM, yuliya Feldman
>> <yufeld...@yahoo.com.invalid> wrote:
>>> Are you suggesting using Myriad as meta-scheduler for Mesos?
>>> If yes, then:1. Why would one need Mesos or Myriad at all - use YARN for
>> everything?2. Mesos scheduler is much more robust then YARN scheduler as
>> it's based on the offers - it was done having multiple subsystems in mind
>> unlike YARN
>>> AFAIK Marathon serves very thin purpose in Myriad - literally just to
>> spin up Myriad Master
>>> Just my 2c.
>>> Thanks,Yuliya
>>> On Tuesday, April 24, 2018, 10:56:37 AM PDT, Javi Roman <
>> jroman.espi...@gmail.com> wrote:
>>>
>>> Hi!
>>>
>>> Thinking a little more deeply this YARN feature (Hadoop YARN Service
>>> framework) could affect Apache Myriad regarding new development goals.
>>> Apache Myriad based on YARN 3.x could be a possible substitute for
>>> general purpose meta-scheduling Mesos frameworks such as Marathon or
>>> Apache Aurora. We can rewrite the Myriad UI in order to allow run
>>> YARN native applications, Docker containers or regular applications.
>>>
>>> It's just a crazy idea: DC/OS + Apache Myriad (natively installed as
>>> systemd unit) instead of Marathon, working as meta-scheduler for
>>> Mesos.
>>>
>>> What do you think?
>>> --
>>> Javi Roman
>>>
>>> Twitter: @javiromanrh
>>> GitHub: github.com/javiroman
>>> Linkedin: es.linkedin.com/in/javiroman
>>> Big Data Blog: dataintensive.info
>>>
>>>
>>> On Mon, Apr 23, 2018 at 8:02 AM, Javi Roman <jroman.espi...@gmail.com>
>> wrote:
>>>> Hi Darin,
>>>>
>>>> You are right Slider is a kind of framework for long running
>>>> applications into YARN, and it's a way of running applications on YARN
>>>> without changing the application code, there is no need to develop a
>>>> custom Application Master or other YARN code. Apparently you can run
>>>> unmodified applications on YARN via Slider.
>>>>
>>>> This shift towards Hadoop code (instead of separate project, incubator
>>>> slider) doesn't affect Myriad at all. However I was thinking any kind
>>>> of collaboration with Slider project, and this collaboration is
>>>> unclear right now, because of fuzzy continuity of Slider.
>>>> --
>>>> Javi Roman
>>>>
>>>> Twitter: @javiromanrh
>>>> GitHub: github.com/javiroman
>>>> Linkedin: es.linkedin.com/in/javiroman
>>>> Big Data Blog: dataintensive.info
>>>>
>>>>
>>>> On Mon, Apr 23, 2018 at 1:25 AM, Darin Johnson <dbjohnson1...@gmail.com>
>> wrote:
>>>>> Javi can you explain how this effects myriad? My understanding is that
>>>>> slider is similar to marathon but for yarn.
>>>>>
>>>>> On Sat, Apr 21, 2018, 2:23 AM Javi Roman <jroman.espi...@gmail.com>
>> wrote:
>>>>>
>>>>>> FYI
>>>>>>
>>>>>> Regarding the recent conversation about YARN applications development
>>>>>> via Apache Slider, as a different approach for creating data services
>>>>>> applications on Apache Mesos (via Myriad,) "article 4" here [1], will
>>>>>> probably have to wait until the resolution of this discussion at
>>>>>> Apache Slider list [2]. Basically the Slider community is discussing
>>>>>> about the retirement of the project, because the Slider core is being
>>>>>> included in Hadoop 3 [3].
>>>>>>
>>>>>> [1] https://goo.gl/RVkFUF
>>>>>> [2] https://goo.gl/CPR2TF
>>>>>> [3] https://issues.apache.org/jira/browse/YARN-5079
>>>>>>
>>>>>> --
>>>>>> Javi Roman
>>>>>>
>>>>>> Twitter: @javiromanrh
>>>>>> GitHub: github.com/javiroman
>>>>>> Linkedin: es.linkedin.com/in/javiroman
>>>>>> Big Data Blog: dataintensive.info
>>>>>>
>>>
>>
>
>
>
> --
> Brandon
