I am not sure we should care about role being set or not, what if in the future
we will have multiple rolesNot even sure if presence/absence of role should
play role (no pun intended :) ).
From: Darin Johnson <dbjohnson1...@gmail.com>
To: Dev <dev@myriad.incubator.apache.org>; yuliya Feldman
<yufeld...@yahoo.com>
Sent: Wednesday, October 19, 2016 7:17 PM
Subject: Re: [DISCUSS] handling roles in Myriad code
Ah so if I understand correctly, if frameworkRole='*' is present in the
config, it's handled as thought it's the framework role. I believe when I
was testing I was using frameworkRole="test" or commenting out
frameworkRole="test". It looks as though in MyriadConfiguration,
getFrameworkRole now returns "*" even if not set.
Seems like we should be able to add a check like r.hasRole() &&
r.getRole().equals(role)
&& !role.equals("*") in a few places. Though it may be better
to pass think about a better approach here.
Darin
On Wed, Oct 19, 2016 at 9:28 PM, yuliya Feldman <yufeld...@yahoo.com.invalid
> wrote:
> Hello Darrin,
> I kind of see the point regarding JHS ports. May be there is truth to it.
> Regarding my issues with role/no role.
> I had this issue for NMs with random ports (not hardcoded), as it has
> different code path when role is present and when it is not. My impression
> those are bugs.
> I am happy to point you to the places in the code that caused issues on
> master (at least for me).[1] does not increment numDefaultValues if role is
> set (which is always set), subsequently [2] has issues[3] same thing -
> fills out list only if there is no role, but again it is always there, just
> set to "*"
>
>
> Regarding:>>> To handle nodemanager persistence I think we should work
> with Klaus's PR's to get thecorrect ports, though we'll need to use some
> disk persistence as well to
> keep the NM state.
> Disk persistence won't help here (not even sure NM has much state to
> persist - even if it does it should be taken care by YARN), as containers
> have to reconnect to NM after it restarts, so they have to know RPC port.
> Thanks,Yuliya
> [1] https://github.com/apache/incubator-myriad/blob/master/
> myriad-scheduler/src/main/java/org/apache/myriad/scheduler/resource/
> RangeResource.java#L85
> [2] https://github.com/apache/incubator-myriad/blob/master/
> myriad-scheduler/src/main/java/org/apache/myriad/scheduler/resource/
> RangeResource.java#L128
>
> [3] https://github.com/apache/incubator-myriad/blob/master/
> myriad-scheduler/src/main/java/org/apache/myriad/scheduler/resource/
> RangeResource.java#L140
>
>
> From: Darin Johnson <dbjohnson1...@gmail.com>
> To: Dev <dev@myriad.incubator.apache.org>; yuliya Feldman <
> yufeld...@yahoo.com>
> Sent: Wednesday, October 19, 2016 6:04 PM
> Subject: Re: [DISCUSS] handling roles in Myriad code
>
> Yuyiya,
>
> Yes on master a lot of refactoring was done, in particular you specify
> ports other than 0 in the myriad-default.yaml, it will only return those
> ports (not random ones). This was done in part because the we were
> attempting the use the JHS on a port like 32001, but it the port was
> already in use by another app and hence the port wasn't offered myriad was
> still launching the JHS only to have it crash.
>
> If you want to use static ports you can just not put anything in the
> myriad-default.yaml and configure the yarn-site.xml and mapred-site.xml as
> usual (they should be outside the range mesos offers). To handle
> nodemanager persistence I think we should work with Klaus's PR's to get the
> correct ports, though we'll need to use some disk persistance as well to
> keep the NM state.
>
> As for a bug in NM's getting zero ports could you send a copy of your
> configuration and I'll try to recreate the problem?
>
> On Wed, Oct 19, 2016 at 3:29 PM, yuliya Feldman
> <yufeld...@yahoo.com.invalid
> > wrote:
>
> > Hello there,
> > I wanted to discuss current handling of roles in Myriad code.
> Specifically
> > on "master" branch. Most likely due to heavy refactoring.
> > As far as I can see we try to handle presence or absence of a role on a
> > resource(s) based on the fact that framework may or may not have a
> role.On
> > the other hand we always set framework role to "*" - which means it will
> > always have a role, just that role will be "default".
> > So far I encountered couple of bugs related to roles in RangeResource
> > related to ports and inability to spin up NodeManagers, as no ports were
> > assigned because of the fact how we handle roles.
> > I would like @Adam and other Mesosphere folks to comment on how should we
> > handle relationship between frameworkRole and resource role(s)
> > Thanks,Yuliya
>
>
>
>
