Ah so if I understand correctly, if frameworkRole='*' is present in the config, it's handled as thought it's the framework role. I believe when I was testing I was using frameworkRole="test" or commenting out frameworkRole="test". It looks as though in MyriadConfiguration, getFrameworkRole now returns "*" even if not set.
Seems like we should be able to add a check like r.hasRole() && r.getRole().equals(role) && !role.equals("*") in a few places. Though it may be better to pass think about a better approach here. Darin On Wed, Oct 19, 2016 at 9:28 PM, yuliya Feldman <yufeld...@yahoo.com.invalid > wrote: > Hello Darrin, > I kind of see the point regarding JHS ports. May be there is truth to it. > Regarding my issues with role/no role. > I had this issue for NMs with random ports (not hardcoded), as it has > different code path when role is present and when it is not. My impression > those are bugs. > I am happy to point you to the places in the code that caused issues on > master (at least for me).[1] does not increment numDefaultValues if role is > set (which is always set), subsequently [2] has issues[3] same thing - > fills out list only if there is no role, but again it is always there, just > set to "*" > > > Regarding:>>> To handle nodemanager persistence I think we should work > with Klaus's PR's to get thecorrect ports, though we'll need to use some > disk persistence as well to > keep the NM state. > Disk persistence won't help here (not even sure NM has much state to > persist - even if it does it should be taken care by YARN), as containers > have to reconnect to NM after it restarts, so they have to know RPC port. > Thanks,Yuliya > [1] https://github.com/apache/incubator-myriad/blob/master/ > myriad-scheduler/src/main/java/org/apache/myriad/scheduler/resource/ > RangeResource.java#L85 > [2] https://github.com/apache/incubator-myriad/blob/master/ > myriad-scheduler/src/main/java/org/apache/myriad/scheduler/resource/ > RangeResource.java#L128 > > [3] https://github.com/apache/incubator-myriad/blob/master/ > myriad-scheduler/src/main/java/org/apache/myriad/scheduler/resource/ > RangeResource.java#L140 > > > From: Darin Johnson <dbjohnson1...@gmail.com> > To: Dev <dev@myriad.incubator.apache.org>; yuliya Feldman < > yufeld...@yahoo.com> > Sent: Wednesday, October 19, 2016 6:04 PM > Subject: Re: [DISCUSS] handling roles in Myriad code > > Yuyiya, > > Yes on master a lot of refactoring was done, in particular you specify > ports other than 0 in the myriad-default.yaml, it will only return those > ports (not random ones). This was done in part because the we were > attempting the use the JHS on a port like 32001, but it the port was > already in use by another app and hence the port wasn't offered myriad was > still launching the JHS only to have it crash. > > If you want to use static ports you can just not put anything in the > myriad-default.yaml and configure the yarn-site.xml and mapred-site.xml as > usual (they should be outside the range mesos offers). To handle > nodemanager persistence I think we should work with Klaus's PR's to get the > correct ports, though we'll need to use some disk persistance as well to > keep the NM state. > > As for a bug in NM's getting zero ports could you send a copy of your > configuration and I'll try to recreate the problem? > > On Wed, Oct 19, 2016 at 3:29 PM, yuliya Feldman > <yufeld...@yahoo.com.invalid > > wrote: > > > Hello there, > > I wanted to discuss current handling of roles in Myriad code. > Specifically > > on "master" branch. Most likely due to heavy refactoring. > > As far as I can see we try to handle presence or absence of a role on a > > resource(s) based on the fact that framework may or may not have a > role.On > > the other hand we always set framework role to "*" - which means it will > > always have a role, just that role will be "default". > > So far I encountered couple of bugs related to roles in RangeResource > > related to ports and inability to spin up NodeManagers, as no ports were > > assigned because of the fact how we handle roles. > > I would like @Adam and other Mesosphere folks to comment on how should we > > handle relationship between frameworkRole and resource role(s) > > Thanks,Yuliya > > > >