I'd really like to see if Mohit could answer how the dcos-commons library might be able to help here. Also can dcos commons work with vanilla mesos or just dcos?
On Wed, Oct 19, 2016 at 9:51 PM, Klaus Ma <klaus1982...@gmail.com> wrote: > And for the role, we also need to handle the principal with it. For > example, it need principal to use reserved resources if necessary. > > ---- > Da (Klaus), Ma (马达) | PMP® | Software Architect > Platform OpenSource Technology, STG, IBM GCG > +86-10-8245 4084 | klaus1982...@gmail.com | http://k82.me > > On Thu, Oct 20, 2016 at 9:28 AM, yuliya Feldman > <yufeld...@yahoo.com.invalid > > wrote: > > > Hello Darrin, > > I kind of see the point regarding JHS ports. May be there is truth to it. > > Regarding my issues with role/no role. > > I had this issue for NMs with random ports (not hardcoded), as it has > > different code path when role is present and when it is not. My > impression > > those are bugs. > > I am happy to point you to the places in the code that caused issues on > > master (at least for me).[1] does not increment numDefaultValues if role > is > > set (which is always set), subsequently [2] has issues[3] same thing - > > fills out list only if there is no role, but again it is always there, > just > > set to "*" > > > > > > Regarding:>>> To handle nodemanager persistence I think we should work > > with Klaus's PR's to get thecorrect ports, though we'll need to use some > > disk persistence as well to > > keep the NM state. > > Disk persistence won't help here (not even sure NM has much state to > > persist - even if it does it should be taken care by YARN), as containers > > have to reconnect to NM after it restarts, so they have to know RPC port. > > Thanks,Yuliya > > [1] https://github.com/apache/incubator-myriad/blob/master/ > > myriad-scheduler/src/main/java/org/apache/myriad/scheduler/resource/ > > RangeResource.java#L85 > > [2] https://github.com/apache/incubator-myriad/blob/master/ > > myriad-scheduler/src/main/java/org/apache/myriad/scheduler/resource/ > > RangeResource.java#L128 > > > > [3] https://github.com/apache/incubator-myriad/blob/master/ > > myriad-scheduler/src/main/java/org/apache/myriad/scheduler/resource/ > > RangeResource.java#L140 > > > > > > From: Darin Johnson <dbjohnson1...@gmail.com> > > To: Dev <dev@myriad.incubator.apache.org>; yuliya Feldman < > > yufeld...@yahoo.com> > > Sent: Wednesday, October 19, 2016 6:04 PM > > Subject: Re: [DISCUSS] handling roles in Myriad code > > > > Yuyiya, > > > > Yes on master a lot of refactoring was done, in particular you specify > > ports other than 0 in the myriad-default.yaml, it will only return those > > ports (not random ones). This was done in part because the we were > > attempting the use the JHS on a port like 32001, but it the port was > > already in use by another app and hence the port wasn't offered myriad > was > > still launching the JHS only to have it crash. > > > > If you want to use static ports you can just not put anything in the > > myriad-default.yaml and configure the yarn-site.xml and mapred-site.xml > as > > usual (they should be outside the range mesos offers). To handle > > nodemanager persistence I think we should work with Klaus's PR's to get > the > > correct ports, though we'll need to use some disk persistance as well to > > keep the NM state. > > > > As for a bug in NM's getting zero ports could you send a copy of your > > configuration and I'll try to recreate the problem? > > > > On Wed, Oct 19, 2016 at 3:29 PM, yuliya Feldman > > <yufeld...@yahoo.com.invalid > > > wrote: > > > > > Hello there, > > > I wanted to discuss current handling of roles in Myriad code. > > Specifically > > > on "master" branch. Most likely due to heavy refactoring. > > > As far as I can see we try to handle presence or absence of a role on a > > > resource(s) based on the fact that framework may or may not have a > > role.On > > > the other hand we always set framework role to "*" - which means it > will > > > always have a role, just that role will be "default". > > > So far I encountered couple of bugs related to roles in RangeResource > > > related to ports and inability to spin up NodeManagers, as no ports > were > > > assigned because of the fact how we handle roles. > > > I would like @Adam and other Mesosphere folks to comment on how should > we > > > handle relationship between frameworkRole and resource role(s) > > > Thanks,Yuliya > > > > > > > > >
