>>> how does the AM determine the map of nodes. IIUC, it is not possible now. Probably a bug which was missed till now!! OR Does all the applications need to know all the nodes updates in a cluster? It could be 1000 nodes , 10K nodes.
And how about when node label is configured.? Thanks & Regards Rohith Sharma K S -----Original Message----- From: Steve Loughran [mailto:[email protected]] Sent: 19 October 2015 15:01 To: [email protected] Subject: Re: When does the RM ever send out node map updates? Thanks for that snippet. Which brings me back to a key question: how does the AM determine the map of nodes. And before someone says "the yarn client lets you do that", bear in mind this is for a long-lived restartable AM; what may have been valid when the client launched it 3 weeks earlier may be invalid on restart. If it were in the RegisterApplicationMasterResponse I'd pick it up there & wait for changes —but you don't get a nodemap then Any suggestions? > On 19 Oct 2015, at 06:25, Rohith Sharma K S <[email protected]> wrote: > > NM node updates are sent to ApplicationMaster only when NodeManager is > added/removed/'change in NM state i.e Usable/Unusable'. Otherwise no updates > are sent to AM even at least one time in application life time!! > > I believe it is implemented purposefully to intimate AM about only unusable > nodes. > > Thanks & Regards > Rohith Sharma K S > > -----Original Message----- > From: Steve Loughran [mailto:[email protected]] > Sent: 18 October 2015 23:48 > To: [email protected] > Subject: When does the RM ever send out node map updates? > > > I've been modifying the slider AM to wait for NM updates of node maps -but > I'm not seeing any onNodesUpdated() callbacks, at least for an AM which > doesn't instantiate any containers. > > When/how do those updates come? >
