Great! Thanks for the heads up. On Tue, Jun 9, 2015 at 12:16 PM, yuliya Feldman <[email protected] > wrote:
> Paul, > Swapnil is also closely looking into Myriad HA considering FGS, so it > would be a good idea for two of you to sync up as he also did some > investigation of what needs to be in StateStore and what does not need to > be stored. > He may not reply right away as he is in Hadoop Summit today. > From: Paul Read <[email protected]> > To: [email protected] > Sent: Tuesday, June 9, 2015 8:55 AM > Subject: StateStore and FineGrainedScaling (FGS) > > I'm working on the StateStore which at first blush does not seem that > difficult. However with the approaching FGS I suspect I need to store > additional information. > > I assume HA would need to reconstruct the FGS state. > > Assuming this is correct the following would need to happen: > > 1. Re-init the YarnNodeCapacityManager with new OfferLifeCycleManager and > NodeStore > a. rebuild OfferLifecycleManager > b. store NodeStore > c. store offerFeedMap > d. store consumerOfferMap > > 2. Possibly store the Pseudo Tasks and reconstruct the > PseudoTaskResourceManager > > 3. Store and rebuild the SchedulerState > > I was originally thinking that the Offers under consideration would need to > be stored but those may be encapsulated in the tasking...correct? > > Thanks > > > >
