Hi Jim, Nothing major, but looking at the current FGS experience, I have something to share.
Currently, if the user launches a NM with "zero" profile, that NM is eligible for FGS. This means, the YARN containers that will be allocated to that NM will be based upon the Mesos offers received for that node. The fact that "zero" profile corresponds to FGS might be confusing. a) we need a better/clearer way to express this fact. Perhaps a different name for the "zero" profile? b) UI can perhaps show a "help" message explaining that a "zero" profile NM pertains to FGS. Thanks, Santosh On Thu, Aug 6, 2015 at 5:25 PM, Swapnil Daingade <[email protected] > wrote: > Hi Jim, > > I cannot think of any UI work for HA. > I'll let Santosh comment for FGS. > > Regards > Swapnil > > > On Thu, Aug 6, 2015 at 12:38 PM, Jim Klucar <[email protected]> wrote: > > > any UI or Rest tweaks desired for this? > > > > On Thu, Aug 6, 2015 at 3:01 PM, Swapnil Daingade < > > [email protected] > > > wrote: > > > > > Just learned from Santosh that my email last night did not make it to > the > > > mailing list. > > > resending... > > > > > > Regards > > > Swapnil > > > > > > > > > ---------- Forwarded message ---------- > > > From: Swapnil Daingade <[email protected]> > > > Date: Thu, Aug 6, 2015 at 12:00 AM > > > Subject: Re: Myriad 0.1 release scope > > > To: [email protected] > > > > > > > > > > > > I think Myriad HA is also in good shape. We discussed work preserving > NM > > > restart internally and it looks like it is disabled by default in > Apache > > > Hadoop. > > > > > > > > > https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerRestart.html > > > > > > All 3 major distro's seem to have it disabled by default. I think we > can > > > revisit it in the next Myriad release. > > > For the time being we can continue with the NM + Executor merge. It too > > has > > > been tried out and reviewed by Darin and Santosh. > > > > > > The current PR we have for Myriad state store implementation should > work > > > for phase1. > > > There is some additional work required for supporting FGS that I am > > working > > > on. > > > Should be able to complete by the time FGS is merged into phase1. > > > > > > Regards > > > Swapnil > > > > > > > > > > > > Santosh Marella wrote: > > > > > > I feel FGS is very close to making it into 0.1. PR 116 addresses moving > > to > > > hadoop 2.7 and making FGS and CGS coexist. This PR was recently > reviewed > > by > > > Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x > > > clusters and it seemed to have worked as expected. Unless there are > more > > > reviews/feedback, it can be merged into issue_14. Once PR 116 is merged > > > into issue_14, issue_14 can be merged into phase1. > > > > > > Thanks, > > > Santosh > > > > > > On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon <[email protected]> > > > <[email protected]> wrote: > > > > > > > > > > > > We do have a JIRA 0.1.0 "fix version" field, but none of our issues use > > it > > > yet. > > > I think the goal was just to take what we have and make it work under > > > Apache infrastructure, then vote on that for 0.1.0. > > > Although other features like HA or FGS would be great, let's try to get > > our > > > first Apache release out ASAP. > > > We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with > > > other issues/features. Roadmap would be great. > > > (I'm just summarizing what we discussed a month or two ago. Feel free > to > > > correct me or disagree with this approach.) > > > > > > On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade < > > > [email protected] > > > > > > wrote: > > > > > > > > > Hi all, > > > > > > Was wondering what would be the scope for the Myriad 0.1 release. > > > It would be nice to have a roadmap page somewhere and target > > > features to releases (JIRA 'fix version' field perhaps) > > > > > > Regards > > > Swapnil > > > > > >
