Let's add the label and discuss the technical aspects on the JIRA. I just added comments there too.
On Mon, Nov 23, 2015 at 9:25 PM, Varun Saxena <[email protected]> wrote: > I think this JIRA should be complete by the time we merge it into trunk. > > It would be nice to have the CLI work with ATSv2 if application is not > found in RM. > > Regards, > Varun Saxena. > > On Tue, 24 Nov 2015 at 10:48, Naganarasimha G R (Naga) < > [email protected]> wrote: > >> Hi Sangjin, >> >> Seems like the list is fine, can we have YARN-4343 too in the list ? >> >> + Naga >> ________________________________________ >> From: Varun Saxena [[email protected]] >> Sent: Saturday, November 14, 2015 01:46 >> To: Sangjin Lee >> Cc: Joep Rottinghuis; Karthik Kambatla; [email protected]; Li >> Lu; Junping Du; [email protected]; Vinod Kumar Vavilapalli; >> Naganarasimha Garla; Robert Kanter; Vrushali Channapattan >> Subject: Re: [YARN-2928] first drop on trunk >> >> The list >> >> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20yarn-2928-1st-milestone >> looks good to me. >> >> Regards, >> Varun Saxena. >> >> >> On Sat, Nov 14, 2015 at 12:09 AM, Sangjin Lee <[email protected]> wrote: >> >> > I took a stab at labeling JIRAs that I think we need for the first >> > milestone drop, and came up with 17 JIRAs: >> > >> https://issues.apache.org/jira/browse/YARN-4239?jql=labels%20%3D%20yarn-2928-1st-milestone >> > >> > Please take a look at them, and comment on them. >> > >> > Thanks, >> > Sangjin >> > >> > On Wed, Nov 4, 2015 at 5:10 PM, Joep Rottinghuis <[email protected]> >> wrote: >> > >> >> Agreed with the theme and lists for the merge back of the first >> >> alpha-quality release of ATS v2 to trunk. Thanks for putting forth the >> >> suggestion with the two lists Sangjin. >> >> >> >> As we discussed in the weekly sync with the ATS v2 team, right after >> >> we've merged back, the next thing to decide is what the story will be >> to >> >> run both ATS v1 and ATS v2 in parallel so that we can start discussing >> when >> >> we can do a merge to branch-2. >> >> We'll want people (including ourselves) to be able to run ATS v2 on a >> >> trial basis w/o impacting existing use-cases on a branch-2 based >> release. >> >> The merge-back to branch-2 will be a separate topic of discussion >> though. >> >> >> >> Thanks, >> >> >> >> Joep >> >> >> >> >> >> On Wed, Nov 4, 2015 at 12:18 PM, Karthik Kambatla <[email protected]> >> >> wrote: >> >> >> >>> +1 >> >>> >> >>> In favor of merging a version working end-to-end at alpha quality to >> >>> trunk. >> >>> >> >>> On Mon, Nov 2, 2015 at 10:51 AM, Sangjin Lee <[email protected]> >> wrote: >> >>> >> >>>> Just a reminder that I'd like input from folks that are working on >> this >> >>>> feature (as well as anyone who's interested or course!). >> >>>> >> >>>> I propose creating a label to start labeling things we'd like to see >> >>>> done before the first milestone. How about "yarn-2928-1st-milestone"? >> >>>> >> >>>> Thanks, >> >>>> Sangjin >> >>>> >> >>>> On Wed, Oct 28, 2015 at 10:30 AM, Varun Saxena < >> [email protected] >> >>>> > wrote: >> >>>> >> >>>>> Hi Sangjin, >> >>>>> >> >>>>> Thanks for initiating the discussion. >> >>>>> I think YARN-3862 needs to go in as well. YARN-3862 and YARN-3863 >> are >> >>>>> somewhat related. >> >>>>> >> >>>>> Regards, >> >>>>> Varun Saxena. >> >>>>> >> >>>>> On Wed, Oct 28, 2015 at 5:30 AM, Sangjin Lee <[email protected]> >> wrote: >> >>>>> >> >>>>>> In the past several months, the timeline service v.2 team has made >> a >> >>>>>> tremendous progress. We have a working storage implementation >> based on >> >>>>>> HBase, timeline collectors, timeline readers with some of the more >> >>>>>> important queries and filters, integration with RM and NM, >> distributed >> >>>>>> shell and mapreduce, and some basic UI to boot. We're getting real >> close to >> >>>>>> a complete end-to-end flow (no pun intended). Kudos to the team >> (cc'ed >> >>>>>> here)! >> >>>>>> >> >>>>>> I think it is time to discuss defining a merge to trunk of an >> >>>>>> alpha-quality release as our first milestone so that a wider >> audience has a >> >>>>>> chance to try it out. This doesn't replace the timeline service >> (ATS) v.1 >> >>>>>> yet, but it would be a great chance to get feedback. >> >>>>>> >> >>>>>> I think the theme is essentially a basic but complete end-to-end >> flow >> >>>>>> that includes the write path and the read path and some UI. These >> are the >> >>>>>> key major things we may want to complete before we consider >> merging the >> >>>>>> first milestone: >> >>>>>> - application aggregation (YARN-3816 >> >>>>>> <https://issues.apache.org/jira/browse/YARN-3816>) >> >>>>>> - flow run compaction work (YARN-4062 >> >>>>>> <https://issues.apache.org/jira/browse/YARN-4062>) >> >>>>>> - finalize the metrics storage (YARN-4053 >> >>>>>> <https://issues.apache.org/jira/browse/YARN-4053>) >> >>>>>> - improve queries and filters (YARN-3863 >> >>>>>> <https://issues.apache.org/jira/browse/YARN-3863>) >> >>>>>> - UI POC based on the new YARN UI framework (YARN-4097 >> >>>>>> <https://issues.apache.org/jira/browse/YARN-4097>, YARN-4239 >> >>>>>> <https://issues.apache.org/jira/browse/YARN-4239>?) >> >>>>>> >> >>>>>> In addition to these, we would like to close a few more JIRAs that >> >>>>>> we're currently working on. Also, in terms of the app integration, >> we can >> >>>>>> debate whether we stick to the distributed shell for now or spend >> some more >> >>>>>> effort to round out the mapreduce support. >> >>>>>> >> >>>>>> I also think the following major things are probably out of scope >> for >> >>>>>> this first drop: >> >>>>>> - time-based (offline) user/queue aggregation based on Phoenix ( >> >>>>>> YARN-3817 <https://issues.apache.org/jira/browse/YARN-3817>) >> >>>>>> - fault-tolerant storage (YARN-4061 >> >>>>>> <https://issues.apache.org/jira/browse/YARN-4061>) >> >>>>>> - timeline collector as a separate daemon (YARN-3033 >> >>>>>> <https://issues.apache.org/jira/browse/YARN-3033>) >> >>>>>> - timeline collector containerization >> >>>>>> - compatibility with v.1 (YARN-3196 >> >>>>>> <https://issues.apache.org/jira/browse/YARN-3196>, YARN-3865 >> >>>>>> <https://issues.apache.org/jira/browse/YARN-3865>) >> >>>>>> - support for off-cluster timeline clients (YARN-3981 >> >>>>>> <https://issues.apache.org/jira/browse/YARN-3981>) >> >>>>>> >> >>>>>> We should discuss whether we agree on the theme of the first >> >>>>>> milestone (mentioned above). Given that, then, we should discuss >> what makes >> >>>>>> it and what doesn't (basically the above 2 lists). >> >>>>>> >> >>>>>> We also should discuss the rough time frame to complete this. This >> >>>>>> email is to open the discussion. Your thoughts are welcome. Thanks! >> >>>>>> >> >>>>>> Sangjin >> >>>>>> >> >>>>> >> >>>>> >> >>>> >> >>> >> >> >> > >> >
