I took a stab at labeling JIRAs that I think we need for the first milestone drop, and came up with 17 JIRAs: https://issues.apache.org/jira/browse/YARN-4239?jql=labels%20%3D%20yarn-2928-1st-milestone
Please take a look at them, and comment on them. Thanks, Sangjin On Wed, Nov 4, 2015 at 5:10 PM, Joep Rottinghuis <[email protected]> wrote: > Agreed with the theme and lists for the merge back of the first > alpha-quality release of ATS v2 to trunk. Thanks for putting forth the > suggestion with the two lists Sangjin. > > As we discussed in the weekly sync with the ATS v2 team, right after > we've merged back, the next thing to decide is what the story will be to > run both ATS v1 and ATS v2 in parallel so that we can start discussing when > we can do a merge to branch-2. > We'll want people (including ourselves) to be able to run ATS v2 on a > trial basis w/o impacting existing use-cases on a branch-2 based release. > The merge-back to branch-2 will be a separate topic of discussion though. > > Thanks, > > Joep > > > On Wed, Nov 4, 2015 at 12:18 PM, Karthik Kambatla <[email protected]> > wrote: > >> +1 >> >> In favor of merging a version working end-to-end at alpha quality to >> trunk. >> >> On Mon, Nov 2, 2015 at 10:51 AM, Sangjin Lee <[email protected]> wrote: >> >>> Just a reminder that I'd like input from folks that are working on this >>> feature (as well as anyone who's interested or course!). >>> >>> I propose creating a label to start labeling things we'd like to see >>> done before the first milestone. How about "yarn-2928-1st-milestone"? >>> >>> Thanks, >>> Sangjin >>> >>> On Wed, Oct 28, 2015 at 10:30 AM, Varun Saxena <[email protected]> >>> wrote: >>> >>>> Hi Sangjin, >>>> >>>> Thanks for initiating the discussion. >>>> I think YARN-3862 needs to go in as well. YARN-3862 and YARN-3863 are >>>> somewhat related. >>>> >>>> Regards, >>>> Varun Saxena. >>>> >>>> On Wed, Oct 28, 2015 at 5:30 AM, Sangjin Lee <[email protected]> wrote: >>>> >>>>> In the past several months, the timeline service v.2 team has made a >>>>> tremendous progress. We have a working storage implementation based on >>>>> HBase, timeline collectors, timeline readers with some of the more >>>>> important queries and filters, integration with RM and NM, distributed >>>>> shell and mapreduce, and some basic UI to boot. We're getting real close >>>>> to >>>>> a complete end-to-end flow (no pun intended). Kudos to the team (cc'ed >>>>> here)! >>>>> >>>>> I think it is time to discuss defining a merge to trunk of an >>>>> alpha-quality release as our first milestone so that a wider audience has >>>>> a >>>>> chance to try it out. This doesn't replace the timeline service (ATS) v.1 >>>>> yet, but it would be a great chance to get feedback. >>>>> >>>>> I think the theme is essentially a basic but complete end-to-end flow >>>>> that includes the write path and the read path and some UI. These are the >>>>> key major things we may want to complete before we consider merging the >>>>> first milestone: >>>>> - application aggregation (YARN-3816 >>>>> <https://issues.apache.org/jira/browse/YARN-3816>) >>>>> - flow run compaction work (YARN-4062 >>>>> <https://issues.apache.org/jira/browse/YARN-4062>) >>>>> - finalize the metrics storage (YARN-4053 >>>>> <https://issues.apache.org/jira/browse/YARN-4053>) >>>>> - improve queries and filters (YARN-3863 >>>>> <https://issues.apache.org/jira/browse/YARN-3863>) >>>>> - UI POC based on the new YARN UI framework (YARN-4097 >>>>> <https://issues.apache.org/jira/browse/YARN-4097>, YARN-4239 >>>>> <https://issues.apache.org/jira/browse/YARN-4239>?) >>>>> >>>>> In addition to these, we would like to close a few more JIRAs that >>>>> we're currently working on. Also, in terms of the app integration, we can >>>>> debate whether we stick to the distributed shell for now or spend some >>>>> more >>>>> effort to round out the mapreduce support. >>>>> >>>>> I also think the following major things are probably out of scope for >>>>> this first drop: >>>>> - time-based (offline) user/queue aggregation based on Phoenix ( >>>>> YARN-3817 <https://issues.apache.org/jira/browse/YARN-3817>) >>>>> - fault-tolerant storage (YARN-4061 >>>>> <https://issues.apache.org/jira/browse/YARN-4061>) >>>>> - timeline collector as a separate daemon (YARN-3033 >>>>> <https://issues.apache.org/jira/browse/YARN-3033>) >>>>> - timeline collector containerization >>>>> - compatibility with v.1 (YARN-3196 >>>>> <https://issues.apache.org/jira/browse/YARN-3196>, YARN-3865 >>>>> <https://issues.apache.org/jira/browse/YARN-3865>) >>>>> - support for off-cluster timeline clients (YARN-3981 >>>>> <https://issues.apache.org/jira/browse/YARN-3981>) >>>>> >>>>> We should discuss whether we agree on the theme of the first milestone >>>>> (mentioned above). Given that, then, we should discuss what makes it and >>>>> what doesn't (basically the above 2 lists). >>>>> >>>>> We also should discuss the rough time frame to complete this. This >>>>> email is to open the discussion. Your thoughts are welcome. Thanks! >>>>> >>>>> Sangjin >>>>> >>>> >>>> >>> >> >
