Re: [VOTE] Propose to start new Hadoop sub project "submarine"

[varunsax...@apache.org]

+1


On Sat, 2 Feb 2019, 03:54 Wangda Tan,  wrote:

> Hi all,
>
> According to positive feedbacks from the thread [1]
>
> This is vote thread to start a new subproject named "hadoop-submarine"
> which follows the release process already established for ozone.
>
> The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.
>
> Thanks,
> Wangda Tan
>
> [1]
>
> https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E
>

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

[varunsax...@apache.org]

+1

On Fri, 6 Jul 2018 at 3:07 AM, Subru Krishnan  wrote:

> Folks,
>
> There was a merge commit accidentally pushed to trunk, you can find the
> details in the mail thread [1].
>
> I have raised an INFRA ticket [2] to reset/force push to clean up trunk.
>
> Can we have a quick vote for INFRA sign-off to proceed as this is blocking
> all commits?
>
> Thanks,
> Subru
>
> [1]
>
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201807.mbox/%3CCAHqguubKBqwfUMwhtJuSD7X1Bgfro_P6FV%2BhhFhMMYRaxFsF9Q%40mail.gmail.com%3E
> [2] https://issues.apache.org/jira/browse/INFRA-16727
>

Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk

[varunsax...@apache.org]

Hi All,

Please note that ATSv2 has been merged to branch-2.
Refer to https://github.com/apache/hadoop/commit/
54a694172d7162b2ff45d6e1780c0523cfda66e7

We will continue monitoring if everything works well post-merge.

Regards,
Varun Saxena.

On Mon, Oct 16, 2017 at 9:54 PM, Vrushali C <vrushalic2...@gmail.com> wrote:

> Timeline Service v2 should be landing on branch2 shortly.
>
> thanks
> Vrushali
>
> On Thu, Sep 7, 2017 at 4:57 PM, Vrushali C <vrushalic2...@gmail.com>
> wrote:
>
>> Thanks everyone.
>>
>> It has been over a week (~9 days) since TSv2 has been merged to trunk
>> with no problems thus far. We are now thinking about merging timeline
>> service v2 to branch2 some time in the next few weeks.
>>
>> So far, we have been maintaining a branch2 based YARN-5355_branch2 along
>> with our trunk based feature branch YARN-5355. Varun Saxena has been
>> diligently rebasing it to stay current with branch2.
>>
>> Currently we are in the process of testing it just like we did our due
>> diligence with the trunk based YARN-5355 branch and will ensure the TSv2
>> branch2 code is a stable state to be merged.
>>
>> We will send out another email when we are ready to merge to branch2.
>> thanks
>> Vrushali
>>
>> On Thu, Aug 31, 2017 at 12:33 PM, Subramaniam V K <subru...@gmail.com>
>> wrote:
>>
>>> Good to see this merged. I have initiated a separate thread with a
>>> smaller set of stakeholders to discuss inclusion in 2.9. We'll report back
>>> to the 2.9 release thread as soon as we reach consensus.
>>>
>>> On Thu, Aug 31, 2017 at 10:39 AM, Ravi Prakash <ravihad...@gmail.com>
>>> wrote:
>>>
>>>> +1 to maintaining history.
>>>>
>>>> On Wed, Aug 30, 2017 at 11:38 PM, varunsax...@apache.org <
>>>> varun.saxena.apa...@gmail.com> wrote:
>>>>
>>>> > Yes, I had used "git merge --no-ff"  while merging ATSv2 to trunk.
>>>> > Maintaining history I believe can be useful as it can make reverts
>>>> > easier if at all required.
>>>> > And can be an easy reference point to look at who had contributed what
>>>> > without having to go back to the branch.
>>>> >
>>>> > Regards,
>>>> > Varun Saxena.
>>>> >
>>>> > On Thu, Aug 31, 2017 at 3:56 AM, Vrushali C <vrushalic2...@gmail.com>
>>>> > wrote:
>>>> >
>>>> > > Thanks Sangjin for the link to the previous discussions on this! I
>>>> think
>>>> > > that helps answer Steve's questions.
>>>> > >
>>>> > > As decided on that thread [1], YARN-5355 as a feature branch was
>>>> merged
>>>> > to
>>>> > > trunk via "git merge --no-ff" .
>>>> > >
>>>> > > Although trunk already had TSv2 code (alpha1) prior to this merge,
>>>> we
>>>> > > chose to develop on a feature branch YARN-5355 so that we could
>>>> control
>>>> > > when changes went into trunk and didn't inadvertently disrupt trunk.
>>>> > >
>>>> > > Is the latest merge causing any conflicts or issues for s3guard,
>>>> Steve?
>>>> > >
>>>> > > thanks
>>>> > > Vrushali
>>>> > > [1] https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac2b3c7
>>>> 6afd9ef
>>>> > > f1f78b177fabe9c4a96d9b3d0b@1440189889@%3Ccommon-dev.hadoop.a
>>>> pache.org%3E
>>>> > >
>>>> > >
>>>> > > On Wed, Aug 30, 2017 at 2:37 PM, Sangjin Lee <sj...@apache.org>
>>>> wrote:
>>>> > >
>>>> > >> I recall this discussion about a couple of years ago:
>>>> > >> https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac
>>>> > >> 2b3c76afd9eff1f78b177fabe9c4a96d9b3d0b@1440189889@%3Ccommon-
>>>> > >> dev.hadoop.apache.org%3E
>>>> > >>
>>>> > >> On Wed, Aug 30, 2017 at 2:32 PM, Steve Loughran <
>>>> ste...@hortonworks.com
>>>> > >
>>>> > >> wrote:
>>>> > >>
>>>> > >>> I'd have assumed it would have gone in as one single patch,
>>>> rather than
>>>> > >>> a full history. I don't see why the trunk needs all the
>>>&g

Re: 2.9.0 status update (10/6/2017)

[varunsax...@apache.org]

Hi All,

Please note that ATSv2 has been merged to branch-2.
Refer to
https://github.com/apache/hadoop/commit/54a694172d7162b2ff45d6e1780c0523cfda66e7

We will continue monitoring if everything works well post-merge.

Regards,
Varun Saxena.

On Sat, Oct 7, 2017 at 5:30 AM, Subramaniam V K  wrote:

> We have pushed out the release to 10th November 2017:
> https://cwiki.apache.org/confluence/display/HADOOP/Roadmap
>
> The *final *feature freeze date is now 20th October 2017:
>
>- Merge vote is ongoing for API based (Capacity) Scheduler configuration
>(YARN-5734) and HDFS Router based federation (HDFS-10467
>). Both of them seem
>on track to pass.
>- Backport of ATS v2 (YARN-2928) and YARN Web UI v2 (YARN-3368) are in
>progress and should be done given the extension.
>
>
> There are 21 outstanding blockers to be addressed by 27th October 2017:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.9+Release
>
> Andrew is also tracking most of these as they are common for 3.0.0 also.
> We'll follow up additionally on those specifically targeted for 2.9.0.
>
> -Subru/Arun
>

Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk

[varunsax...@apache.org]

Yes, I had used "git merge --no-ff"  while merging ATSv2 to trunk.
Maintaining history I believe can be useful as it can make reverts
easier if at all required.
And can be an easy reference point to look at who had contributed what
without having to go back to the branch.

Regards,
Varun Saxena.

On Thu, Aug 31, 2017 at 3:56 AM, Vrushali C <vrushalic2...@gmail.com> wrote:

> Thanks Sangjin for the link to the previous discussions on this! I think
> that helps answer Steve's questions.
>
> As decided on that thread [1], YARN-5355 as a feature branch was merged to
> trunk via "git merge --no-ff" .
>
> Although trunk already had TSv2 code (alpha1) prior to this merge, we
> chose to develop on a feature branch YARN-5355 so that we could control
> when changes went into trunk and didn't inadvertently disrupt trunk.
>
> Is the latest merge causing any conflicts or issues for s3guard, Steve?
>
> thanks
> Vrushali
> [1] https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac2b3c76afd9ef
> f1f78b177fabe9c4a96d9b3d0b@1440189889@%3Ccommon-dev.hadoop.apache.org%3E
>
>
> On Wed, Aug 30, 2017 at 2:37 PM, Sangjin Lee <sj...@apache.org> wrote:
>
>> I recall this discussion about a couple of years ago:
>> https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac
>> 2b3c76afd9eff1f78b177fabe9c4a96d9b3d0b@1440189889@%3Ccommon-
>> dev.hadoop.apache.org%3E
>>
>> On Wed, Aug 30, 2017 at 2:32 PM, Steve Loughran <ste...@hortonworks.com>
>> wrote:
>>
>>> I'd have assumed it would have gone in as one single patch, rather than
>>> a full history. I don't see why the trunk needs all the evolutionary
>>> history of a build.
>>>
>>> What should our policy/process be here?
>>>
>>> I do currently plan to merge the s3guard in as one single squashed
>>> patch; just getting HADOOP-14809 sorted first.
>>>
>>>
>>> > On 30 Aug 2017, at 07:09, Vrushali C <vrushalic2...@gmail.com> wrote:
>>> >
>>> > I'm adding my +1 (binding) to conclude the vote.
>>> >
>>> > With 13 +1's (11 binding) and no -1's, the vote passes. We'll get on
>>> with
>>> > the merge to trunk shortly. Thanks everyone!
>>> >
>>> > Regards
>>> > Vrushali
>>> >
>>> >
>>> > On Tue, Aug 29, 2017 at 10:54 AM, varunsax...@apache.org <
>>> > varun.saxena.apa...@gmail.com> wrote:
>>> >
>>> >> +1 (binding).
>>> >>
>>> >> Kudos to all the team members for their great work!
>>> >>
>>> >> Being part of the ATSv2 team, I have been involved with either
>>> development
>>> >> or review of most of the JIRAs'.
>>> >> Tested ATSv2 in both secure and non-secure mode. Also verified that
>>> there
>>> >> is no impact when ATSv2 is turned off.
>>> >>
>>> >> Regards,
>>> >> Varun Saxena.
>>> >>
>>> >> On Tue, Aug 22, 2017 at 12:02 PM, Vrushali Channapattan <
>>> >> vrushalic2...@gmail.com> wrote:
>>> >>
>>> >>> Hi folks,
>>> >>>
>>> >>> Per earlier discussion [1], I'd like to start a formal vote to merge
>>> >>> feature branch YARN-5355 [2] (Timeline Service v.2) to trunk. The
>>> vote
>>> >>> will
>>> >>> run for 7 days, and will end August 29 11:00 PM PDT.
>>> >>>
>>> >>> We have previously completed one merge onto trunk [3] and Timeline
>>> Service
>>> >>> v2 has been part of Hadoop release 3.0.0-alpha1.
>>> >>>
>>> >>> Since then, we have been working on extending the capabilities of
>>> Timeline
>>> >>> Service v2 in a feature branch [2] for a while, and we are reasonably
>>> >>> confident that the state of the feature meets the criteria to be
>>> merged
>>> >>> onto trunk and we'd love folks to get their hands on it in a test
>>> capacity
>>> >>> and provide valuable feedback so that we can make it
>>> production-ready.
>>> >>>
>>> >>> In a nutshell, Timeline Service v.2 delivers significant scalability
>>> and
>>> >>> usability improvements based on a new architecture. What we would
>>> like to
>>> >>> merge to trunk is termed "alpha 2" (milestone 2). The feature has a
>>> >>> complete end-to-end read

Re: [DISCUSS] Branches and versions for Hadoop 3

[varunsax...@apache.org]

Hi Andrew,

We have completed the merge of TSv2 to trunk.
You can now go ahead with the branching.

Regards,
Varun Saxena.

On Tue, Aug 29, 2017 at 11:35 PM, Andrew Wang 
wrote:

> Sure. Ping me when the TSv2 goes in, and I can take care of branching.
>
> We're still waiting on the native services and S3Guard merges, but I don't
> want to hold branching to the last minute.
>
> On Tue, Aug 29, 2017 at 10:51 AM, Vrushali C 
> wrote:
>
>> Hi Andrew,
>> As Rohith mentioned, if you are good with it, from the TSv2 side, we are
>> ready to go for merge tonight itself (Pacific time)  right after the voting
>> period ends. Varun Saxena has been diligently rebasing up until now so most
>> likely our merge should be reasonably straightforward.
>>
>> @Wangda: your resource profile vote ends tomorrow, could we please
>> coordinate our merges?
>>
>> thanks
>> Vrushali
>>
>>
>> On Mon, Aug 28, 2017 at 10:45 PM, Rohith Sharma K S <
>> rohithsharm...@apache.org> wrote:
>>
>>> On 29 August 2017 at 06:24, Andrew Wang 
>>> wrote:
>>>
>>> > So far I've seen no -1's to the branching proposal, so I plan to
>>> execute
>>> > this tomorrow unless there's further feedback.
>>> >
>>> For on going branch merge threads i.e TSv2, voting will be closing
>>> tomorrow. Does it end up in merging into trunk(3.1.0-SNAPSHOT) and
>>> branch-3.0(3.0.0-beta1-SNAPSHOT) ? If so, would you be able to wait for
>>> couple of more days before creating branch-3.0 so that TSv2 branch merge
>>> would be done directly to trunk?
>>>
>>>
>>>
>>> >
>>> > Regarding the above discussion, I think Jason and I have essentially
>>> the
>>> > same opinion.
>>> >
>>> > I hope that keeping trunk a release branch means a higher bar for
>>> merges
>>> > and code review in general. In the past, I've seen some patches
>>> committed
>>> > to trunk-only as a way of passing responsibility to a future user or
>>> > reviewer. That doesn't help anyone; patches should be committed with
>>> the
>>> > intent of running them in production.
>>> >
>>> > I'd also like to repeat the above thanks to the many, many contributors
>>> > who've helped with release improvements. Allen's work on
>>> create-release and
>>> > automated changes and release notes were essential, as was Xiao's work
>>> on
>>> > LICENSE and NOTICE files. I'm also looking forward to Marton's site
>>> > improvements, which addresses one of the remaining sore spots in the
>>> > release process.
>>> >
>>> > Things have gotten smoother with each alpha we've done over the last
>>> year,
>>> > and it's a testament to everyone's work that we have a good
>>> probability of
>>> > shipping beta and GA later this year.
>>> >
>>> > Cheers,
>>> > Andrew
>>> >
>>> >
>>>
>>
>>
>

Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk

[varunsax...@apache.org]

+1 (binding).

Kudos to all the team members for their great work!

Being part of the ATSv2 team, I have been involved with either development
or review of most of the JIRAs'.
Tested ATSv2 in both secure and non-secure mode. Also verified that there
is no impact when ATSv2 is turned off.

Regards,
Varun Saxena.

On Tue, Aug 22, 2017 at 12:02 PM, Vrushali Channapattan <
vrushalic2...@gmail.com> wrote:

> Hi folks,
>
> Per earlier discussion [1], I'd like to start a formal vote to merge
> feature branch YARN-5355 [2] (Timeline Service v.2) to trunk. The vote will
> run for 7 days, and will end August 29 11:00 PM PDT.
>
> We have previously completed one merge onto trunk [3] and Timeline Service
> v2 has been part of Hadoop release 3.0.0-alpha1.
>
> Since then, we have been working on extending the capabilities of Timeline
> Service v2 in a feature branch [2] for a while, and we are reasonably
> confident that the state of the feature meets the criteria to be merged
> onto trunk and we'd love folks to get their hands on it in a test capacity
> and provide valuable feedback so that we can make it production-ready.
>
> In a nutshell, Timeline Service v.2 delivers significant scalability and
> usability improvements based on a new architecture. What we would like to
> merge to trunk is termed "alpha 2" (milestone 2). The feature has a
> complete end-to-end read/write flow with security and read level
> authorization via whitelists. You should be able to start setting it up and
> testing it.
>
> At a high level, the following are the key features that have been
> implemented since alpha1:
> - Security via Kerberos Authentication and delegation tokens
> - Read side simple authorization via whitelist
> - Client configurable entity sort ordering
> - Richer REST APIs for apps, app attempts, containers, fetching metrics by
> timerange, pagination, sub-app entities
> - Support for storing sub-application entities (entities that exist outside
> the scope of an application)
> - Configurable TTLs (time-to-live) for tables, configurable table prefixes,
> configurable hbase cluster
> - Flow level aggregations done as dynamic (table level) coprocessors
> - Uses latest stable HBase release 1.2.6
>
> There are a total of 82 subtasks that were completed as part of this
> effort.
>
> We paid close attention to ensure that once disabled Timeline Service v.2
> does not impact existing functionality when disabled (by default).
>
> Special thanks to a team of folks who worked hard and contributed towards
> this effort with patches, reviews and guidance: Rohith Sharma K S, Varun
> Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli, Joep
> Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack.
>
> Regards,
> Vrushali
>
> [1] http://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg27383.html
> [2] https://issues.apache.org/jira/browse/YARN-5355
> [3] https://issues.apache.org/jira/browse/YARN-2928
> [4] https://github.com/apache/hadoop/commits/YARN-5355
>

Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk

[varunsax...@apache.org]

Thanks Subru for voting.


> What are the timelines you are looking for getting this into branch-2?

We haven't yet decided on it and were thinking of discussing this in detail
within the team after merge to trunk.
The timelines would depend on whether we release whatever we merge to trunk
in 2.9 or would we want to get in few other features which people would
like to see in 2.9
This would require some discussion with the stakeholders.
We were thinking of having a short discussion with you guys as well to find
out whether there are any further gaps in ATSv2 with respect to federation
support and if they can be filled before 2.9 release.

Assuming 2.9 is targetted for October end, we would have to start a merge
vote in September end or October 1st week which leaves us with very little
time to take up large changes anyways.

We do maintain a branch-2 version of ATSv2(YARN-5355_branch2) though which
we rebase with branch-2 regularly. So, if we decide to merge in branch-2
without any additional changes, we would be able to go for branch-2 merge
discussion and vote almost immediately.

Regards,
Varun Saxena.




On Sat, Aug 26, 2017 at 2:00 AM, Subramaniam V K  wrote:

> +1 (binding).
>
> I have been following the effort and had few design discussions around the
> team especially about how it integrates with Federation. Overall I feel
> it's a welcome improvement to YARN.
>
> What are the timelines you are looking for getting this into branch-2?
>
> Thanks,
> Subru
>
> On Fri, Aug 25, 2017 at 10:04 AM, Sangjin Lee  wrote:
>
> > +1 (binding)
> >
> > I've built the current branch, and checked out a few basic areas
> including
> > documentation. Also perused the most recent changes that went in.
> >
> > Thanks much for the great team work! I look forward to seeing it in
> action.
> >
> > Regards,
> > Sangjin
> >
> > On Fri, Aug 25, 2017 at 9:27 AM, Haibo Chen 
> > wrote:
> >
> > > +1 from my side.
> > >
> > > More from the perspective of ensuring there is no impact of ATSv2 when
> it
> > > is off (by default), I deployed the latest YARN-5355 bits into a few
> > > clusters and ran internal Smoke tests. The tests shows no impact when
> > ATSv2
> > > is off.
> > >
> > > Best,
> > > Haibo
> > >
> > > On Thu, Aug 24, 2017 at 7:51 AM, Sunil G  wrote:
> > >
> > > > Thank you very much Vrushali, Rohith, Varun and other folks who made
> > this
> > > > happen. Great work, really appreciate the same!!
> > > >
> > > > +1 (binding) from my side:
> > > >
> > > > # Tested ATSv2 cluster in a secure cluster. Ran some basic jobs
> > > > # Accessed new YARN UI which shows various flows/flow activity etc.
> > Seems
> > > > fine.
> > > > # Based on code, looks like all apis are compatible.
> > > > # REST api docs looks fine as well, I guess we could improve that a
> bit
> > > > more post merge as well.
> > > > # Adding to additional thoughts which are discussed here, native
> > service
> > > > also could publish events to atsv2. I think that work is also
> happened
> > in
> > > > branch.
> > > >
> > > > Looking forward to a much wider adoption of ATSv2 with more projects.
> > > >
> > > > Thanks
> > > > Sunil
> > > >
> > > >
> > > > On Tue, Aug 22, 2017 at 12:02 PM Vrushali Channapattan <
> > > > vrushalic2...@gmail.com> wrote:
> > > >
> > > > > Hi folks,
> > > > >
> > > > > Per earlier discussion [1], I'd like to start a formal vote to
> merge
> > > > > feature branch YARN-5355 [2] (Timeline Service v.2) to trunk. The
> > vote
> > > > will
> > > > > run for 7 days, and will end August 29 11:00 PM PDT.
> > > > >
> > > > > We have previously completed one merge onto trunk [3] and Timeline
> > > > Service
> > > > > v2 has been part of Hadoop release 3.0.0-alpha1.
> > > > >
> > > > > Since then, we have been working on extending the capabilities of
> > > > Timeline
> > > > > Service v2 in a feature branch [2] for a while, and we are
> reasonably
> > > > > confident that the state of the feature meets the criteria to be
> > merged
> > > > > onto trunk and we'd love folks to get their hands on it in a test
> > > > capacity
> > > > > and provide valuable feedback so that we can make it
> > production-ready.
> > > > >
> > > > > In a nutshell, Timeline Service v.2 delivers significant
> scalability
> > > and
> > > > > usability improvements based on a new architecture. What we would
> > like
> > > to
> > > > > merge to trunk is termed "alpha 2" (milestone 2). The feature has a
> > > > > complete end-to-end read/write flow with security and read level
> > > > > authorization via whitelists. You should be able to start setting
> it
> > up
> > > > and
> > > > > testing it.
> > > > >
> > > > > At a high level, the following are the key features that have been
> > > > > implemented since alpha1:
> > > > > - Security via Kerberos Authentication and delegation tokens
> > > > > - Read side simple authorization via whitelist
> > > > > - Client configurable entity sort 

Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk

[varunsax...@apache.org]

Hi All,

Just to update.
Folks who may be interested in going through the documentation for the
feature can refer to current documentation attached in pdf format, on the
umbrella JIRA i.e. YARN-5355.

Furthermore, we have run the YARN-5355 patch(with all the changes made in
the branch) against trunk and the build is almost green barring a few
checkstyle issues. The test failures which come up in the build are
outstanding issues on trunk.
Refer to https://issues.apache.org/jira/browse/YARN-5355?focusedCo
mmentId=16138266=com.atlassian.jira.plugin.system.
issuetabpanels:comment-tabpanel#comment-16138266

Thanks,
Varun Saxena.

On Tue, Aug 22, 2017 at 12:02 PM, Vrushali Channapattan <
vrushalic2...@gmail.com> wrote:

> Hi folks,
>
> Per earlier discussion [1], I'd like to start a formal vote to merge
> feature branch YARN-5355 [2] (Timeline Service v.2) to trunk. The vote will
> run for 7 days, and will end August 29 11:00 PM PDT.
>
> We have previously completed one merge onto trunk [3] and Timeline Service
> v2 has been part of Hadoop release 3.0.0-alpha1.
>
> Since then, we have been working on extending the capabilities of Timeline
> Service v2 in a feature branch [2] for a while, and we are reasonably
> confident that the state of the feature meets the criteria to be merged
> onto trunk and we'd love folks to get their hands on it in a test capacity
> and provide valuable feedback so that we can make it production-ready.
>
> In a nutshell, Timeline Service v.2 delivers significant scalability and
> usability improvements based on a new architecture. What we would like to
> merge to trunk is termed "alpha 2" (milestone 2). The feature has a
> complete end-to-end read/write flow with security and read level
> authorization via whitelists. You should be able to start setting it up and
> testing it.
>
> At a high level, the following are the key features that have been
> implemented since alpha1:
> - Security via Kerberos Authentication and delegation tokens
> - Read side simple authorization via whitelist
> - Client configurable entity sort ordering
> - Richer REST APIs for apps, app attempts, containers, fetching metrics by
> timerange, pagination, sub-app entities
> - Support for storing sub-application entities (entities that exist outside
> the scope of an application)
> - Configurable TTLs (time-to-live) for tables, configurable table prefixes,
> configurable hbase cluster
> - Flow level aggregations done as dynamic (table level) coprocessors
> - Uses latest stable HBase release 1.2.6
>
> There are a total of 82 subtasks that were completed as part of this
> effort.
>
> We paid close attention to ensure that once disabled Timeline Service v.2
> does not impact existing functionality when disabled (by default).
>
> Special thanks to a team of folks who worked hard and contributed towards
> this effort with patches, reviews and guidance: Rohith Sharma K S, Varun
> Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli, Joep
> Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack.
>
> Regards,
> Vrushali
>
> [1] http://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg27383.html
> [2] https://issues.apache.org/jira/browse/YARN-5355
> [3] https://issues.apache.org/jira/browse/YARN-2928
> [4] https://github.com/apache/hadoop/commits/YARN-5355
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[varunsax...@apache.org]

Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen  wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du  wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > 
> > From: John Zhuge 
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org;
> > yarn-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   
> > fs.adl.impl
> > org.apache.hadoop.fs.adl.AdlFileSystem
> >   
> >
> >   
> > fs.AbstractFileSystem.adl.impl
> > org.apache.hadoop.fs.adl.Adl
> >   
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du   > j...@hortonworks.com>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > 
> > From: John Zhuge >
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-...@hadoop.apache.org;
> > hdfs-...@hadoop.apache.org;
> > yarn-...@hadoop.apache.org;
> > mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du  > u...@hortonworks.com>> wrote:
> > Hi all,
> >  With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> >  This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> >   More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >   New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> >   The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> >   The maven artifacts are available via repository.apache.org
>  > repository.apache.org> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> >   Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>