Re: Apache Hadoop 3.1.1 release plan

2018-05-10 Thread Wangda Tan
p Map/Reduce") AND status = Resolved AND fixVersion = 3.1.1 [2] project in (YARN, HADOOP, MAPREDUCE, "Hadoop Development Tools") AND priority in (Blocker, Critical) AND resolution = Unresolved AND "Target Version/s" = 3.1.1 ORDER BY priority DESC On Thu, May 10, 2018

Re: Apache Hadoop 3.1.1 release plan

2018-05-10 Thread Wangda Tan
gt; Yes, Billie is reviewing YARN-8265 and I am helping in YARN-8236. > > - Sunil > > > On Thu, May 10, 2018 at 2:25 PM Brahma Reddy Battula < > brahmareddy.batt...@huawei.com> wrote: > >> Thanks Wangda Tan for driving the 3.1.1 release.Yes,This can be better >&

Apache Hadoop 3.1.1 release plan

2018-04-19 Thread Wangda Tan
to cut branch-3.1.1 on May 01 and vote for RC on the same day. Please feel free to share your insights. Thanks, Wangda Tan [1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop Map/Reduce") AND fixVersion = 3.1.1

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-10 Thread Wangda Tan
is is similar for all releases now, and I believe it should be fixed. > > Thanks, > --Konstantin > > On Fri, Apr 6, 2018 at 2:01 PM, Wangda Tan <wheele...@gmail.com> wrote: > >> Thanks guys for the additional votes! I just sent out announcement email. >> >

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-06 Thread Wangda Tan
gt; - Verified checksums for source and binary artifacts > > - Built from source > > - Deployed a single node cluster > > - Verified web UIs, include Namenode, RM, etc. > > * Tried shell commands of HDFS and YARN > > * Ran sample MR jobs, include PI, Sleep, Terasort, etc.

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-05 Thread Wangda Tan
Thanks everybody for voting! The vote passes successfully with 11 binding +1 votes, 8 non-binding +1 votes and no -1s. I will work on the staging and releases. Best, Wangda Tan On Thu, Apr 5, 2018 at 10:46 AM, Vinod Kumar Vavilapalli <vino...@apache.org > wrote: > That is a great ob

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-03 Thread Wangda Tan
st replace both the maven jars as well as > the binaries to be consistent and correct. And then (b) Give a couple more > days for folks who tested on the binaries to reverify - I count one such > clear vote as of now. > > Thanks > +Vinod > > > On Apr 3, 2018, at 3:30 PM, Wangda Tan

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-03 Thread Wangda Tan
hecked jars on r.a.o. > * Built from source > * Deployed to 3 node secure cluster with NameNode HA > * Verified HDFS web UIs > * Tried out HDFS shell commands > * Ran sample MapReduce jobs > > Thanks! > > > --------

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-02 Thread Wangda Tan
://people.apache.org/~wangda/hadoop-3.1.0-RC1/). I think we don't need roll another RC, just update Maven repo should be sufficient. Best, Wangda On Mon, Apr 2, 2018 at 2:39 PM, Wangda Tan <wheele...@gmail.com> wrote: > Hi Arpit, > > Thanks for pointing out this. > > I just remove

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC1)

2018-04-02 Thread Wangda Tan
Thanks Arpit for reporting this and helping investigate the issue offline! I follow the same tutorial so the previously RC1 maven repo has the same issue. Just deployed jars by removing -DskipShade and shaded jars have correct sizes. Will post an update to RC1 voting thread. - Wangda On Mon,

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-02 Thread Wangda Tan
gt; SHOULD NOT supply a MD5 checksum file (because MD5 is too broken). > > https://www.apache.org/dev/release-distribution#sigs-and-sums > > > > On Apr 2, 2018, at 7:03 AM, Wangda Tan <wheele...@gmail.com> wrote: > > Hi Gera, > > It's my bad, I thought only s

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-04-02 Thread Wangda Tan
s coming >> correct. >> - Verified old UI and new YARN UI for labels. >> - Submitted apps to labelled cluster and it works fine. >> - Also performed few cli commands related to nodelabel. >>- Test basic HA cases and seems correct. >&

[VOTE] Release Apache Hadoop 3.1.0 (RC1)

2018-03-29 Thread Wangda Tan
Hi folks, Thanks to the many who helped with this release since Dec 2017 [1]. We've created RC1 for Apache Hadoop 3.1.0. The artifacts are available here: http://people.apache.org/~wangda/hadoop-3.1.0-RC1 The RC tag in git is release-3.1.0-RC1. Last git commit SHA is

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

2018-03-29 Thread Wangda Tan
AfQ> . Thanks, Wangda On Thu, Mar 29, 2018 at 11:16 AM, Wangda Tan <wheele...@gmail.com> wrote: > Thanks Junping for reporting this, > > I did append my keys to https://svn.apache.org/ > repos/asf/hadoop/common/dist/KEYS, I'm not sure how to propagate it to > https:/

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

2018-03-29 Thread Wangda Tan
s/dist/release/hadoop/common/KEYS. Would you > please upload it there then we can go ahead to verify the signature of > following RCs.Thanks! > > Thanks, > > Junping > > Wangda Tan <wheele...@gmail.com>于2018年3月30日 周五上午12:01写道: > >> Thanks all for doing testing an

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

2018-03-29 Thread Wangda Tan
nf mutation disabled in this case > - verified capacity scheduler conf mutation with leveldb storage > - verified refreshQueues mutation is disabled in this case > > > Jonathan Hung > > On Thu, Mar 22, 2018 at 9:10 AM, Wangda Tan <wheele...@gmail.com> wrote: >

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

2018-03-22 Thread Wangda Tan
1086/ > > > Thanks, > Bharat > > > On 3/21/18, 11:44 PM, "Wangda Tan" <wheele...@gmail.com> wrote: > > Hi folks, > > Thanks to the many who helped with this release since Dec 2017 [1]. > We've > created RC0 for Apache Hadoop 3.1.0. The arti

[VOTE] Release Apache Hadoop 3.1.0 (RC0)

2018-03-22 Thread Wangda Tan
Hi folks, Thanks to the many who helped with this release since Dec 2017 [1]. We've created RC0 for Apache Hadoop 3.1.0. The artifacts are available here: http://people.apache.org/~wangda/hadoop-3.1.0-RC0/ The RC tag in git is release-3.1.0-RC0. The maven artifacts are available via

Branch-3.1.0 Created

2018-03-20 Thread Wangda Tan
Devs, Today I updated fix version of all 3.1.0 JIRAs and moved all unresolved JIRA with target version = 3.1.0 to 3.2.0. I've created branch-3.1.0 and working on the RC0 now (still got various of issues while doing the release, but hopefully, I can get the RC0 done soon). If you have *ANY fix*

Re: About reset branch-3.1 to trunk before release.

2018-03-20 Thread Wangda Tan
.1 > previously since I noticed that the branch will be reset to trunk. > > If you have no other concern, I will check pick HADOOP-15262 to the new > 3.1 branch today. > > > Thanks, > Sammi > -----Original Message- > From: Wangda Tan [mailto:wheele...@gmail

Re: About reset branch-3.1 to trunk before release.

2018-03-19 Thread Wangda Tan
. AliyunOSS: move files under a directory in parallel when rename a directory. Contributed by Jinhu Wu. MAPREDUCE-7066. TestQueue fails on Java9 YARN-8028. Support authorizeUserAccessToQueue in RMWebServices. Contributed by Wangda Tan. YARN-8040. [UI2] New YARN UI webapp does not respect

Re: About reset branch-3.1 to trunk before release.

2018-03-19 Thread Wangda Tan
.jp> wrote: > +1 for resetting branch-3.1. > > Thanks, > Akira > > > On 2018/03/18 12:51, Wangda Tan wrote: > >> Thanks for sharing your thoughts. >> >> We have done build and single node cluster deploy / test for the latest >> trunk code (co

Re: About reset branch-3.1 to trunk before release.

2018-03-17 Thread Wangda Tan
From the list, it doesn't look like resetting will destabilize 3.1, +1 for > doing this. > > Thanks > +Vinod > > > On Mar 14, 2018, at 1:54 PM, Wangda Tan <wheele...@gmail.com> wrote: > > > > Hi mapreduce/yarn/common/hdfs-devs, > > > > As of now, we have all blo

About reset branch-3.1 to trunk before release.

2018-03-14 Thread Wangda Tan
Hi mapreduce/yarn/common/hdfs-devs, As of now, we have all blockers done for 3.1.0 release [1]. The release is running behind schedule due to a few security-related issues. Because of this and since branch-3.1 is cut 5 weeks before on Feb 8, trunk 3.2 is already diverging. There're 64 commits in

Re: [VOTE] Merging branch HDFS-7240 to trunk

2018-03-02 Thread Wangda Tan
I like the idea of same source / same release and put Ozone's source under a different directory. Like Owen mentioned, It gonna be important for all parties to keep a regular and shorter release cycle for Hadoop, e.g. 3-4 months between minor releases. Users can try features and give feedbacks to

[jira] [Reopened] (HADOOP-15007) Stabilize and document Configuration element

2018-02-27 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reopened HADOOP-15007: - > Stabilize and document Configuration elem

3.1.0 release status update - Feb 17, 2018

2018-02-17 Thread Wangda Tan
Hi All, Thanks for helping with 3.1.0 releases. After many patches resolved or moved, now we have 9 blockers/criticals [1] (which is down from 33 blockers/criticals in one week before!). Among these 9 blockers/critical: - All 3 YARN JIRAs will be resolved in a few days. - 3 fs/s3 related JIRAs

How to get Jenkins job create/edit permission?

2018-02-12 Thread Wangda Tan
Hi guys, I was trying to run JACC build for 3.1.0 release. When I run JACC to compare branch-2.9 or branch-3.0 with branch-3.1, I found the Jenkins build got killed after 1 hour. For example: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/1828/consoleFull I think there're

Re: [NOTICE] branch-3.1 created. 3.1.0 code freeze started.

2018-02-10 Thread Wangda Tan
.1.0: 3.1.0 > branch-3.1: 3.1.x > trunk: 3.2.x > > Code freezes in branch-3.1.0 which only allows blocks. > However, did you mean you created branch-3.1 for 3.1.0? In this case, what > branch should be used for 3.1.x? > > -- > Weiwei > > On 11 Feb 2018, 11:11 AM +

[NOTICE] branch-3.1 created. 3.1.0 code freeze started.

2018-02-10 Thread Wangda Tan
Hi all, As proposed in [1], code freeze of 3.1.0 is already started. I created branch-3.1 from trunk and will set target version of trunk to 3.2.0. Please note that only blockers/criticals can be committed to 3.1.0, and all such commits should be cherry-picked to branch-3.1. [1]

3.1.0 Code Freeze Will Be Started From Feb 08

2018-02-05 Thread Wangda Tan
ion/s" = 3.1.0 ORDER BY key ASC, priority DESC Thanks, Wangda Tan

Re: Hadoop 3.1.0 release discussion

2018-01-31 Thread Wangda Tan
I would tend to agree with that. Let's see what we > can to do get profiles nailed down so that 3.1 can go forward. > > Daniel > > > On 1/18/18 10:25 AM, Wangda Tan wrote: > > Thanks Daniel, > > We need to make a decision about this: https://issues.apac

Re: Hadoop 3.1.0 release discussion

2018-01-30 Thread Wangda Tan
ouple of days I will be in travel, I would not be able to put my > efforts in it. Meanwhile Rakesh/Surendra will update if any changes in the > plan. > > > > So, to conclude, you can proceed cutting the branch and we will merge > after reviews closed, that may go into 3.2. &g

Re: Hadoop 3.1.0 release discussion

2018-01-30 Thread Wangda Tan
ize the > comments and decide. > Will keep updated here. > > Regards, > Uma > > > On 1/28/18, 5:31 PM, "Wangda Tan" <wheele...@gmail.com> wrote: > > Hi Uma, > > Thanks, I saw HDFS-13050 has been resolved 4 hours ago, I don't see any > ot

Re: Hadoop 3.1.0 release discussion

2018-01-28 Thread Wangda Tan
and we finished the key > implementation from HDFS-12995, by HDFS-13075. > > We are planning to start vote by tomorrow (29th PST time). So, we request > you to give us time for running vote. We will keep SPS off by default. So, > interested users only can enable explicitly. > &

Re: Hadoop 3.1.0 release discussion

2018-01-26 Thread Wangda Tan
, Gangumalla, Uma <uma.ganguma...@intel.com> wrote: > Sure, Wangda. > > Regards, > Uma > > On 1/18/18, 10:19 AM, "Wangda Tan" <wheele...@gmail.com> wrote: > > Thanks Uma, > > Could you update this thread once the merge vote started? > >

Re: [VOTE] Merge YARN-6592 feature branch to trunk

2018-01-26 Thread Wangda Tan
+1 On Sat, Jan 27, 2018 at 2:05 AM, Chris Douglas wrote: > +1 Looking forward to this. -C > > On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh wrote: > > Hello yarn-dev@ > > > > Based on the positive feedback from the DISCUSS thread [1], I'd like to > >

Re: [DISCUSS] Merge YARN-6592 to trunk

2018-01-25 Thread Wangda Tan
Thanks Arun +1, Best, Wangda On Fri, Jan 26, 2018 at 7:46 AM, Arun Suresh wrote: > Hello yarn-dev@ > > We feel that the YARN-6592 dev branch mostly in shape to be merged into > trunk. This branch adds support for placing containers in YARN using rich > placement

Re: Hadoop 3.1.0 release discussion

2018-01-18 Thread Wangda Tan
rethink some of the design choices. > > Daniel > > > On 1/17/18 11:33 AM, Wangda Tan wrote: > >> Hi All, >> >> Since we're fast approaching previously proposed feature freeze date (Jan >> 30, about 13 days from today). If you've any features which live in a >>

Re: Hadoop 3.1.0 release discussion

2018-01-18 Thread Wangda Tan
h the tasks sooner > before the deadline. > > Regards, > Uma > > On 1/17/18, 11:35 AM, "Wangda Tan" <wheele...@gmail.com> wrote: > > Hi All, > > Since we're fast approaching previously proposed feature freeze date > (Jan > 30, about 13

Re: Hadoop 3.1.0 release discussion

2018-01-17 Thread Wangda Tan
0 PM, Wangda Tan <wheele...@gmail.com> wrote: > Hi all, > > Congratulations on the 3.0.0-GA release! > > As we discussed in the previous email thread [1], I'd like to restart > 3.1.0 release plans. > > a) Quick summary: > a.1 Release status > We started 3.1 release di

Hadoop 3.1.0 release discussion

2017-12-15 Thread Wangda Tan
th Vinod to get help on release-management given this is my first release. He agreed to help do this release jointly. Thoughts? Thanks, Wangda Tan [1] https://lists.apache.org/thread.html/c11506c3250c9481852130616b3cb09a0e222f5c2465c015f9906dab@%3Cyarn-dev.hadoop.apache.org%3E [2] "project in

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-13 Thread Wangda Tan
Thanks Andrew for driving this. +1 (Binding) Ran SLS + CS's Perf unit test and saw similar performance compared to trunk. Compiled from source, deployed single node cluster and ran several jobs. Best, Wangda On Wed, Dec 13, 2017 at 7:29 AM, Sunil G wrote: > +1

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

2017-12-12 Thread Wangda Tan
+1 (Binding). Built from source and deployed a cluster, jobs can run successfully. Thanks Junping for driving this. Best, Wangda On Tue, Dec 12, 2017 at 5:18 PM, Chandni Singh wrote: > +1 > Built the source code. > Deployed a single node cluster and ran example

Re: [VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk

2017-11-30 Thread Wangda Tan
+1, thanks Sunil! Wangda Tan On Wed, Nov 29, 2017 at 5:56 PM, Sunil G <sun...@apache.org> wrote: > Hi All, > > > Based on the discussion at [1], I'd like to start a vote to merge feature > branch > > YARN-5881 to trunk. Vote will run for 7 days, ending Wedn

Re: [DISCUSS] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk

2017-11-27 Thread Wangda Tan
to-end and we are running this in our > development cluster for last couple of months and undergone good amount of > testing. Branch code is run against trunk and tracked via [4]. > > We would love to get your thoughts before opening a voting thread. > > Special thanks to a team

Re: Apache Hadoop 2.8.3 Release Plan

2017-11-21 Thread Wangda Tan
Thanks Junping for driving this. For the bug fix vs. improvement, it is actually very hard to define, improvement could be self-contained and useful, bug fix could be dangerous in some cases. To me, If an improvement fixed some existing use case, and the fix is self-contained. I will be open to

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

2017-11-16 Thread Wangda Tan
+1 (Binding). Built from source, deployed pseudo cluster, ran sample jobs. Thanks, Wangda On Wed, Nov 15, 2017 at 8:34 PM, Brahma Reddy Battula wrote: > +1 ( non-binding) > > -Built from the source > -Installed 3 Node HA cluster and a pseudo cluster > -Verified through hdfs

Re: [DISCUSS] A final minor release off branch-2?

2017-11-13 Thread Wangda Tan
Thanks Vinod for staring this, I'm also leaning towards the plan (A): * (A)-- Make 2.9.x the last minor release off branch-2-- Have a maintenance release that bridges 2.9 to 3.x-- Continue to make more maintenance releases on 2.8 and 2.9 as necessary* The only part I'm not sure

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC0)

2017-11-07 Thread Wangda Tan
Sunil / Rohith, Could you check if your configs are same as Jonathan posted configs? https://issues.apache.org/jira/browse/YARN-7453?focusedCommentId=16242693=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16242693 And could you try if using Jonathan's configs can still

Re: [VOTE] Merge yarn-native-services branch into trunk

2017-11-03 Thread Wangda Tan
Thanks all for the great works! +1 (Binding). Tried to use native services to build and run applications successfully. - Wangda On Fri, Nov 3, 2017 at 11:50 AM, Arun Suresh wrote: > +1 (binding) > > Cheers > -Arun > > On Nov 3, 2017 11:44 AM, "Chandni Singh"

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

2017-10-23 Thread Wangda Tan
Thanks Junping, +1 (binding). Deployed a single node cluster and ran some simple jobs. - Wangda On Mon, Oct 23, 2017 at 4:05 PM, Chris Douglas wrote: > +1 (binding) > > Looked through the src distribution. Checksum, signatures match, ran > some of the unit tests. Also

Re: Do we still have nightly (or even weekly) unit test run for Hadoop projects?

2017-10-19 Thread Wangda Tan
te: > >> Yes, qbt runs nightly and it sends e-mail to dev lists. >> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/ >> >> Regards, >> Akira >> >> >> On 2017/10/19 7:54, Wangda Tan wrote: >> >>> Hi, >>

Do we still have nightly (or even weekly) unit test run for Hadoop projects?

2017-10-18 Thread Wangda Tan
Hi, Do we still have nightly (or even weekly) unit test run for Hadoop projects? I couldn't find it on Jenkins dashboard and I haven't seen reports set to dev lists for a while. Thanks, Wangda

Re: [VOTE] Merge feature branch YARN-5734 (API based scheduler configuration) to trunk, branch-3.0, branch-2

2017-10-02 Thread Wangda Tan
zookeeper implementation are supported. > > > > There were 15 subtasks completed for this feature. > > > > Huge thanks to everyone who helped with reviews, commits, guidance, and > > technical discussion/design, including Carlo Curino, Xuan Gong, Subru > > Krishnan,

[jira] [Reopened] (HADOOP-14670) Increase minimum cmake version for all platforms

2017-09-15 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reopened HADOOP-14670: - > Increase minimum cmake version for all platfo

Re: [DISCUSS] official docker image(s) for hadoop

2017-09-13 Thread Wangda Tan
+1 to add Hadoop docker image for easier testing / prototyping, it gonna be super helpful! Thanks, Wangda On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi < miklos.szeg...@cloudera.com> wrote: > Marton, thank you for working on this. I think Official Docker images for > Hadoop would be very

Re: [VOTE] Merge YARN-3926 (resource profile) to trunk

2017-09-12 Thread Wangda Tan
> types to allow a user to request a group of resources as a profile, much > > like EC2 instance types, e.g. "fast-compute" might mean 32GB RAM, 8 > > vcores, and 2 GPUs. > > > > Daniel > > > > On 8/23/17 11:49 AM, Wangda Tan wrote: > > >

Re: Release 2.9.0 update

2017-09-11 Thread Wangda Tan
Thanks Arun Suresh for creating the page. +1 to have a page to track all planned releases, I also updated 3.1 release plans. Several suggestions: 1) Only put large features / changes which needed to be included by the release, otherwise we will have too long list of JIRAs. 2) For release

Re: [DISCUSS] Looking to Apache Hadoop 3.1 release

2017-09-07 Thread Wangda Tan
N feature I'd > like to add to 3.1.0 is YARN Oversubscription (YARN-1011) > > Best, > Haibo > > On Wed, Sep 6, 2017 at 11:13 AM, Wangda Tan <wheele...@gmail.com> wrote: > >> Hi all, >> >> As we discussed on [1], there were proposals from Steve

[DISCUSS] Looking to Apache Hadoop 3.1 release

2017-09-06 Thread Wangda Tan
if you have any suggestions/concerns. Thanks, Wangda Tan [1] http://markmail.org/message/hwar5f5ap654ck5o?q= Branch+merges+and+3%2E0%2E0-beta1+scope [2] http://markmail.org/message/hwar5f5ap654ck5o?q=Branch+ merges+and+3%2E0%2E0-beta1+scope#query:Branch%20merges% 20and%203.0.0-beta1%20scope+page:1+mid

Re: [VOTE] Merge yarn-native-services branch into trunk

2017-09-01 Thread Wangda Tan
+1 (Binding), I tried to use YARN service assembly before to run different kinds of jobs (for example, distributed Tensorflow), it is really easy for end user to run jobs on YARN. Thanks to the whole team for the great job! Best, Wangda On Fri, Sep 1, 2017 at 3:33 PM, Gour Saha

Re: [DISCUSS] Branches and versions for Hadoop 3

2017-08-29 Thread Wangda Tan
-beta1 > (3.0.0-beta1-SNAPSHOT). This way we can merge GA features to branch-3.0 but > not branch-3.0.0-beta1. > > Best, > Andrew > > On Tue, Aug 29, 2017 at 11:18 AM, Wangda Tan <wheele...@gmail.com> wrote: > >> Vrushali, >> >> Sure we can wait TSv2 merg

Re: [DISCUSS] Branches and versions for Hadoop 3

2017-08-29 Thread Wangda Tan
Vrushali, Sure we can wait TSv2 merged before merge resource profile branch. Andrew, My understanding is you're going to cut branch-3.0 for 3.0-beta1, and the same branch (branch-3.0) will be used for 3.0-GA as well. So my question is, there're several features (TSv2, resource profile,

Re: [DISCUSS] Branches and versions for Hadoop 3

2017-08-25 Thread Wangda Tan
Hi Andrew, Thanks for updating the proposal, +1. - Wangda On Fri, Aug 25, 2017 at 12:16 PM, Allen Wittenauer wrote: > > > On Aug 25, 2017, at 10:36 AM, Andrew Wang > wrote: > > > Until we need to make incompatible changes, there's no

Re: Branch merges and 3.0.0-beta1 scope

2017-08-25 Thread Wangda Tan
Resource profile is similar to TSv2, the feature is: - Alpha feature, we will not freeze new added APIs. And all added APIs are explicitly marked to @Unstable. - Allow rolling upgrade from branch-2. - Touched existing code, but we have, and will continue tests to make sure changes are safe.

[VOTE] Merge YARN-3926 (resource profile) to trunk

2017-08-23 Thread Wangda Tan
and contributed towards this effort including design discussion/development/reviews, etc.: Varun Vasudev, Sunil Govind, Daniel Templeton, Vinod Vavilapalli, Yufei Gu, Karthik Kambatla, Jason Lowe, Arun Suresh. Regards, Wangda Tan [1] http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/20170

Re: Branch merges and 3.0.0-beta1 scope

2017-08-21 Thread Wangda Tan
Andrew, Thanks for your help to pushing this release. Echoing what Vinod said, all contributors in these branches are putting months to years of time working on these features, we don't have to decide excluded features now since we have 25 days till 3.0-beta1 planned release time. The best

Re: [DISCUSS] Merge YARN resource profile (YARN-3926) branch into trunk

2017-08-18 Thread Wangda Tan
t. > > Best, > Andrew > > On Thu, Aug 17, 2017 at 8:27 PM, Wangda Tan <wheele...@gmail.com> wrote: > >> +hdfs/common/mr >> >> On Thu, Aug 17, 2017 at 1:28 PM, Wangda Tan <wheele...@gmail.com> wrote: >> >> > Hi all, >> > >&

Re: [DISCUSS] Merge YARN resource profile (YARN-3926) branch into trunk

2017-08-17 Thread Wangda Tan
+hdfs/common/mr On Thu, Aug 17, 2017 at 1:28 PM, Wangda Tan <wheele...@gmail.com> wrote: > Hi all, > > I want to hear your thoughts of merging YARN resource profile branch into > trunk in the next few weeks. The goal is to get it in for Hadoop 3.0 beta1. > > *Regardi

[jira] [Reopened] (HADOOP-13835) Move Google Test Framework code from mapreduce to hadoop-common

2017-08-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reopened HADOOP-13835: - > Move Google Test Framework code from mapreduce to hadoop-com

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

2017-03-20 Thread Wangda Tan
Thanks Junping for doing this. *+1 (Binding),* Built from source code, deployed a single node cluster. Enabled node labels and tried to run sample jobs. Haven't seen any issue so far. Thanks, Wangda On Mon, Mar 20, 2017 at 10:02 AM, Junping Du wrote: > Thank you for

Re: Maven build: YARN timeline service downloading maven-metadata from personal repository?

2016-12-15 Thread Wangda Tan
into what's happening? > > On Wed, Dec 14, 2016 at 11:01 AM, Wangda Tan <wheele...@gmail.com> wrote: > >> Thanks folks for reply, I tried to change some parameters of maven (like >> useCache=true), but it still doesn'

Re: Maven build: YARN timeline service downloading maven-metadata from personal repository?

2016-12-14 Thread Wangda Tan
d help with this >>>> > build time taken issue. I will start looking into this upgrade right >>>> > away. >>>> > >>>> > Thanks >>>> > Vrushali >>>> > >>>> > > On Dec 13, 2016, at 3:02 PM, Li Lu <llu.u...@gmai

Re: Maven build: YARN timeline service downloading maven-metadata from personal repository?

2016-12-13 Thread Wangda Tan
e's question, I don't think maven downloads anything from > that location (it's a very old content). It just does a wasted effort by > hitting this repo. > > On Mon, Oct 3, 2016 at 10:25 AM, Wangda Tan <wheele...@gmail.com> wrote: > >> Filed: https://issues.apache.org/j

Re: Jenkins patch-compile fails on trunk

2016-11-10 Thread Wangda Tan
This looks like bug caused by npm, looking into it now. Thanks, Wangda On Thu, Nov 10, 2016 at 4:00 PM, Zhe Zhang wrote: > https://builds.apache.org/job/PreCommit-HDFS-Build/17517/ > artifact/patchprocess/patch-compile-root.txt > > Looks related to yarn-ui? The HDFS-10872

Re: [VOTE] Release Apache Hadoop 2.6.5 (RC1)

2016-10-07 Thread Wangda Tan
Thanks Sangjin for cutting this release! +1 (Binding) - Downloaded binary tar ball and setup a single node cluster. - Submit a few applications and which can successfully run. Thanks, Wangda On Fri, Oct 7, 2016 at 10:33 AM, Zhihai Xu wrote: > Thanks Sangjin for

Re: Maven build: YARN timeline service downloading maven-metadata from personal repository?

2016-10-03 Thread Wangda Tan
Filed: https://issues.apache.org/jira/browse/HBASE-16749 On Mon, Oct 3, 2016 at 10:18 AM, Wangda Tan <wheele...@gmail.com> wrote: > Thanks Sangjin/Ted/Steve for your comments/suggestions, I will file a > HBase JIRA later. > > Regards, > Wangda > > On Mon, Oct 3, 2016

Re: Maven build: YARN timeline service downloading maven-metadata from personal repository?

2016-10-03 Thread Wangda Tan
list the ASF snapsphot repo first > > On 1 Oct 2016, at 17:37, Wangda Tan <wheele...@gmail.com> wrote: > > > > Hi YARN-dev, > > > > (cc common-dev), > > > > YARN timeline service currently sometimes downloads maven-metadata.xml >

Maven build: YARN timeline service downloading maven-metadata from personal repository?

2016-10-01 Thread Wangda Tan
Hi YARN-dev, (cc common-dev), YARN timeline service currently sometimes downloads maven-metadata.xml from a personal apache site, log looks like: [INFO] [INFO] Building Apache Hadoop YARN Timeline Service

Re: [VOTE] Release Apache Hadoop 2.7.3 RC2

2016-08-24 Thread Wangda Tan
+1 Binding. - Built and deploy a single node cluster from source code. - Ran some example jobs. Thanks, Wangda On Tue, Aug 23, 2016 at 12:35 PM, Naganarasimha Garla < naganarasimha...@apache.org> wrote: > +1 (non-binding) > > - Downloaded source and tar verified the checksum > - Installed

[jira] [Resolved] (HADOOP-12516) jdiff fails with error 'duplicate comment id' about MetricsSystem.register_changed

2016-08-24 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-12516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved HADOOP-12516. - Resolution: Duplicate This is already fixed by HADOOP-13428, closing as dup. > jdiff fa

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-07-27 Thread Wangda Tan
Thanks Andrew for sharing your thoughts, It looks better if we can put multiple versions on the fix version, with that we can at least do some queries on JIRA to check the issues like "in branch-2.6.5 but not in branch-2.7.4". I still have a couple of questions: *1) How CHANGES.txt (or release

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-26 Thread Wangda Tan
actively working on these items, any helps will be appreciated. Thanks, Wangda On Tue, Jul 26, 2016 at 11:16 AM, Vinod Kumar Vavilapalli < vino...@apache.org> wrote: > +1 > > Thanks > +Vinod > > On Jul 26, 2016, at 7:39 AM, Wangda Tan <wheele...@gmail.com> wrote:

[jira] [Created] (HADOOP-13428) Fix hadoop-common to generate jdiff

2016-07-26 Thread Wangda Tan (JIRA)
Wangda Tan created HADOOP-13428: --- Summary: Fix hadoop-common to generate jdiff Key: HADOOP-13428 URL: https://issues.apache.org/jira/browse/HADOOP-13428 Project: Hadoop Common Issue Type: Bug

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-26 Thread Wangda Tan
lt;bus...@cloudera.com> wrote: > > Just so I don't waste time chasing my tail, should I interpret this > email and the associated JIRA as the PMC preferring I not spend > volunteer time providing a compatibility breakdown as previously > discussed? > >> On Mon, Jul 25, 20

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-25 Thread Wangda Tan
Hi Andrew, Please wait updating fix version for branch-2 committed tickets before we get a consensus on this. Updating fix versions for them could bring lots of troubles for on going two releases (2.7.3 / 2.8.0). Thanks, Wangda On Mon, Jul 25, 2016 at 11:02 AM, Andrew Wang

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-25 Thread Wangda Tan
I just filed ticket https://issues.apache.org/jira/browse/HADOOP-13423 to track running JDIFF on trunk and analyze results for Hadoop-common. I will work on that and keep the JIRA and this thread updated. We need to do the same work for YARN/MR/HDFS. On Mon, Jul 25, 2016 at 5:47 PM, Wangda Tan

[jira] [Created] (HADOOP-13423) Run JDiff on trunk for Hadoop-Common and analyze results

2016-07-25 Thread Wangda Tan (JIRA)
Wangda Tan created HADOOP-13423: --- Summary: Run JDiff on trunk for Hadoop-Common and analyze results Key: HADOOP-13423 URL: https://issues.apache.org/jira/browse/HADOOP-13423 Project: Hadoop Common

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-25 Thread Wangda Tan
I agree with what Vinod mentioned: we need to revisit all incompatible changes and revert unnecessary ones. Even if we don't have any compatibility guarantees between 2.x and 3.x. But make user to be less frustrated while trying 3.x is always a better option to me. To achieve this we need to run

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-25 Thread Wangda Tan
Thanks Vinod. +1 (Binding) - Built from source and deploy a pseudo cluster locally - Run DS on YARN - With/Without node labels enabled. - Wangda On Mon, Jul 25, 2016 at 1:55 PM, Mingliang Liu wrote: > Thanks Vinod. > > +1 (non-binding) > > * Downloaded and built from

Re: Cannot create release artifacts for branch-2.8

2016-06-07 Thread Wangda Tan
kport should be possible, > and hopefully without backporting all the conflicting JIRAs. Maybe another > bash-savvy contributor could attempt it? > > > > On Tue, Jun 7, 2016 at 2:45 PM, Wangda Tan <wheele...@gmail.com> wrote: > >> > >> > I'm not the one

Re: Cannot create release artifacts for branch-2.8

2016-06-07 Thread Wangda Tan
> > I'm not the one who cherry-picked them into branch-2, so I don't > care either way. I've been extremely open about targeting my code for the > past three years for trunk. With few exceptions, I generally only commit > code I write or even review to branch-2 if someone practically begs

Re: Cannot create release artifacts for branch-2.8

2016-06-07 Thread Wangda Tan
om> wrote: > > > On Jun 6, 2016, at 10:50 PM, Wangda Tan <wheele...@gmail.com> wrote: > > > > Hi Hadoop Devs, > > > > As you know, we're pushing 2.8.0 releases recently, there're couple of > > issues that block creating release artifacts from sour

Cannot create release artifacts for branch-2.8

2016-06-06 Thread Wangda Tan
Hi Hadoop Devs, As you know, we're pushing 2.8.0 releases recently, there're couple of issues that block creating release artifacts from source code. I tried following approaches: 1) Run build through Hadoop Jenkins Job: https://builds.apache.org/job/HADOOP2_Release_Artifacts_Builder/ 2) Run

[jira] [Resolved] (HADOOP-13156) create-release.sh doesn't work for branch-2.8

2016-05-18 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved HADOOP-13156. - Resolution: Duplicate This is duplicated to HADOOP-12892 > create-release.sh doesn't w

Re: [DISCUSS] upgrade tomcat from 6.0.44 to 6.0.45 and use non apache.org mirror

2016-05-17 Thread Wangda Tan
of tomcat packages. On Tue, May 17, 2016 at 3:11 PM, Allen Wittenauer <allenwittena...@yahoo.com > wrote: > > > On May 17, 2016, at 2:08 PM, Wangda Tan <wheele...@gmail.com> wrote: > > > > Then I investigated a little more, there're two issues from what I can

[DISCUSS] upgrade tomcat from 6.0.44 to 6.0.45 and use non apache.org mirror

2016-05-17 Thread Wangda Tan
Hi common-devs, When I tried to build Hadoop trunk today, Maven build failed because kms failed to download: http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.44/bin/apache-tomcat-6.0.44.tar.gz . When I open the link from browser, it shows bandwidth limit exceeded: === Bandwidth limit

Re: Tons of "javac: invalid target release: 1.8" failures happen on YARN precommit runs

2016-05-17 Thread Wangda Tan
Thanks Allen, I just triggered other tests to run, will report here if any other issues found. - Wangda On Tue, May 17, 2016 at 11:10 AM, Allen Wittenauer <a...@apache.org> wrote: > > > On May 17, 2016, at 10:54 AM, Wangda Tan <wheele...@gmail.com> wrote: > >

Tons of "javac: invalid target release: 1.8" failures happen on YARN precommit runs

2016-05-17 Thread Wangda Tan
Hi YARN-dev/Common-dev, I noticed lots of "javac: invalid target release: 1.8" failures happen on YARN precommit runs since yesterday, not sure if other projects like HDFS face the same issue. Filed HADOOP-13167 to track the issue. Thanks, Wangda

<    1   2   3   >