Re: [VOTE] Merging branch HDFS-7240 to trunk

Hi Sanjay, thanks for the response, replying inline: - NN on top HDSL where the NN uses the new block layer (Both Daryn and Owen > acknowledge the benefit of the new block layer). We have two choices here > ** a) Evolve NN so that it can interact with both old and new block layer, > ** b)

Re: [VOTE] Merging branch HDFS-7240 to trunk

On Fri, Mar 2, 2018 at 4:24 PM, Owen O'Malley <owen.omal...@gmail.com> > wrote: > >> On Thu, Mar 1, 2018 at 11:03 PM, Andrew Wang <andrew.w...@cloudera.com> >> wrote: >> >> Owen mentioned making a Hadoop subproject; we'd have to >> > hash out what exactl

Re: [VOTE] Merging branch HDFS-7240 to trunk

Hi Sanjay, I have different opinions about what's important and how to eventually integrate this code, and that's not because I'm "conveniently ignoring" your responses. I'm also not making some of the arguments you claim I am making. Attacking arguments I'm not making is not going to change my

Re: [VOTE] Merging branch HDFS-7240 to trunk

pluggable. At that point (likely in the Hadoop 4 timeframe), we'll be ready to pursue HDFS-on-HDSL integration. Best, Andrew On Tue, Feb 27, 2018 at 11:23 PM, Andrew Wang <andrew.w...@cloudera.com> wrote: > > > > > > > > > > *Hi Jitendra and all,Thanks for putting

Re: [VOTE] Merging branch HDFS-7240 to trunk

*Hi Jitendra and all,Thanks for putting this together. I caught up on the discussion on JIRA and document at HDFS-10419, and still have the same concerns raised earlier

Re: [DISCUSS] 2.9+ stabilization branch

3. > GPU support would be a serious catalyst for us to move forward, which I > also heard from other organizations interested in ML. > > Thanks, > --Konstantin > > On Tue, Feb 27, 2018 at 1:28 PM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > >> Hi Arun

Re: [DISCUSS] 2.9+ stabilization branch

Hi Arun/Subru, Bumping the minimum Java version is a major change, and incompatible for users who are unable to upgrade their JVM version. We're beyond the EOL for Java 7, but as we know from our experience with Java 6, there are plenty of users who stick on old Java versions. Bumping the Java

Re: Apache Hadoop 3.0.1 Release plan

Hi Eddy, thanks for taking this on, Historically we've waited for the first RC to cut the release branch since it keeps things simpler for committers. Also, could you check the permissions on your JIRA filter? It shows as private for me. Best, Andrew On Tue, Jan 9, 2018 at 11:17 AM, Lei Xu

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

ot; to match > the pattern for prior releases. > > Thanks, > Jonathan Kelly > > On Thu, Dec 14, 2017 at 10:45 AM Andrew Wang <andrew.w...@cloudera.com> > wrote: > >> Hi all, >> >> I'm pleased to announce that Apache Hadoop 3.0.0 is generally available >

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

pit Agarwal <aagar...@hortonworks.com> wrote: > That makes sense for Beta users but most of our users will be upgrading > from a previous GA release and the changelog will mislead them. The webpage > does not mention this is a delta from the beta release. > > > > > > *Fr

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

t; I assume you only counted 3.0.0 changes in this log excluding > alphas/betas. However, users shouldn’t have to manually compile > incompatibilities by summing up a/b release notes. Can we fix the changelog > after the fact? > > > > > On 12/14/17, 10:45 AM, "Andrew Wang

[ANNOUNCE] Apache Hadoop 3.0.0 GA is released

Hi all, I'm pleased to announce that Apache Hadoop 3.0.0 is generally available (GA). 3.0.0 GA consists of 302 bug fixes, improvements, and other enhancements since 3.0.0-beta1. This release marks a point of quality and stability for the 3.0.0 release line, and users of earlier 3.0.0-alpha and

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

release. > > > > > > Thanks, > > > > Junping > > > > > > From: Robert Kanter <rkan...@cloudera.com> > > Sent: Tuesday, December 12, 2017 3:10 PM > > To: Arun Suresh > > Cc: Andrew Wang;

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

get/cat on "/" with XOR-2-1 policy. > > Thanks Andrew for this great effort! > > Best, > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > > Hi Wei-Chiu, > > > > The patchprocess directory is left ove

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

nitpicking, but you might want to double check so we don't > ship anything private to you in public :) > > > > On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <ajay.ku...@hortonworks.com> > wrote: > >> +1 (non-binding) >> Thanks for driving this, Andrew Wang!! >&g

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

17, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote: > > > > Hi Andrew, > > > > Thanks much for your effort! Just to be clear, could you please state the > > git commit id of the RC1 we're voting for? > > > > Sangjin > > > > On Fri, Dec

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

ss > because of the mutability of tags. > > On Mon, Dec 11, 2017 at 10:31 AM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > >> Sorry, forgot to push the tag. It's up there now. >> >> On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli < >> vino...@apach

[VOTE] Release Apache Hadoop 3.0.0 RC1

Hi all, Let me start, as always, by thanking the efforts of all the contributors who contributed to this release, especially those who jumped on the issues found in RC0. I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302 fixed JIRAs since the previous 3.0.0-beta1 release.

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

FYI that we got our last blocker in today, so I'm currently rolling RC1. Stay tuned! On Thu, Nov 30, 2017 at 8:32 AM, Allen Wittenauer wrote: > > > On Nov 30, 2017, at 1:07 AM, Rohith Sharma K S < > rohithsharm...@apache.org> wrote: > > > > > > >. If ATSv1 isn’t

2017-12-01 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-12-01 Haven't written one of these in a month. I had high hopes for RC0, but it failed due to HADOOP-15058 - create-release site build

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Hi folks, Thanks again for the testing help with the RC. Here's our dashboard for the 3.0.0 release: https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12329849 Right now we're tracking three blockers: * HADOOP-15058 is the create-release fix, I just put up a patch which needs

Re: Apache Hadoop 2.8.3 Release Plan

gt; > Pls let me know what you think. Thanks! > > > > Thanks, > > > Junping > > > -- > *From:* Andrew Wang <andrew.w...@cloudera.com> > *Sent:* Monday, November 20, 2017 10:22 PM > *To:* Zheng, Kai > *Cc:* Junping Du

Re: Apache Hadoop 2.8.3 Release Plan

> > > >> If we're struggling with being able to deliver new features in a safe > and timely fashion, let's try to address that... > > This is interesting. Do you aware any means to do that? Thanks! > > I've mentioned this a few times on the lists before, but our biggest gap in keeping branches

Re: Apache Hadoop 2.8.3 Release Plan

I'm against including new features in maintenance releases, since they're meant to be bug-fix only. If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning of "maintenance release". Best, Andrew On Mon, Nov

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

On Mon, Nov 20, 2017 at 9:59 PM, Sangjin Lee <sj...@apache.org> wrote: > > On Mon, Nov 20, 2017 at 9:46 PM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > >> Thanks for the spot Sangjin. I think this bug introduced in >> create-release by HADOOP-14835.

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Thanks for the thorough review Vinod, some inline responses: *Issues found during testing* > > Major > - The previously supported way of being able to use different tar-balls > for different sub-modules is completely broken - common and HDFS tar.gz are > completely empty. > Is this something

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

;> for every new 'mvn install' even though there are no source changes - we >>> should see how this can be avoided. >>> - Compatibility notes >>> -- NM's env list is curtailed unlike in 2.x (For e.g, >>> HADOOP_MAPRED_HOME is not automatically inherited. Correct

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

? It would be a courtesy to our broader > community and I see no harm in giving everyone a few days to evaluate it > more thoroughly. > > On a lighter note, your deadline is also 4 minutes short of the required 5 > days. :) > > Regards, > Arpit > > > > On 11/14/17

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

> Thanks, > Akira > > > On 2017/11/15 6:34, Andrew Wang wrote: > >> Hi folks, >> >> Thanks as always to the many, many contributors who helped with this >> release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are >> available here: >> >

Re: [DISCUSS] A final minor release off branch-2?

Hi Junping, On Wed, Nov 15, 2017 at 1:37 AM, Junping Du wrote: > Thanks Vinod to bring up this discussion, which is just in time. > > I agree with most responses that option C is not a good choice as our > community bandwidth is precious and we should focus on very limited

Re: [DISCUSS] A final minor release off branch-2?

To follow up on my earlier email, I don't think there's need for a bridge release given that we've successfully tested rolling upgrade from 2.x to 3.0.0. I expect we'll keep making improvements to smooth over any additional incompatibilities found, but there isn't a requirement that a user upgrade

[VOTE] Release Apache Hadoop 3.0.0 RC0

Hi folks, Thanks as always to the many, many contributors who helped with this release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are available here: http://people.apache.org/~wang/3.0.0-RC0/ This vote will run 5 days, ending on Nov 19th at 1:30pm Pacific. 3.0.0 GA contains 291

Re: Heads up: branching branch-3.0.0 for GA

Branching is complete. Please use the 3.0.1 fix version for further commits to branch-3.0. Ping me if you want something in branch-3.0.0 since I'm rolling RC0 now. On Tue, Nov 14, 2017 at 11:08 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: > Hi folks, > > We've resolved a

Heads up: branching branch-3.0.0 for GA

Hi folks, We've resolved all the blockers for 3.0.0 and the release notes and changelog look good, so I'm going to cut the branch and get started on the RC. * branch-3.0 will advance to 3.0.1-SNAPSHOT * branch-3.0.0 will go to 3.0.0 Please keep this in mind when committing. Cheers, Andrew

Re: [DISCUSS] A final minor release off branch-2?

What are the known gaps that need bridging between 2.x and 3.x? >From an HDFS perspective, we've tested wire compat, rolling upgrade, and rollback. >From a YARN perspective, we've tested wire compat and rolling upgrade. Arun just mentioned an NM rollback issue that I'm not familiar with.

2017-10-31 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+ updates 2017-10-31 Lots of progress towards GA, we look on track for cutting RC0 this week. I ran the versions script to check the branch matches up with JIRA and fixed things up, and also checked that the changelog and

[jira] [Resolved] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked

[ https://issues.apache.org/jira/browse/MAPREDUCE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved MAPREDUCE-6987. Resolution: Duplicate Fix Version/s: (was: 3.1.0

[jira] [Reopened] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked

[ https://issues.apache.org/jira/browse/MAPREDUCE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened MAPREDUCE-6987: > JHS Log Scanner and Cleaner bloc

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

FWIW we've been running branch-3.0 unit tests successfully internally, though we have separate jobs for Common, HDFS, YARN, and MR. The failures here are probably a property of running everything in the same JVM, which I've found problematic in the past due to OOMs. On Tue, Oct 24, 2017 at 4:04

2017-10-20 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-10-20 Apologies for skipping the update last week. Here's how we're tracking for GA. Highlights: - Merge of HDFS router-based federation and API-based scheduler configuration with no reported

Re: 2017-10-06 Hadoop 3 release status update

On Oct 6, 2017, at 1:31 PM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > > > > - Still waiting on Allen to review YARN native services feature. > > Fake news. > > I’m still -1 on it, at least prior to a patch that posted late > yesterday. I’l

2017-10-06 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-10-06 The beta1 RC0 vote passed, and beta1 is out! Now tracking GA features. Highlights: - 3.0.0-beta1 has been released! - Router-based federation merge vote should be about to pass - API-based

Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

M HA/work-preservring-restart/ NM-work-preserving restart > for ATSv2 entities. > - verified all ATSv2 REST end points to retrieve the entities > - ran sample MR jobs and distributed jobs > > Thanks & Regards > Rohith Sharma K S > > On 4 October 2017 at 05:31, Andrew W

Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

Connecting to >> > > https://localhost:14000/logLevel?log=org.apache.hadoop.fs. >> http.server. >> > HttpFSServer >> > >Exception in thread "main" >> > > org.apache.hadoop.security.authentication.client. >>

Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

, Brahma Reddy Battula <bra...@apache.org> wrote: > > Thanks Andrew. > > +1 (non binding) > > --Built from source > --installed 3 node HA cluster > --Verified shell commands and UI > --Ran wordcount/pic jobs > > > > > On Fri, 29 Sep 2017 at 5:34 AM

2017-09-20 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-29 After about a month of slip, RC0 has been sent out for a VOTE. Focus now turns to GA, where we will attempt to keep the original beta1 target date (early November). Highlights: - RC0 vote was sent

Re: [DISCUSS] Merging API-based scheduler configuration to trunk/branch-2

Hi Jonathan, I'm okay with putting this into branch-3.0 for GA if it can be merged within the next two weeks. Even though beta1 has slipped by a month, I want to stick to the targeted GA data of Nov 1st as much as possible. Of course, let's not sacrifice quality or stability for speed; if

[VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

Hi all, Let me start, as always, by thanking the many, many contributors who helped with this release! I've prepared an RC0 for 3.0.0-beta1: http://home.apache.org/~wang/3.0.0-beta1-RC0/ This vote will run five days, ending on Nov 3rd at 5PM Pacific. beta1 contains 576 fixed JIRA issues

Re: Heads up: branching branch-3.0.0-beta1 off of branch-3.0

Branch has been cut, branch-3.0 is now open for commits for 3.0.0 GA. HEAD of branch-3.0.0-beta1 is 2223393ad1d5ffdd62da79e1546de79c6259dc12. On Thu, Sep 28, 2017 at 10:52 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: > Hi folks, > > We've driven the blocker count down t

Heads up: branching branch-3.0.0-beta1 off of branch-3.0

Hi folks, We've driven the blocker count down to 0, and I went through and made sure the fix versions and release notes and so on are all lined up. I'm going to cut branch-3.0.0-beta1 off branch-3.0 and try and get RC0 out today. Cheers, Andrew

2017-09-22 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-22 We've had some late breaking blockers related to Docker support that are delaying the release. We're on a day-by-day slip at this point. Highlights: - I did a successful test create-release

Re: [DISCUSS] moving to Apache Yetus Audience Annotations

's too late to get in for 3.0, I can make a jira and put it > on the back burner for when trunk goes to 4.0? > > On Fri, Sep 22, 2017 at 12:49 PM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > >> Is this itself an incompatible change? I imagine the bytecode will be >

Re: [DISCUSS] moving to Apache Yetus Audience Annotations

Is this itself an incompatible change? I imagine the bytecode will be different. I think we're too late to do this for beta1 given that I want to cut an RC0 today. On Fri, Sep 22, 2017 at 7:03 AM, Sean Busbey wrote: > When Apache Yetus formed, it started with several key

2017-09-19 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-19 Sorry for the late update. We're down to one blocker and one EC must do! Made great progress over the last week and a bit. We will likely cut RC0 this week. Highlights: - Down to just two blocker

Re: [VOTE] Merge yarn-native-services branch into trunk

ks, > Jian > > > > On Sep 7, 2017, at 3:59 PM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > > > > Hi folks, > > > > This vote closes today. I see a -1 from Allen on inclusion in beta1. I > see > > there's active fixing going on, but given

2017-09-07 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-07 Slightly early update since I'll be out tomorrow. We're one week out, and focus is on blocker burndown. Highlights: - 3.1.0 release planning is underway, led by Wangda. Target release date is in

Re: [VOTE] Merge yarn-native-services branch into trunk

Hi folks, This vote closes today. I see a -1 from Allen on inclusion in beta1. I see there's active fixing going on, but given that we're one week out from RC0, I think we should drop this from beta1. Allen, Jian, others, is this reasonable? What release should we retarget this for? I don't have

[jira] [Resolved] (MAPREDUCE-6941) The default setting doesn't work for MapReduce job

[ https://issues.apache.org/jira/browse/MAPREDUCE-6941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved MAPREDUCE-6941. Resolution: Not A Problem I'm going to close this based on Ray's analysis. Junping

2017-09-01 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-01 We're two weeks out from beta1, focus is on blocker burndown. Highlights: - S3Guard merged! - TSv2 alpha2 merged! - branch-3.0 has been cut after discussion on dev lists. Red flags: - 10

Heads up: branch-3.0 has been cut, commit here for 3.0.0-beta1

Hi folks, I've proceeded with the plan from our earlier thread and cut branch-3.0. The branches and maven versions are now set as follows: trunk: 3.1.0-SNAPSHOT branch-3.0: 3.0.0-beta1-SNAPSHOT branch-2's are still the same. This means if you want to commit something for beta1, commit it to

Re: Branch merges and 3.0.0-beta1 scope

Hi Vinod, On Fri, Aug 25, 2017 at 2:42 PM, Vinod Kumar Vavilapalli wrote: > > From a release management perspective, it's *extremely* reasonable to > block the inclusion of new features a month from the planned release date. > A typical software development lifecycle

Re: [DISCUSS] Branches and versions for Hadoop 3

hold commit until you cut the two branches >> and TSv2 get committed. >> >> Thanks, >> Wangda >> >> On Tue, Aug 29, 2017 at 11:25 AM, Andrew Wang <andrew.w...@cloudera.com> >> wrote: >> >> > Hi Wangda, >> > >> > I'll cut

Re: [DISCUSS] Branches and versions for Hadoop 3

TSv2, resource profile, YARN-5734) are > targeted to merge to 3.0-GA but not 3.0-beta1, which branch we should > commit to, and when we can commit? Also, similar to 3.0.0-alpha1 to 4, you > will cut branch-3.0.0-beta1, correct? > > Thanks, > Wangda > > > On Tue, Aug 29, 2017

Re: [DISCUSS] Branches and versions for Hadoop 3

> @Wangda: your resource profile vote ends tomorrow, could we please > coordinate our merges? > > thanks > Vrushali > > > On Mon, Aug 28, 2017 at 10:45 PM, Rohith Sharma K S < > rohithsharm...@apache.org> wrote: > >> On 29 August 2017 at 06:24, Andrew Wang

Re: [DISCUSS] Branches and versions for Hadoop 3

So far I've seen no -1's to the branching proposal, so I plan to execute this tomorrow unless there's further feedback. Regarding the above discussion, I think Jason and I have essentially the same opinion. I hope that keeping trunk a release branch means a higher bar for merges and code review

2017-08-25 Hadoop 3 release status update

Hi all, I've written up a status report for the current state of Hadoop 3 on the wiki. I've also pasted it below for your convenience. Cheers, Andrew https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-08-25 Another month flew by without an update. This is

Re: Branch merges and 3.0.0-beta1 scope

ll continue tests to make sure >> changes are safe. >> >> Discussed with Andrew offline, we decided to not put this to beta1 since >> beta1 is not far away. But we want to put it before GA if sufficient tests >> are done. >> >> Thanks, >> Wangda &g

Re: Branch merges and 3.0.0-beta1 scope

at 10:54 AM, Rohith Sharma K S < > rohithsharm...@apache.org> wrote: > >> On 25 August 2017 at 22:39, Andrew Wang <andrew.w...@cloudera.com> wrote: >> >> > Hi Rohith, >> > >> > Given that we're advertising TSv2 as an alpha feature,

[DISCUSS] Branches and versions for Hadoop 3

Hi folks, With 3.0.0-beta1 fast approaching, I wanted to go over the proposed branching strategy. In the early 2.x days, moving trunk immediately to 3.0.0 was a mistake. branch-2 and trunk were virtually identical, and increased backport complexity. Until we need to make incompatible changes,

Re: Branch merges and 3.0.0-beta1 scope

Hi Jason, I agree with this proposal. I'll start another email thread spelling this out, and gather additional feedback. Best, Andrew On Fri, Aug 25, 2017 at 6:27 AM, Jason Lowe <jl...@oath.com> wrote: > Andrew Wang wrote: > > >> This means I'll cut branch-3 and >>

Re: Branch merges and 3.0.0-beta1 scope

compatibility would be a > matter. > During this gap i.e between beta-GA release, should we maintain > compatibility ? > If my understanding is right then TSv2 have to be merged with beta1 > release. In TSv2 phase-2, we have compatibility changes from phase-1. > > > Thanks &

Re: Branch merges and 3.0.0-beta1 scope

Glad to see the discussion continued in my absence :) >From a release management perspective, it's *extremely* reasonable to block the inclusion of new features a month from the planned release date. A typical software development lifecycle includes weeks of feature freeze and weeks of code

Branch merges and 3.0.0-beta1 scope

Hi folks, As you might have seen, we've had a number of branch merges floated this past week targeted for 3.0.0-beta1, which is planned for about a month from now. In total, I'm currently tracking these branches: YARN-2915: YARN federation (recently merged) HADOOP-13345: S3Guard (currently

Re: [DISCUSS] Merge yarn-native-services branch into trunk

Hi Jian, thanks for the reply, On Thu, Aug 17, 2017 at 1:03 PM, Jian He wrote: > Thanks Andrew for the comments. Answers below: > > - There are no new APIs added in YARN/Hadoop core. In fact, all the new > code are running outside of existing system and they are optional

Re: [DISCUSS] Merge YARN resource profile (YARN-3926) branch into trunk

Hi Wangda, Can this feature be disabled? Is it on or off by default? We're 1 month from the target release for beta1, so I don't want to introduce risk to existing code paths. TSv2 and S3Guard and YARN Federation are all okay in that regard. I'm also not clear on what work is remaining, there

Re: [DISCUSS] Merging YARN-5355 (Timeline Service v.2) to trunk

anks > Vrushali > [link1] https://issues.apache.org/jira/projects/YARN/versions/12337991 > > > On Wed, Aug 16, 2017 at 10:47 AM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > >> Hi Vrushali, >> >> Glad to hear this major dev milestone is nearin

Re: [DISCUSS] Merging YARN-5355 (Timeline Service v.2) to trunk

Hi Vrushali, Glad to hear this major dev milestone is nearing completion! Repeating my request on other merge [DISCUSS] threads, could you comment on testing and API stability of this merge? Our timeline for beta1 is about a month out, so there's not much time to fix things beforehand. Looking

Re: [DISCUSS] Merge yarn-native-services branch into trunk

Hi Jian, Hadoop 3.0.0-beta1 is planned for mid-September. If the plan is to merge in hopefully the next two weeks, that's very, very close to the goal release date. We've already got a pile of blockers and criticals to resolve before then. Could you comment on testing and API stability for this

Re: Are binary artifacts are part of a release?

ASF >> primarily produces and publishes source-code >> > so voting artifacts should be optimized for evaluation of that." >> > >> > Thanks, >> > --Konst >> > >> > On Mon, Jul 31, 2017 at 4:51 PM, Allen Wittenauer < >&g

Are binary artifacts are part of a release?

bits and no src, as you > > guys requested. > > Will keep it as RC0 as there is no source code change and it comes from > the > > same build. > > Hope this is satisfactory. > > > > Thanks, > > --Konstantin > > > > On Mon, Jul 31, 2017 at 1

[jira] [Created] (MAPREDUCE-6924) Revert MAPREDUCE-6199 MAPREDUCE-6286 and MAPREDUCE-5875

Andrew Wang created MAPREDUCE-6924: -- Summary: Revert MAPREDUCE-6199 MAPREDUCE-6286 and MAPREDUCE-5875 Key: MAPREDUCE-6924 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6924 Project: Hadoop Map

[jira] [Resolved] (MAPREDUCE-6924) Revert MAPREDUCE-6199 MAPREDUCE-6286 and MAPREDUCE-5875

[ https://issues.apache.org/jira/browse/MAPREDUCE-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved MAPREDUCE-6924. Resolution: Fixed Fix Version/s: 3.0.0-beta1 Resolving this changelog tracking

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

I agree with Brahma on the two issues flagged (having src in the binary tarball, missing native libs). These are regressions from prior releases. As an aside, "we release binaries as a convenience" doesn't relax the quality bar. The binaries are linked on our website and distributed through

3.0.0-beta1 release plan and branch cuts

Hi all, Here's a long overdue update on Hadoop 3. We've made great progress through the alpha releases, thanks to the hard work of numerous contributors. With alpha4 out the door, it's time to look toward beta1! I updated the wiki page with a proposed release date for beta1: September 15th.

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha4-RC0

YARN-5670. I'm not sure this is a blocker, as > docker containers can still be run, but with degraded functionality. I'm +1 > if others feel this isn't a blocker. > > Thanks, > -Shane > > On Fri, Jun 30, 2017 at 10:29 AM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > >>

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha4-RC0

Thanks for taking a look Steve, On Fri, Jun 30, 2017 at 4:59 AM, Steve Loughran <ste...@hortonworks.com> wrote: > > On 30 Jun 2017, at 03:40, Andrew Wang <andrew.w...@cloudera.com> wrote: > > Hi all, > > As always, thanks to the many, many contributors who he

[VOTE] Release Apache Hadoop 3.0.0-alpha4-RC0

Hi all, As always, thanks to the many, many contributors who helped with this release! I've prepared an RC0 for 3.0.0-alpha4: http://home.apache.org/~wang/3.0.0-alpha4-RC0/ The standard 5-day vote would run until midnight on Tuesday, July 4th. Given that July 4th is a holiday in the US, I

Heads up: branching 3.0.0-alpha4, use -beta1 for new commits to trunk

Hi folks, I'm in the process of moving out all the JIRA versions from 3.0.0-alpha4 to 3.0.0-beta1 in preparation for a 3.0.0-alpha4 release. I'm hoping to get an RC up tomorrow for a vote, with possibly extending it given the holiday week in the US for July 4th. Please use the beta1 target/fix

3.0.0-alpha3 JIRA version has been renamed to 3.0.0-alpha4

Hi all, The Hadoop PMC is reserving the 3.0.0-alpha3 version for an upcoming release. I've renamed the versions already in JIRA. More information on this to come next week. Please use the renamed "3.0.0-alpha4" version for commits to trunk from here on out. Thanks, Andrew

Reminder to always set x.0.0 and x.y.0 fix versions when backporting

Hi folks, I've noticed with the backporting efforts for 2.8.1, we're losing some x.y.0 fix versions (e.g. 2.9.0). Our fix version scheme is described here (also quoted) https://hadoop.apache.org/versioning.html 1. For each *minor* release line, set the *lowest unreleased a.b.c version,

Re: About 2.7.4 Release

Can we wait for 2.7.4 first? There are still backports happening to branch-2.7. After that, there shouldn't be many backports since both 2.8.x and 2.7.x will be up-to-date with what's in 3.0.0-alpha1 and 3.0.0-alpha2. On Tue, May 2, 2017 at 4:07 PM, Allen Wittenauer

Re: About 2.7.4 Release

On Mon, May 1, 2017 at 3:44 PM, Allen Wittenauer <a...@effectivemachines.com> wrote: > > > On May 1, 2017, at 2:27 PM, Andrew Wang <andrew.w...@cloudera.com> > wrote: > > I believe I asked about this on dev-yetus a while back. I'd prefer that > the presence

Re: About 2.7.4 Release

I didn't close JIRAs after the 3.0.0-alpha1 or alpha2 releases since closing makes the JIRAs read-only. This makes it more annoying to backport to older releases and for concurrent releases in general. I believe I asked about this on dev-yetus a while back. I'd prefer that the presence of the fix

Re: Automated documentation build for Apache Hadoop

Nice work Akira! Appreciate the help with trunk development. On Mon, Apr 3, 2017 at 1:56 AM, Akira Ajisaka wrote: > Hi folks, > > I've created a repository to build and push Apache Hadoop document (trunk) > via Travis CI. > https://github.com/aajisaka/hadoop-document > >

Re: [DISCUSS] Changing the default class path for clients

; > > > I agreed with Andrew too. Users have relied for years on `hadoop > classpath` for their script to launch jobs or other tools, perhaps no the > best idea to change the behavior without providing a proper deprecation > path. > > > > thanks! > > esteban. >

Re: Can we update protobuf's version on trunk?

with proto3 syntax specified (had to drop the >> >> > 'optional' >> >> > qualifier -- not allowed in proto3): >> >> > $ more pb_drops_two.proto >> >> > syntax = "proto3"; >> >> > message Test { >> >> > string one = 1; >> >>

Re: Can we update protobuf's version on trunk?

> > > If unknown fields are dropped, then applications proxying tokens and > other > >> data between servers will effectively corrupt those messages, unless we > >> make everything opaque bytes, which- absent the convenient, prenominate > >> semantics managing the conversion- obviate the

Re: Can we update protobuf's version on trunk?

I've been investigating this a bit. I'm hoping Chris can ring in, since he's identified wire compatibility issues. Replying inline to Chris' comment

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has more changes than 2.8.0. It looks like the 2.8.0 bin tarball has an extra 1.5GB of docs when extracted compared to 3.0.0-alpha2. I think it's from the extra src-html content: -> % find share/doc -name src-html | xargs du

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping, inline, >From my understanding, this issue is related to our previous > improvements with separating client and server jars in HDFS-6200. If we use > the new "client" jar in NN HA deployment, then we will hit the issue > reported. > >From my read of the poms, hadoop-client depends on

  1   2   3   >