Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Hi folks, Thanks again for the testing help with the RC. Here's our dashboard for the 3.0.0 release: https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12329849 Right now we're tracking three blockers: * HADOOP-15058 is the create-release fix, I just put up a patch which needs

Re: Apache Hadoop 2.8.3 Release Plan

Agree with Konstantin. This two issues has been opened for a while but could not reach a consensus on the fix, hope this gets enough attention from the community and get them resolved. Thanks -- Weiwei On 22 Nov 2017, 11:18 AM +0800, Konstantin Shvachko , wrote: I would

Re: Apache Hadoop 2.8.3 Release Plan

I would consider these two blockers for 2.8.3 as they crash NN: https://issues.apache.org/jira/browse/HDFS-12638 https://issues.apache.org/jira/browse/HDFS-12832 Thanks, --Konstantin On Tue, Nov 21, 2017 at 11:16 AM, Junping Du wrote: > Thanks Andrew and Wangda for

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/600/ [Nov 20, 2017 3:37:04 PM] (billie) YARN-7529. TestYarnNativeServices#testRecoverComponentsAfterRMRestart() [Nov 20, 2017 6:28:12 PM] (manojpec) HDFS-12730. Verify open files captured in the snapshots

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Hi Vinod, bq. (b) We need to figure out if this V1 TimelineService should even be support given ATSv2. Yes, I am following this discussion. Let me chat with Rohith and Varun about this and we will respond on this thread. As such, my preliminary thoughts are that we should continue to support

Re: [DISCUSS] A final minor release off branch-2?

Hi Andrew, bq. Source and binary compatibility are not required for 3.0.0. It's a new major release, and there are known, documented incompatibilities in this regard. Technically, it is true. However, in practically, we should retain compatibility as much as we can. Otherwise, we could break

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

>> - $HADOOP_YARN_HOME/sbin/yarn-daemon.sh start historyserver doesn't even >> work. Not just deprecated in favor of timelineserver as was advertised. > > This works for me in trunk and the bash code doesn’t appear to have > changed in a very long time. Probably something local to your

Re:

Just filed HADOOP-15059 to track this. Thanks, Junping From: Junping Du Sent: Tuesday, November 21, 2017 1:09 PM To: Vinod Kumar Vavilapalli; Allen Wittenauer Cc: common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Another issue I just found is we have token format compatibility issue between 2.x and 3.0. I tried to run a simple MR job on 3.0 RC0 against with 2.9.0 tarball which is failed. This incompatibility change should also break other applications which will break rolling upgrade. I think it is

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

>>> - Cannot enable new UI in YARN because it is under a non-default >>> compilation flag. It should be on by default. >>> >> >> The yarn-ui profile has always been off by default, AFAIK. It's documented >> to turn it on in BUILDING.txt for release builds, and we do it in >> create-release. >>

[jira] [Created] (MAPREDUCE-7012) 3.0 deployment cannot work with old version MR tar ball which break rolling upgrade

Junping Du created MAPREDUCE-7012: - Summary: 3.0 deployment cannot work with old version MR tar ball which break rolling upgrade Key: MAPREDUCE-7012 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7012

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

>> - One decommissioned node in YARN ResourceManager UI always appears to >> start with, even when there are no NodeManagers that are started yet: >> Info :-1, DECOMMISSIONED, null rack. It shows up only in the UI though, >> not in the CLI node -list >> > > Is this a blocker? Could we get a

Re: Apache Hadoop 2.8.3 Release Plan

Thanks Andrew and Wangda for comments! To me, an improvement with 17 patches is not a big problem as this is self-contained and I didn't see a single line of delete/update on existing code - well, arguably, patches with only adding code can also have big impact but not likely the case here.

Re: Apache Hadoop 2.8.3 Release Plan

Thanks Junping for driving this. For the bug fix vs. improvement, it is actually very hard to define, improvement could be self-contained and useful, bug fix could be dangerous in some cases. To me, If an improvement fixed some existing use case, and the fix is self-contained. I will be open to

Re: Apache Hadoop 2.8.3 Release Plan

The Aliyun OSS code isn't a small improvement. If you look at Sammi's comment , it's a 17 patch series that is being backported in one

Re: Apache Hadoop 2.8.3 Release Plan

Doh.  Mailer dropped some of the lists.  Replying again to avoid fragmenting the discussion... Still +1 to Andrew's comments. Daniel On 11/21/17 7:53 AM, Daniel Templeton wrote: +1 Daniel On 11/20/17 10:22 PM, Andrew Wang wrote: I'm against including new features in maintenance releases,

Re: Apache Hadoop 2.8.3 Release Plan

+1 Daniel On 11/20/17 10:22 PM, Andrew Wang wrote: I'm against including new features in maintenance releases, since they're meant to be bug-fix only. If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning

Re: Apache Hadoop 2.8.3 Release Plan

Thanks Kai for calling out this feature/improvement for attention and Andrew for comments. While I agree that maintenance release should focus on important bug fix only, I doubt we have strict rules to disallow any features/improvements to land on maint release especially when those are small