Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Hi folks, Thanks again for the testing help with the RC. Here's our dashboard for the 3.0.0 release: https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12329849 Right now we're tracking three blockers: * HADOOP-15058 is the create-release fix, I just put up a patch which needs re

Re: Apache Hadoop 2.8.3 Release Plan

Agree with Konstantin. This two issues has been opened for a while but could not reach a consensus on the fix, hope this gets enough attention from the community and get them resolved. Thanks -- Weiwei On 22 Nov 2017, 11:18 AM +0800, Konstantin Shvachko , wrote: I would consider these two blo

Re: Apache Hadoop 2.8.3 Release Plan

I would consider these two blockers for 2.8.3 as they crash NN: https://issues.apache.org/jira/browse/HDFS-12638 https://issues.apache.org/jira/browse/HDFS-12832 Thanks, --Konstantin On Tue, Nov 21, 2017 at 11:16 AM, Junping Du wrote: > Thanks Andrew and Wangda for comments! > > To me, an impr

[jira] [Created] (YARN-7554) TestCryptoStreamsWithOpensslAesCtrCryptoCodec fails on Debian 9

Miklos Szegedi created YARN-7554: Summary: TestCryptoStreamsWithOpensslAesCtrCryptoCodec fails on Debian 9 Key: YARN-7554 URL: https://issues.apache.org/jira/browse/YARN-7554 Project: Hadoop YARN

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/600/ [Nov 20, 2017 3:37:04 PM] (billie) YARN-7529. TestYarnNativeServices#testRecoverComponentsAfterRMRestart() [Nov 20, 2017 6:28:12 PM] (manojpec) HDFS-12730. Verify open files captured in the snapshots acros

[jira] [Created] (YARN-7553) TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky

Miklos Szegedi created YARN-7553: Summary: TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky Key: YARN-7553 URL: https://issues.apache.org/jira/browse/YARN-7553 Project: Hadoop YARN

[jira] [Resolved] (YARN-7551) yarn.resourcemanager.reservation-system.max-periodicity is not in yarn-default.xml

[ https://issues.apache.org/jira/browse/YARN-7551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved YARN-7551. Resolution: Not A Problem OK. > yarn.resourcemanager.reservation-system.max-periodicity is n

[jira] [Created] (YARN-7552) RM REST containers endpoints are not documented

Daniel Templeton created YARN-7552: -- Summary: RM REST containers endpoints are not documented Key: YARN-7552 URL: https://issues.apache.org/jira/browse/YARN-7552 Project: Hadoop YARN Issue T

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Hi Vinod, bq. (b) We need to figure out if this V1 TimelineService should even be support given ATSv2. Yes, I am following this discussion. Let me chat with Rohith and Varun about this and we will respond on this thread. As such, my preliminary thoughts are that we should continue to support Time

[jira] [Created] (YARN-7551) yarn.resourcemanager.reservation-system.max-periodicity is not in yarn-default.xml

Daniel Templeton created YARN-7551: -- Summary: yarn.resourcemanager.reservation-system.max-periodicity is not in yarn-default.xml Key: YARN-7551 URL: https://issues.apache.org/jira/browse/YARN-7551 Pr

Re: [DISCUSS] A final minor release off branch-2?

Hi Andrew, bq. Source and binary compatibility are not required for 3.0.0. It's a new major release, and there are known, documented incompatibilities in this regard. Technically, it is true. However, in practically, we should retain compatibility as much as we can. Otherwise, we could break do

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

>> - $HADOOP_YARN_HOME/sbin/yarn-daemon.sh start historyserver doesn't even >> work. Not just deprecated in favor of timelineserver as was advertised. > > This works for me in trunk and the bash code doesn’t appear to have > changed in a very long time. Probably something local to your in

Re:

Just filed HADOOP-15059 to track this. Thanks, Junping From: Junping Du Sent: Tuesday, November 21, 2017 1:09 PM To: Vinod Kumar Vavilapalli; Allen Wittenauer Cc: common-...@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org; hdfs-

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Another issue I just found is we have token format compatibility issue between 2.x and 3.0. I tried to run a simple MR job on 3.0 RC0 against with 2.9.0 tarball which is failed. This incompatibility change should also break other applications which will break rolling upgrade. I think it is anoth

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

>>> - Cannot enable new UI in YARN because it is under a non-default >>> compilation flag. It should be on by default. >>> >> >> The yarn-ui profile has always been off by default, AFAIK. It's documented >> to turn it on in BUILDING.txt for release builds, and we do it in >> create-release. >> >

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

>> - One decommissioned node in YARN ResourceManager UI always appears to >> start with, even when there are no NodeManagers that are started yet: >> Info :-1, DECOMMISSIONED, null rack. It shows up only in the UI though, >> not in the CLI node -list >> > > Is this a blocker? Could we get a JIRA?

[jira] [Created] (YARN-7550) Allow YARN HA to be fault tolerant on missing DNS entries

Istvan Vajnorak created YARN-7550: - Summary: Allow YARN HA to be fault tolerant on missing DNS entries Key: YARN-7550 URL: https://issues.apache.org/jira/browse/YARN-7550 Project: Hadoop YARN

Re: Apache Hadoop 2.8.3 Release Plan

Thanks Andrew and Wangda for comments! To me, an improvement with 17 patches is not a big problem as this is self-contained and I didn't see a single line of delete/update on existing code - well, arguably, patches with only adding code can also have big impact but not likely the case here. Wh

Re: Apache Hadoop 2.8.3 Release Plan

Thanks Junping for driving this. For the bug fix vs. improvement, it is actually very hard to define, improvement could be self-contained and useful, bug fix could be dangerous in some cases. To me, If an improvement fixed some existing use case, and the fix is self-contained. I will be open to br

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

On Mon, Nov 20, 2017 at 11:33 PM, Allen Wittenauer wrote: > > The original release script and instructions broke the build up > into three or so steps. When I rewrote it, I kept that same model. It’s > probably time to re-think that. In particular, it should probably be one > big step th

[jira] [Created] (YARN-7549) RM cluster scheduler API does not show resource types usage

Grant Sohn created YARN-7549: Summary: RM cluster scheduler API does not show resource types usage Key: YARN-7549 URL: https://issues.apache.org/jira/browse/YARN-7549 Project: Hadoop YARN Issue

Re: Apache Hadoop 2.8.3 Release Plan

The Aliyun OSS code isn't a small improvement. If you look at Sammi's comment , it's a 17 patch series that is being backported in one

[jira] [Created] (YARN-7548) TestCapacityOverTimePolicy.testAllocation is flaky

Haibo Chen created YARN-7548: Summary: TestCapacityOverTimePolicy.testAllocation is flaky Key: YARN-7548 URL: https://issues.apache.org/jira/browse/YARN-7548 Project: Hadoop YARN Issue Type: Bug

Re: Apache Hadoop 2.8.3 Release Plan

Doh.  Mailer dropped some of the lists.  Replying again to avoid fragmenting the discussion... Still +1 to Andrew's comments. Daniel On 11/21/17 7:53 AM, Daniel Templeton wrote: +1 Daniel On 11/20/17 10:22 PM, Andrew Wang wrote: I'm against including new features in maintenance releases, si

[jira] [Created] (YARN-7547) Throttle Localization for Opportunistic Containers in the NM

Arun Suresh created YARN-7547: - Summary: Throttle Localization for Opportunistic Containers in the NM Key: YARN-7547 URL: https://issues.apache.org/jira/browse/YARN-7547 Project: Hadoop YARN Iss

Re: Apache Hadoop 2.8.3 Release Plan

Thanks Kai for calling out this feature/improvement for attention and Andrew for comments. While I agree that maintenance release should focus on important bug fix only, I doubt we have strict rules to disallow any features/improvements to land on maint release especially when those are small

[jira] [Created] (YARN-7546) In queue page node details are hidden on click

Vasudevan Skm created YARN-7546: --- Summary: In queue page node details are hidden on click Key: YARN-7546 URL: https://issues.apache.org/jira/browse/YARN-7546 Project: Hadoop YARN Issue Type: B