Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-07-31 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/480/ [Jul 31, 2017 2:09:13 AM] (aajisaka) YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization [Jul 31, 2017 5:08:30 AM] (aajisaka) HADOOP-14690. RetryInvocationHandler should override toStri

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Masatake Iwasaki
Thanks for putting this up, Konstantin. I'm +1(non-binding) on the source tarball. * verified signature and mds, * built from source on CentOS 7 and OpenJDK 8 with native profile, * built RPMs by bigtop, deployed 5 nodes cluster by docker provisioner, ran smoke-tests of hdfs, yarn and mapreduce

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Konstantin Shvachko
Hi Brahma Reddy Battula, Formally Apache releases sources. We provide binaries as a reference for convenience. The release instructions for Hadoop 2.7 line at https://wiki.apache.org/hadoop/HowToReleasePreDSBCR don't give much guidance on how to actually build and package the binary tarball. I int

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Konstantin Shvachko
Hi John, Thank you for checking and voting. As far as I know test failures on 2.7.4 are intermittent. We have a jira for that https://issues.apache.org/jira/browse/HDFS-11985 but decided it should not block the release. The "dr,who" thing is a configuration issue. This page may be helpful: http://

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Allen Wittenauer
> On Jul 31, 2017, at 11:20 AM, Konstantin Shvachko > wrote: > > https://wiki.apache.org/hadoop/HowToReleasePreDSBCR FYI: If you are using ASF Jenkins to create an ASF release artifact, it's pretty much an automatic vote failure as any such release is in violation of

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Konstantin Shvachko
To avoid any confusion in this regard. I built RC0 manually in compliance with Apache release policy http://www.apache.org/legal/release-policy.html I edited the HowToReleasePreDSBCR page to make sure people don't use Jenkins option for building. A side note. This particular build is broken anyway

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Andrew Wang
I agree with Brahma on the two issues flagged (having src in the binary tarball, missing native libs). These are regressions from prior releases. As an aside, "we release binaries as a convenience" doesn't relax the quality bar. The binaries are linked on our website and distributed through offici

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread John Zhuge
Just confirmed that HADOOP-13707 does fix the NN servlet issuet in branch-2.7. On Mon, Jul 31, 2017 at 11:38 AM, Konstantin Shvachko wrote: > Hi John, > > Thank you for checking and voting. > As far as I know test failures on 2.7.4 are intermittent. We have a jira > for that > https://issues.apa

[jira] [Created] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829

2017-07-31 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created MAPREDUCE-6922: - Summary: MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829 Key: MAPREDUCE-6922 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922

[jira] [Resolved] (MAPREDUCE-6288) mapred job -status fails with AccessControlException

2017-07-31 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du resolved MAPREDUCE-6288. --- Resolution: Done I have revert MAPREDUCE-6286, MAPREDUCE-6199 and MAPREDUCE-5875 from trun

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Hanisha Koneru
Hi Konstantin, Thanks for preparing the 2.7.4-RC0 release. - Built from source on Mac OS X 10.11.6 with Java 1.7.0_79 - Deployed both binary and built source to a pseudo cluster - Passed the following sanity checks - Basic dfs operations - Wordcount +1 (non-binding) Thanks, Hanisha

[jira] [Created] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

2017-07-31 Thread Robert Schmidtke (JIRA)
Robert Schmidtke created MAPREDUCE-6923: --- Summary: YARN Shuffle I/O for small partitions Key: MAPREDUCE-6923 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6923 Project: Hadoop Map/Reduce

[jira] [Resolved] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.

2017-07-31 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du resolved MAPREDUCE-6286. --- Resolution: Won't Fix MAPREDUCE-5875 is revert, so patch here is not needed. reopen and re

[jira] [Reopened] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.

2017-07-31 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du reopened MAPREDUCE-6286: --- > A typo in HistoryViewer makes some code useless, which causes counter limits > are not rese

[jira] [Reopened] (MAPREDUCE-6199) AbstractCounters are not reset completely on deserialization

2017-07-31 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du reopened MAPREDUCE-6199: --- > AbstractCounters are not reset completely on deserialization > -

[jira] [Resolved] (MAPREDUCE-6199) AbstractCounters are not reset completely on deserialization

2017-07-31 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du resolved MAPREDUCE-6199. --- Resolution: Won't Fix > AbstractCounters are not reset completely on deserialization > ---

[jira] [Resolved] (MAPREDUCE-6924) Revert MAPREDUCE-6199 MAPREDUCE-6286 and MAPREDUCE-5875

2017-07-31 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved MAPREDUCE-6924. Resolution: Fixed Fix Version/s: 3.0.0-beta1 Resolving this changelog tracking JI

[jira] [Created] (MAPREDUCE-6924) Revert MAPREDUCE-6199 MAPREDUCE-6286 and MAPREDUCE-5875

2017-07-31 Thread Andrew Wang (JIRA)
Andrew Wang created MAPREDUCE-6924: -- Summary: Revert MAPREDUCE-6199 MAPREDUCE-6286 and MAPREDUCE-5875 Key: MAPREDUCE-6924 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6924 Project: Hadoop Map/

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Chen Liang
Hi Konstantin, Thanks for putting up 2.7.4-RC0 release! +1 (non-binding). With following checks: - build the source on MacOS 10.12.5 with java version 1.8.0_91. - deployed on a single node pseudo-custer - tested HDFS basics: write/read/append/acl - mapreduce job wordcount/bbp/grep Thanks! Chen

[jira] [Created] (MAPREDUCE-6925) CLONE - Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild

2017-07-31 Thread Junping Du (JIRA)
Junping Du created MAPREDUCE-6925: - Summary: CLONE - Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild Key: MAPREDUCE-6925 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6925

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Konstantin Shvachko
Maven dependencies should be fine. For the packaging, here is the exact phrasing from the sited release-policy document relevant to binaries: "As a convenience to users that might not have the appropriate tools to build a compiled version of the source, binary/bytecode packages MAY be distributed

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Chris Douglas
On Mon, Jul 31, 2017 at 3:02 PM, Konstantin Shvachko wrote: > For the packaging, here is the exact phrasing from the sited release-policy > document relevant to binaries: > "As a convenience to users that might not have the appropriate tools to > build a compiled version of the source, binary/byte

Are binary artifacts are part of a release?

2017-07-31 Thread Andrew Wang
Forking this off to not distract from release activities. I filed https://issues.apache.org/jira/browse/LEGAL-323 to get clarity on the matter. I read the entire webpage, and it could be improved one way or the other. Best, Andrew On Mon, Jul 31, 2017 at 3:56 PM, Chris Douglas wrote: > On Mon,

Re: Are binary artifacts are part of a release?

2017-07-31 Thread Allen Wittenauer
> On Jul 31, 2017, at 4:18 PM, Andrew Wang wrote: > > Forking this off to not distract from release activities. > > I filed https://issues.apache.org/jira/browse/LEGAL-323 to get clarity on the > matter. I read the entire webpage, and it could be improved one way or the > other. IAN

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-07-31 Thread Konstantin Shvachko
Uploaded new binaries hadoop-2.7.4-RC0.tar.gz, which adds lib/native/. Same place: http://home.apache.org/~shv/hadoop-2.7.4-RC0/ Thanks, --Konstantin On Mon, Jul 31, 2017 at 3:56 PM, Chris Douglas wrote: > On Mon, Jul 31, 2017 at 3:02 PM, Konstantin Shvachko > wrote: > > For the packaging, her

Re: Are binary artifacts are part of a release?

2017-07-31 Thread Konstantin Shvachko
The issue was discussed on several occasions in the past. Took me a while to dig this out as an example: http://mail-archives.apache.org/mod_mbox/hadoop-general/20.mbox/%3C4EB0827C.6040204%40apache.org%3E Doug Cutting: "Folks should not primarily evaluate binaries when voting. The ASF primaril

Re: Are binary artifacts are part of a release?

2017-07-31 Thread Allen Wittenauer
... that doesn't contradict anything I said. > On Jul 31, 2017, at 7:23 PM, Konstantin Shvachko wrote: > > The issue was discussed on several occasions in the past. > Took me a while to dig this out as an example: > http://mail-archives.apache.org/mod_mbox/hadoop-general/20.mbox/%3C4EB082

Re: Are binary artifacts are part of a release?

2017-07-31 Thread Konstantin Shvachko
It does not. Just adding historical references, as Andrew raised the question. On Mon, Jul 31, 2017 at 7:38 PM, Allen Wittenauer wrote: > > ... that doesn't contradict anything I said. > > > On Jul 31, 2017, at 7:23 PM, Konstantin Shvachko > wrote: > > > > The issue was discussed on several occ