Re: Bug Bash Post-mortem
Thank you Allen and Barbara for organizing the bug bash and for the post-mortem. I would definitely like to have another one in the fall. From: Allen Wittenauer a...@altiscale.com To: common-dev@hadoop.apache.org common-dev@hadoop.apache.org Cc: mapreduce-...@hadoop.apache.org mapreduce-...@hadoop.apache.org; hdfs-...@hadoop.apache.org hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org yarn-...@hadoop.apache.org Sent: Friday, May 15, 2015 8:05 PM Subject: Bug Bash Post-mortem Hey gang. I promised a post-mortem, but I got distracted with making everyone's lives miserable with new test-patch code. Luckily my coworker Barbara (who some of you met!) wrote some wrap up info about it on our corporate blog, including some interesting stats…. https://www.altiscale.com/hadoop-blog/the-power-of-community-apache-hadoop-global-bug-bash/
[jira] [Created] (HADOOP-11986) Set up a nightly Jenkins build on Solaris.
Chris Nauroth created HADOOP-11986: -- Summary: Set up a nightly Jenkins build on Solaris. Key: HADOOP-11986 URL: https://issues.apache.org/jira/browse/HADOOP-11986 Project: Hadoop Common Issue Type: Sub-task Components: build Reporter: Chris Nauroth HADOOP-11985 proposes improving Solaris support in Hadoop. As part of that, I propose that we set up a nightly Jenkins build on Solaris. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-11985) Improve Solaris support in Hadoop
Alan Burlison created HADOOP-11985: -- Summary: Improve Solaris support in Hadoop Key: HADOOP-11985 URL: https://issues.apache.org/jira/browse/HADOOP-11985 Project: Hadoop Common Issue Type: New Feature Components: build, conf Affects Versions: 2.7.0 Environment: Solaris x86, Solaris sparc Reporter: Alan Burlison Assignee: Alan Burlison At present the Hadoop native components aren't fully supported on Solaris primarily due to differences between Linux and Solaris. This top-level task will be used to group together both existing and new issues related to this work. A second goal is to improve Hadoop performance on Solaris wherever possible. Steve Loughran suggested a top-level JIRA was the best way to manage the work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: JNIFlags.cmake versus FindJNI.cmake
On 16/05/2015 03:24, Mai Haohui wrote: Just checked the repo of cmake and it turns out that FindJNI.cmake is available even before cmake 2.4. I think it makes sense to file a bug to replace it to the standard cmake module. Can you please file a jira for this? Will do, thanks. -- Alan Burlison --
Build failed in Jenkins: Hadoop-Common-trunk #1497
See https://builds.apache.org/job/Hadoop-Common-trunk/1497/changes Changes: [junping_du] YARN-3505 addendum: fix an issue in previous patch. [jlowe] YARN-2421. RM still allocates containers to an app in the FINISHING state. Contributed by Chang Li [jing9] HDFS-8397. Refactor the error handling code in DataStreamer. Contributed by Tsz Wo Nicholas Sze. [wheat9] HDFS-8394. Move getAdditionalBlock() and related functionalities into a separate class. Contributed by Haohui Mai. [wheat9] HDFS-8403. Eliminate retries in TestFileCreation#testOverwriteOpenForWrite. Contributed by Arpit Agarwal. [xgong] YARN-3526. ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster. Contributed by Weiwei Yang -- [...truncated 5175 lines...] Running org.apache.hadoop.util.TestFileBasedIPList Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.175 sec - in org.apache.hadoop.util.TestFileBasedIPList Running org.apache.hadoop.util.TestGenericsUtil Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.341 sec - in org.apache.hadoop.util.TestGenericsUtil Running org.apache.hadoop.util.TestFindClass Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.849 sec - in org.apache.hadoop.util.TestFindClass Running org.apache.hadoop.util.TestLightWeightGSet Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.174 sec - in org.apache.hadoop.util.TestLightWeightGSet Running org.apache.hadoop.util.TestPureJavaCrc32 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.59 sec - in org.apache.hadoop.util.TestPureJavaCrc32 Running org.apache.hadoop.util.TestStringUtils Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.382 sec - in org.apache.hadoop.util.TestStringUtils Running org.apache.hadoop.util.TestProtoUtil Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.272 sec - in org.apache.hadoop.util.TestProtoUtil Running org.apache.hadoop.util.TestSignalLogger Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.187 sec - in org.apache.hadoop.util.TestSignalLogger Running org.apache.hadoop.util.TestDiskChecker Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.168 sec - in org.apache.hadoop.util.TestDiskChecker Running org.apache.hadoop.util.TestShutdownThreadsHelper Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.171 sec - in org.apache.hadoop.util.TestShutdownThreadsHelper Running org.apache.hadoop.util.TestCacheableIPList Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.278 sec - in org.apache.hadoop.util.TestCacheableIPList Running org.apache.hadoop.util.TestLineReader Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.216 sec - in org.apache.hadoop.util.TestLineReader Running org.apache.hadoop.util.TestIdentityHashStore Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.199 sec - in org.apache.hadoop.util.TestIdentityHashStore Running org.apache.hadoop.util.TestClasspath Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.348 sec - in org.apache.hadoop.util.TestClasspath Running org.apache.hadoop.util.TestApplicationClassLoader Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.243 sec - in org.apache.hadoop.util.TestApplicationClassLoader Running org.apache.hadoop.util.TestShell Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.233 sec - in org.apache.hadoop.util.TestShell Running org.apache.hadoop.util.TestShutdownHookManager Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.167 sec - in org.apache.hadoop.util.TestShutdownHookManager Running org.apache.hadoop.util.TestHttpExceptionUtils Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.672 sec - in org.apache.hadoop.util.TestHttpExceptionUtils Running org.apache.hadoop.util.TestJarFinder Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.741 sec - in org.apache.hadoop.util.TestJarFinder Running org.apache.hadoop.util.hash.TestHash Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.43 sec - in org.apache.hadoop.util.hash.TestHash Running org.apache.hadoop.util.TestLightWeightCache Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.278 sec - in org.apache.hadoop.util.TestLightWeightCache Running org.apache.hadoop.util.TestNativeCodeLoader Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.167 sec - in org.apache.hadoop.util.TestNativeCodeLoader Running org.apache.hadoop.util.TestReflectionUtils Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.643 sec - in org.apache.hadoop.util.TestReflectionUtils Running org.apache.hadoop.crypto.TestCryptoStreams Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.65 sec - in org.apache.hadoop.crypto.TestCryptoStreams Running org.apache.hadoop.crypto.TestCryptoStreamsForLocalFS Tests run: 14, Failures: 0, Errors:
Build failed in Jenkins: Hadoop-common-trunk-Java8 #198
See https://builds.apache.org/job/Hadoop-common-trunk-Java8/198/changes Changes: [junping_du] YARN-3505 addendum: fix an issue in previous patch. [jlowe] YARN-2421. RM still allocates containers to an app in the FINISHING state. Contributed by Chang Li [jing9] HDFS-8397. Refactor the error handling code in DataStreamer. Contributed by Tsz Wo Nicholas Sze. [wheat9] HDFS-8394. Move getAdditionalBlock() and related functionalities into a separate class. Contributed by Haohui Mai. [wheat9] HDFS-8403. Eliminate retries in TestFileCreation#testOverwriteOpenForWrite. Contributed by Arpit Agarwal. [xgong] YARN-3526. ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster. Contributed by Weiwei Yang -- [...truncated 5632 lines...] Running org.apache.hadoop.io.TestIOUtils Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.399 sec - in org.apache.hadoop.io.TestIOUtils Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestTextNonUTF8 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.167 sec - in org.apache.hadoop.io.TestTextNonUTF8 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestMD5Hash Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.169 sec - in org.apache.hadoop.io.TestMD5Hash Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestMapFile Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.097 sec - in org.apache.hadoop.io.TestMapFile Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestWritableName Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.176 sec - in org.apache.hadoop.io.TestWritableName Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestSortedMapWritable Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.208 sec - in org.apache.hadoop.io.TestSortedMapWritable Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestSequenceFileSync Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.711 sec - in org.apache.hadoop.io.TestSequenceFileSync Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.retry.TestFailoverProxy Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.354 sec - in org.apache.hadoop.io.retry.TestFailoverProxy Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.retry.TestRetryProxy Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.21 sec - in org.apache.hadoop.io.retry.TestRetryProxy Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestDefaultStringifier Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.337 sec - in org.apache.hadoop.io.TestDefaultStringifier Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestBloomMapFile Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.337 sec - in org.apache.hadoop.io.TestBloomMapFile Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestBytesWritable Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.197 sec - in org.apache.hadoop.io.TestBytesWritable Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestWritableUtils Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.156 sec - in org.apache.hadoop.io.TestWritableUtils Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestBooleanWritable Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.171 sec - in org.apache.hadoop.io.TestBooleanWritable Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestDataByteBuffers Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.348 sec - in org.apache.hadoop.io.TestDataByteBuffers Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.io.TestVersionedWritable Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.15
[jira] [Resolved] (HADOOP-11972) hdfs dfs -copyFromLocal reports File Not Found instead of Permission Denied.
[ https://issues.apache.org/jira/browse/HADOOP-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved HADOOP-11972. --- Resolution: Invalid This is because '.' is translated to /user/hbase for user hbase, and /user/hrt_qa for hrt_qa. If you think this is not the case, please reopen and tell us 1. If your environment is Kerberized? 2. Are you using NFS? 3. What happens when you do the same thing using HDFS CLI hdfs dfs -copyFromLocal reports File Not Found instead of Permission Denied. Key: HADOOP-11972 URL: https://issues.apache.org/jira/browse/HADOOP-11972 Project: Hadoop Common Issue Type: Bug Components: tools Affects Versions: 2.6.0 Environment: Linux hadoop-8309-2.west.isilon.com 2.6.32-504.16.2.el6.centos.plus.x86_64 #1 SMP Wed Apr 22 00:59:31 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux Reporter: David Tucker userA creates a file in /home/userA with 700 permissions. userB tries to copy it to HDFS, and receives a No such file or directory instead of Permission denied. [hrt_qa@hadoop-8309-2 ~]$ touch ./foo [hrt_qa@hadoop-8309-2 ~]$ ls -l ./foo -rw-r--r--. 1 hrt_qa users 0 May 14 16:09 ./foo [hrt_qa@hadoop-8309-2 ~]$ sudo su hbase [hbase@hadoop-8309-2 hrt_qa]$ ls -l ./foo ls: cannot access ./foo: Permission denied [hbase@hadoop-8309-2 hrt_qa]$ hdfs dfs -copyFromLocal ./foo /tmp/foo copyFromLocal: `./foo': No such file or directory -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-11988) Fix typo in the document for hadoop fs -find
Akira AJISAKA created HADOOP-11988:
--
Summary: Fix typo in the document for hadoop fs -find
Key: HADOOP-11988
URL: https://issues.apache.org/jira/browse/HADOOP-11988
Project: Hadoop Common
Issue Type: Bug
Components: documentation
Affects Versions: 2.7.0
Reporter: Akira AJISAKA
Priority: Trivial
{code}
* -printbr /-print0Always
{code}
-print0Always should be -print0 and Always should be moved to the
description below.
{code}
evaluates to true. Causes the current pathname to ...
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11987) JNI build should use default cmake FindJNI.cmake
Alan Burlison created HADOOP-11987: -- Summary: JNI build should use default cmake FindJNI.cmake Key: HADOOP-11987 URL: https://issues.apache.org/jira/browse/HADOOP-11987 Project: Hadoop Common Issue Type: Sub-task Components: native Affects Versions: 2.7.0 Environment: All Reporter: Alan Burlison Assignee: Alan Burlison Priority: Minor From http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201505.mbox/%3C55568DAC.1040303%40oracle.com%3E -- Why does hadoop-common-project/hadoop-common/src/CMakeLists.txt use JNIFlags.cmake in the same directory to set things up for JNI compilation rather than FindJNI.cmake, which comes as a standard cmake module? The checks in JNIFlags.cmake make several assumptions that I believe are only correct on Linux whereas I'd expect FindJNI.cmake to be more platform-independent. -- Just checked the repo of cmake and it turns out that FindJNI.cmake is available even before cmake 2.4. I think it makes sense to file a bug to replace it to the standard cmake module. Can you please file a jira for this? -- This also applies to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/JNIFlags.cmake -- This message was sent by Atlassian JIRA (v6.3.4#6332)
