Build failed in Jenkins: Hadoop-Common-0.23-Build #1105
See https://builds.apache.org/job/Hadoop-Common-0.23-Build/1105/ -- [...truncated 8263 lines...] Running org.apache.hadoop.io.TestBloomMapFile Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.871 sec Running org.apache.hadoop.io.TestObjectWritableProtos Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.322 sec Running org.apache.hadoop.io.TestTextNonUTF8 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046 sec Running org.apache.hadoop.io.nativeio.TestNativeIO Tests run: 9, Failures: 0, Errors: 0, Skipped: 9, Time elapsed: 0.159 sec Running org.apache.hadoop.io.TestSortedMapWritable Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.195 sec Running org.apache.hadoop.io.TestMapFile Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.634 sec Running org.apache.hadoop.io.TestUTF8 Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.447 sec Running org.apache.hadoop.io.TestBoundedByteArrayOutputStream Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.041 sec Running org.apache.hadoop.io.retry.TestRetryProxy Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.205 sec Running org.apache.hadoop.io.retry.TestFailoverProxy Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.192 sec Running org.apache.hadoop.io.TestSetFile Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.134 sec Running org.apache.hadoop.io.serializer.TestWritableSerialization Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.317 sec Running org.apache.hadoop.io.serializer.TestSerializationFactory Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.277 sec Running org.apache.hadoop.io.serializer.avro.TestAvroSerialization Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.536 sec Running org.apache.hadoop.util.TestGenericOptionsParser Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.685 sec Running org.apache.hadoop.util.TestReflectionUtils Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.509 sec Running org.apache.hadoop.util.TestJarFinder Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.694 sec Running org.apache.hadoop.util.TestPureJavaCrc32 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.287 sec Running org.apache.hadoop.util.TestHostsFileReader Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.183 sec Running org.apache.hadoop.util.TestShutdownHookManager Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.146 sec Running org.apache.hadoop.util.TestDiskChecker Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.485 sec Running org.apache.hadoop.util.TestStringUtils Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.133 sec Running org.apache.hadoop.util.TestGenericsUtil Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.258 sec Running org.apache.hadoop.util.TestAsyncDiskService Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.126 sec Running org.apache.hadoop.util.TestProtoUtil Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec Running org.apache.hadoop.util.TestDataChecksum Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.182 sec Running org.apache.hadoop.util.TestRunJar Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.126 sec Running org.apache.hadoop.util.TestOptions Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.079 sec Running org.apache.hadoop.util.TestShell Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.196 sec Running org.apache.hadoop.util.TestIndexedSort Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.569 sec Running org.apache.hadoop.util.TestStringInterner Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.112 sec Running org.apache.hadoop.record.TestRecordVersioning Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.139 sec Running org.apache.hadoop.record.TestBuffer Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.055 sec Running org.apache.hadoop.record.TestRecordIO Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.159 sec Running org.apache.hadoop.security.TestGroupFallback Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.429 sec Running org.apache.hadoop.security.TestGroupsCaching Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.276 sec Running org.apache.hadoop.security.TestProxyUserFromEnv Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.359 sec Running org.apache.hadoop.security.TestUserGroupInformation Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.669 sec Running org.apache.hadoop.security.TestJNIGroupsMapping Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.141 sec
[jira] [Created] (HADOOP-11208) Replace daemon with better name in scripts like hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
Yongjun Zhang created HADOOP-11208: -- Summary: Replace daemon with better name in scripts like hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs Key: HADOOP-11208 URL: https://issues.apache.org/jira/browse/HADOOP-11208 Project: Hadoop Common Issue Type: Improvement Reporter: Yongjun Zhang Per discussion in HDFS-7204, creating this jira. Thanks [~aw] for the work on HDFS-7204. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-11209) Configuration is not thread-safe
Josh Rosen created HADOOP-11209:
---
Summary: Configuration is not thread-safe
Key: HADOOP-11209
URL: https://issues.apache.org/jira/browse/HADOOP-11209
Project: Hadoop Common
Issue Type: Bug
Components: conf
Reporter: Josh Rosen
{{Configuration}} objects are not fully thread-safe, which causes problems in
multi-threaded frameworks like Spark that use these configurations to interact
with existing Hadoop APIs (such as InputFormats).
SPARK-2546 is an example of a problem caused by this lack of thread-safety. In
that bug, multiple concurrent modifications of the same Configuration (in
third-party code) caused an infinite loop because Configuration's internal
{{java.util.HashMap}} is not thread-safe.
One workaround is for our code to clone Configuration objects; unfortunately,
this also suffers from thread-safety issues on older Hadoop versions because
Configuration's constructor wasn't thread-safe (HADOOP-10456).
[Looking at a recent version of
Configuration.java|https://github.com/apache/hadoop/blob/d989ac04449dc33da5e2c32a7f24d59cc92de536/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L666],
it seems that the private {{updatingResource}} HashMap and {{finalParameters}}
HashSet fields the only non-thread-safe collections in Configuration (Java's
{{Properties}} class is thread-safe), so I don't think that it would be hard to
make Configuration fully thread-safe.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11210) Findbugs warning about SpanReceiverHost
Zhijie Shen created HADOOP-11210:
Summary: Findbugs warning about SpanReceiverHost
Key: HADOOP-11210
URL: https://issues.apache.org/jira/browse/HADOOP-11210
Project: Hadoop Common
Issue Type: Bug
Reporter: Zhijie Shen
Priority: Minor
Findbugs warning about SpanReceiverHost has bee reported multiple times in some
Jira:
{quote}
Dereference of the result of readLine() without nullcheck in
org.apache.hadoop.tracing.SpanReceiverHost.getUniqueLocalTraceFileName()
Bug type NP_DEREFERENCE_OF_READLINE_VALUE (click for details)
In class org.apache.hadoop.tracing.SpanReceiverHost
In method
org.apache.hadoop.tracing.SpanReceiverHost.getUniqueLocalTraceFileName()
Value loaded from line
At SpanReceiverHost.java:[line 104]
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HADOOP-11210) Findbugs warning about SpanReceiverHost
[
https://issues.apache.org/jira/browse/HADOOP-11210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi OZAWA resolved HADOOP-11210.
-
Resolution: Duplicate
Findbugs warning about SpanReceiverHost
---
Key: HADOOP-11210
URL: https://issues.apache.org/jira/browse/HADOOP-11210
Project: Hadoop Common
Issue Type: Bug
Reporter: Zhijie Shen
Priority: Minor
Findbugs warning about SpanReceiverHost has bee reported multiple times in
some Jira:
{quote}
Dereference of the result of readLine() without nullcheck in
org.apache.hadoop.tracing.SpanReceiverHost.getUniqueLocalTraceFileName()
Bug type NP_DEREFERENCE_OF_READLINE_VALUE (click for details)
In class org.apache.hadoop.tracing.SpanReceiverHost
In method
org.apache.hadoop.tracing.SpanReceiverHost.getUniqueLocalTraceFileName()
Value loaded from line
At SpanReceiverHost.java:[line 104]
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11211) mapreduce.job.classloader.system.classes property behave differently when the exclusion and inclusion order is differnt
Yitong Zhou created HADOOP-11211: Summary: mapreduce.job.classloader.system.classes property behave differently when the exclusion and inclusion order is differnt Key: HADOOP-11211 URL: https://issues.apache.org/jira/browse/HADOOP-11211 Project: Hadoop Common Issue Type: Bug Reporter: Yitong Zhou If we want to include package foo.bar.* but exclude all sub packages named foo.bar.tar.* in system classes, configuring mapreduce.job.classloader.system.classes=foo.bar.,-foo.bar.tar. won't work. foo.bar.tar will still be pulled in. But if we change the order: mapreduce.job.classloader.system.classes=-foo.bar.tar.,foo.bar., then it will work. This bug is due to the implementation of ApplicationClassLoaser#isSystemClass in hadoop-common, where we simply return the matching result immediately when the class name hits the first match (either positive or negative). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
