Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le

2016-10-04 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/114/

[Oct 3, 2016 1:03:46 PM] (kasha) YARN-5672. FairScheduler: Wrong queue name in 
log when adding
[Oct 3, 2016 2:13:04 PM] (kihwal) HDFS-10619. Cache path in InodesInPath. 
Contributed by Daryn Sharp.
[Oct 3, 2016 4:27:23 PM] (kihwal) HDFS-10940. Reduce performance penalty of 
block caching when not used.
[Oct 3, 2016 5:30:22 PM] (kasha) MAPREDUCE-6638. Do not attempt to recover 
progress from previous job
[Oct 3, 2016 5:53:21 PM] (xyao) HDFS-10690. Optimize insertion/removal of 
replica in ShortCircuitCache.
[Oct 3, 2016 9:35:57 PM] (kasha) YARN-4767. Network issues can cause persistent 
RM UI outage. (Daniel
[Oct 3, 2016 11:01:54 PM] (xiao) HDFS-10918. Add a tool to get 
FileEncryptionInfo from CLI. Contributed
[Oct 4, 2016 1:50:16 AM] (liuml07) HDFS-10810. setreplication removing block 
from underconstrcution
[Oct 4, 2016 2:34:41 AM] (liuml07) HDFS-10934. TestDFSShell#testStat fails 
intermittently. Contributed by




-1 overall


The following subsystems voted -1:
compile unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.TestWriteReadStripedFile 
   hadoop.hdfs.server.datanode.TestDataNodeLifeline 
   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.hdfs.server.datanode.TestDirectoryScanner 
   hadoop.yarn.server.nodemanager.recovery.TestNMLeveldbStateStoreService 
   
hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService
 
   hadoop.yarn.server.nodemanager.TestNodeManagerShutdown 
   hadoop.yarn.server.timeline.TestRollingLevelDB 
   hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices 
   hadoop.yarn.server.timeline.TestTimelineDataManager 
   hadoop.yarn.server.timeline.TestLeveldbTimelineStore 
   hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore 
   hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore 
   
hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer 
   hadoop.yarn.server.timelineservice.storage.common.TestRowKeys 
   hadoop.yarn.server.timelineservice.storage.common.TestKeyConverters 
   hadoop.yarn.server.timelineservice.storage.common.TestSeparator 
   hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore 
   hadoop.yarn.server.resourcemanager.TestRMRestart 
   hadoop.yarn.server.resourcemanager.TestResourceTrackerService 
   hadoop.yarn.server.TestMiniYarnClusterNodeUtilization 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.yarn.client.api.impl.TestNMClient 
   hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore 
   hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient 
   hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore 
   hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorage 
   
hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction
 
   hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun 
   
hadoop.yarn.server.timelineservice.storage.TestPhoenixOfflineAggregationWriterImpl
 
   
hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage
 
   
hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowActivity 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
   hadoop.mapred.TestShuffleHandler 
   hadoop.mapreduce.v2.app.webapp.TestAMWebApp 
   hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService 
   hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers 

Timed out junit tests :

   org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache 
   org.apache.hadoop.mapred.TestMRIntermediateDataEncryption 
  

   compile:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/114/artifact/out/patch-compile-root.txt
  [320K]

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/114/artifact/out/patch-compile-root.txt
  [320K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/114/artifact/out/patch-compile-root.txt
  [320K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/114/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [200K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/114/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
  [48K]

[jira] [Created] (HADOOP-13683) Update jackson from 1.9.13 to 2.x in hadoop-hdfs

2016-10-04 Thread Akira Ajisaka (JIRA) 
Akira Ajisaka created HADOOP-13683:
--

 Summary: Update jackson from 1.9.13 to 2.x in hadoop-hdfs
 Key: HADOOP-13683
 URL: https://issues.apache.org/jira/browse/HADOOP-13683
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Akira Ajisaka






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-13682) Log missing stacktrace in catch block

2016-10-04 Thread Wei-Chiu Chuang (JIRA) 
Wei-Chiu Chuang created HADOOP-13682:


 Summary: Log missing stacktrace in catch block
 Key: HADOOP-13682
 URL: https://issues.apache.org/jira/browse/HADOOP-13682
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Wei-Chiu Chuang
Priority: Minor
 Attachments: common_output

Wrote a simple [code parser|https://github.com/jojochuang/codebetter] to find 
all instances of exception objects that are caught but logged without stack 
trace. Some of them may have legitimate reason not to log stack trace, but 
other may worth an improvement.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-13684) Snappy may complain Hadoop is built without snappy if libhadoop is not found.

2016-10-04 Thread Wei-Chiu Chuang (JIRA) 
Wei-Chiu Chuang created HADOOP-13684:


 Summary: Snappy may complain Hadoop is built without snappy if 
libhadoop is not found.
 Key: HADOOP-13684
 URL: https://issues.apache.org/jira/browse/HADOOP-13684
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Wei-Chiu Chuang
Priority: Minor


If for some reason libhadoop can not be found/loaded, Snappy complains Hadoop 
is not built with Snappy but it actually is.

{code:title=SnappyCodec.java}
public static void checkNativeCodeLoaded() {
  if (!NativeCodeLoader.isNativeCodeLoaded() ||
  !NativeCodeLoader.buildSupportsSnappy()) {
throw new RuntimeException("native snappy library not available: " +
"this version of libhadoop was built without " +
"snappy support.");
  }
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-13685) Document -safely option to rm command.

2016-10-04 Thread Xiaoyu Yao (JIRA) 
Xiaoyu Yao created HADOOP-13685:
---

 Summary: Document -safely option to rm command.
 Key: HADOOP-13685
 URL: https://issues.apache.org/jira/browse/HADOOP-13685
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Xiaoyu Yao
Assignee: Xiaoyu Yao


HADOOP-12358 introduced -safely option to prevent accidental deletion of large 
directories with lots of files with -rmr -skipTrash. This ticket is opened to 
track document the usage of this feature in 
[FileSystemShell.html|https://hadoop.apache.org/docs/r3.0.0-alpha1/hadoop-project-dist/hadoop-common/FileSystemShell.html#rm].
 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-13677) Update jackson from 1.9.13 to 2.x in hadoop-yarn

2016-10-04 Thread Akira Ajisaka (JIRA) 
Akira Ajisaka created HADOOP-13677:
--

 Summary: Update jackson from 1.9.13 to 2.x in hadoop-yarn
 Key: HADOOP-13677
 URL: https://issues.apache.org/jira/browse/HADOOP-13677
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Akira Ajisaka






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-13678) Update jackson from 1.9.13 to 2.x in hadoop-tools

2016-10-04 Thread Akira Ajisaka (JIRA) 
Akira Ajisaka created HADOOP-13678:
--

 Summary: Update jackson from 1.9.13 to 2.x in hadoop-tools
 Key: HADOOP-13678
 URL: https://issues.apache.org/jira/browse/HADOOP-13678
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: tools
Reporter: Akira Ajisaka
Assignee: Akira Ajisaka






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-13679) org.apache.hadoop.fs.Path doesn't specify mentioned differences from URIs

2016-10-04 Thread Daniel Barclay (JIRA) 
Daniel Barclay created HADOOP-13679:
---

 Summary: org.apache.hadoop.fs.Path doesn't specify mentioned 
differences from URIs
 Key: HADOOP-13679
 URL: https://issues.apache.org/jira/browse/HADOOP-13679
 Project: Hadoop Common
  Issue Type: Bug
  Components: documentation
Reporter: Daniel Barclay


The documentation comment on {{org.apache.hadoop.fs.Path.Path(String)}} says:
bq. Path strings are URIs, but with unescaped elements and some additional 
normalization.

However, neither that comment nor {{org.apache.hadoop.fs.Path}}'s documentation 
comment says anything further about exactly what the differences between URI 
and {{org.apache.hadoop.fs.Path}} strings are.


(Knowing what {{Path}} strings are supposed to be is important for determining 
whether the code is buggy or not.

In particular, {{Path(String)}} and friends seem to parse the scheme portion 
incorrectly (taking anything before a (first) colon rather than following the 
RFC 3986 syntax of:
bq. {{scheme  = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )}}
).)


So what exactly are the actual differences between {{Path}} string syntax and 
URI syntax?

(I might be able to submit a PR with documentation changes, but I need the raw 
information first.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2016-10-04 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/

[Oct 3, 2016 1:03:46 PM] (kasha) YARN-5672. FairScheduler: Wrong queue name in 
log when adding
[Oct 3, 2016 2:13:04 PM] (kihwal) HDFS-10619. Cache path in InodesInPath. 
Contributed by Daryn Sharp.
[Oct 3, 2016 4:27:23 PM] (kihwal) HDFS-10940. Reduce performance penalty of 
block caching when not used.
[Oct 3, 2016 5:30:22 PM] (kasha) MAPREDUCE-6638. Do not attempt to recover 
progress from previous job
[Oct 3, 2016 5:53:21 PM] (xyao) HDFS-10690. Optimize insertion/removal of 
replica in ShortCircuitCache.
[Oct 3, 2016 9:35:57 PM] (kasha) YARN-4767. Network issues can cause persistent 
RM UI outage. (Daniel
[Oct 3, 2016 11:01:54 PM] (xiao) HDFS-10918. Add a tool to get 
FileEncryptionInfo from CLI. Contributed
[Oct 4, 2016 1:50:16 AM] (liuml07) HDFS-10810. setreplication removing block 
from underconstrcution
[Oct 4, 2016 2:34:41 AM] (liuml07) HDFS-10934. TestDFSShell#testStat fails 
intermittently. Contributed by




-1 overall


The following subsystems voted -1:
asflicense unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices 
   hadoop.yarn.server.resourcemanager.TestRMRestart 
   hadoop.yarn.server.TestMiniYarnClusterNodeUtilization 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.mapreduce.v2.app.webapp.TestAMWebApp 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/diff-compile-javac-root.txt
  [168K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/diff-checkstyle-root.txt
  [16M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/diff-patch-pylint.txt
  [16K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/diff-patch-shelldocs.txt
  [16K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/whitespace-eol.txt
  [11M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/whitespace-tabs.txt
  [1.3M]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/diff-javadoc-javadoc-root.txt
  [2.2M]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [56K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
  [268K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
  [24K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt
  [124K]

   asflicense:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/184/artifact/out/patch-asflicense-problems.txt
  [4.0K]

Powered by Apache Yetus 0.4.0-SNAPSHOT   http://yetus.apache.org



-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-13680) fs.s3a.readahead.range to use getLongBytes

2016-10-04 Thread Steve Loughran (JIRA) 
Steve Loughran created HADOOP-13680:
---

 Summary: fs.s3a.readahead.range to use getLongBytes
 Key: HADOOP-13680
 URL: https://issues.apache.org/jira/browse/HADOOP-13680
 Project: Hadoop Common
  Issue Type: Sub-task
Reporter: Steve Loughran


The {{fs.s3a.readahead.range}} value is measured in bytes, but can be hundreds 
of KB. Easier to use getLongBytes and set to things like "300k"

This will be backwards compatible with the existing settings if anyone is using 
them, because the no-prefix default will still be bytes



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-13681) Reduce Kafka dependencies in hadoop-kafka module

2016-10-04 Thread Grant Henke (JIRA) 
Grant Henke created HADOOP-13681:


 Summary: Reduce Kafka dependencies in hadoop-kafka module
 Key: HADOOP-13681
 URL: https://issues.apache.org/jira/browse/HADOOP-13681
 Project: Hadoop Common
  Issue Type: Improvement
  Components: metrics
Affects Versions: 3.0.0-alpha1
Reporter: Grant Henke


The newly added Kafka module defines the Kafka dependency as:
{noformat}
...
org.apache.kafka
kafka-2.10
${kafka.version}
...
{noformat}

This is unfavorable because its using the server dependency, which transitively 
has the client jars. The server dependency includes all of the server code and 
some larger transitive dependencies like Scala and Zookeeper.

Instead the pom file should be changed to only depend on the clients jar which 
is a much smaller footprint:
{noformat}
...
org.apache.kafka
kafka-clients
${kafka.version}
...
{noformat}






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Maven build: YARN timeline service downloading maven-metadata from personal repository?

2016-10-04 Thread Sangjin Lee 
Thanks Wangda.

To answer Steve's question, I don't think maven downloads anything from
that location (it's a very old content). It just does a wasted effort by
hitting this repo.

On Mon, Oct 3, 2016 at 10:25 AM, Wangda Tan  wrote:

> Filed: https://issues.apache.org/jira/browse/HBASE-16749
>
> On Mon, Oct 3, 2016 at 10:18 AM, Wangda Tan  wrote:
>
> > Thanks Sangjin/Ted/Steve for your comments/suggestions, I will file a
> > HBase JIRA later.
> >
> > Regards,
> > Wangda
> >
> > On Mon, Oct 3, 2016 at 4:43 AM, Steve Loughran 
> > wrote:
> >
> >> HBase really ought to have a profile for D/Ling from somewhere like
> this,
> >> and, perhaps, list the ASF snapsphot repo first
> >>
> >> On 1 Oct 2016, at 17:37, Wangda Tan  wrote:
> >> >
> >> > Hi YARN-dev,
> >> >
> >> > (cc common-dev),
> >> >
> >> > YARN timeline service currently sometimes downloads maven-metadata.xml
> >> from
> >> > a personal apache site, log looks like:
> >> >
> >> > [INFO] 
> >> > 
> >> > [INFO] Building Apache Hadoop YARN Timeline Service
> >> 3.0.0-alpha2-SNAPSHOT
> >> > [INFO] 
> >> > 
> >> > Downloading: http://conjars.org/repo/org/
> apache/hadoop/hadoop-client/3.
> >> > 0.0-alpha2-SNAPSHOT/maven-metadata.xml
> >> > ...
> >> > Downloading: http://people.apache.org/~garyh/mvn/org/apache/hadoop/
> >> > hadoop-client/3.0.0-alpha2-SNAPSHOT/maven-metadata.xml
> >> > ...
> >> > Downloading: http://people.apache.org/~garyh/mvn/org/apache/hadoop/
> >> > hadoop-client/3.0.0-alpha2-SNAPSHOT/maven-metadata.xml
> >> >
> >> > I noticed this happens for a while, I'm not sure if it causes by my
> >> local
> >>
> >> is it actually D/Lin'g the files? Or is it looking for them and then
> >> going on to try the next repo?
> >>
> >
> >
>