Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

2018-05-17 Thread Apache Jenkins Server 
For more details, see https://builds.apache.org/job/hadoop-trunk-win/470/

[May 16, 2018 2:23:49 PM] (jlowe) YARN-8284. get_docker_command refactoring. 
Contributed by Eric Badger
[May 16, 2018 4:31:46 PM] (inigoiri) HDFS-13557. TestDFSAdmin#testListOpenFiles 
fails on Windows. Contributed
[May 16, 2018 4:38:26 PM] (eyang) YARN-8300.  Fixed NPE in 
DefaultUpgradeComponentsFinder.
[May 16, 2018 5:08:49 PM] (inigoiri) 
HDFS-13550.TestDebugAdmin#testComputeMetaCommand fails on Windows.
[May 16, 2018 6:28:39 PM] (arp) HDFS-13512. WebHdfs getFileStatus doesn't 
return ecPolicy. Contributed
[May 16, 2018 8:00:01 PM] (haibochen) YARN-7933. [atsv2 read acls] Add 
TimelineWriter#writeDomain. (Rohith
[May 16, 2018 9:17:28 PM] (jlowe) YARN-8071. Add ability to specify nodemanager 
environment variables
[May 17, 2018 2:23:02 AM] (inigoiri) HDFS-13559. TestBlockScanner does not 
close TestContext properly.
[May 17, 2018 10:54:51 AM] (rohithsharmaks) YARN-8297. Incorrect ATS Url used 
for Wire encrypted cluster.




-1 overall


The following subsystems voted -1:
compile mvninstall unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h 00m 00s)
unit


Specific tests:

Failed junit tests :

   hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec 
   hadoop.fs.contract.rawlocal.TestRawlocalContractAppend 
   hadoop.fs.TestFsShellCopy 
   hadoop.fs.TestFsShellList 
   hadoop.fs.TestLocalFileSystem 
   hadoop.fs.TestRawLocalFileSystemContract 
   hadoop.fs.TestTrash 
   hadoop.http.TestHttpServer 
   hadoop.http.TestHttpServerLogs 
   hadoop.io.nativeio.TestNativeIO 
   hadoop.ipc.TestIPC 
   hadoop.ipc.TestSocketFactory 
   hadoop.metrics2.impl.TestStatsDMetrics 
   hadoop.security.TestGroupsCaching 
   hadoop.security.TestSecurityUtil 
   hadoop.security.TestShellBasedUnixGroupsMapping 
   hadoop.security.token.TestDtUtilShell 
   hadoop.util.TestNativeCodeLoader 
   hadoop.util.TestNodeHealthScriptRunner 
   hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy 
   hadoop.hdfs.crypto.TestHdfsCryptoStreams 
   hadoop.hdfs.qjournal.client.TestQuorumJournalManager 
   hadoop.hdfs.qjournal.server.TestJournalNode 
   hadoop.hdfs.qjournal.server.TestJournalNodeSync 
   hadoop.hdfs.security.TestDelegationTokenForProxyUser 
   hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks 
   hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages 
   hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy 
   
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement 
   
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestProvidedImpl 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation 
   hadoop.hdfs.server.datanode.TestBlockPoolSliceStorage 
   hadoop.hdfs.server.datanode.TestBlockRecovery 
   hadoop.hdfs.server.datanode.TestBlockScanner 
   hadoop.hdfs.server.datanode.TestDataNodeFaultInjector 
   hadoop.hdfs.server.datanode.TestDataNodeMetrics 
   hadoop.hdfs.server.datanode.TestDataNodeUUID 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure 
   hadoop.hdfs.server.datanode.TestDirectoryScanner 
   hadoop.hdfs.server.datanode.TestHSync 
   hadoop.hdfs.server.datanode.web.TestDatanodeHttpXFrame 
   hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand 
   hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC 
   hadoop.hdfs.server.mover.TestMover 
   hadoop.hdfs.server.mover.TestStorageMover 
   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA 
   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits 
   hadoop.hdfs.server.namenode.ha.TestHAMetrics 
   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA 
   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics 
   
hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot 
   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot 
   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots 
   hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff 
   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport 
   hadoop.hdfs.server.namenode.TestAddBlock 
   hadoop.hdfs.server.namenode.TestAuditLoggerWithCommands 
   hadoop.hdfs.server.namenode.TestCheckpoint

[jira] [Created] (HADOOP-15477) Make unjar in RunJar overrideable

2018-05-17 Thread Johan Gustavsson (JIRA) 
Johan Gustavsson created HADOOP-15477:
-

 Summary: Make unjar in RunJar overrideable
 Key: HADOOP-15477
 URL: https://issues.apache.org/jira/browse/HADOOP-15477
 Project: Hadoop Common
  Issue Type: Improvement
Affects Versions: 3.0.2, 2.7.6, 2.9.1, 2.8.3
Reporter: Johan Gustavsson


Currently Hadoop's RunJar will unjar the jar provided and look for any jars 
inside and add them to the classpath. Since most deployments doesn't use jar in 
jar, but rather uberjars this could be rather time consuming at times and can 
cause issues related to over consumption of inodes, for something that is in 
many cases is not used.

For that purpose there should be an env variable to disable this behavior.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15476) fix logging for split-dns multihome

2018-05-17 Thread Ajay Kumar (JIRA) 
Ajay Kumar created HADOOP-15476:
---

 Summary: fix logging for split-dns multihome 
 Key: HADOOP-15476
 URL: https://issues.apache.org/jira/browse/HADOOP-15476
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Ajay Kumar


Fix debug log statement introduced in [HADOOP-15250].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [DISCUSS] Branch Proposal: HADOOP 15407: ABFS

2018-05-17 Thread Chris Douglas 
There's not a lot of (required) ceremony. Any committer can create the
branch, including branch committers after the PMC adds them (see
bylaws [1]). -C

[1]: http://hadoop.apache.org/bylaws.html

On Thu, May 17, 2018 at 9:16 AM, Steve Loughran  wrote:
> Now, what's next? I know we have the normal vote process to merge a branch 
> back in...what about the branch creation + giving people branch commit rights?
>
> -steve
>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Broken build env

2018-05-17 Thread Miklos Szegedi 
Kihwal,

This appeared recently. These fixed my build inside docker.
echo export PATH=/opt/cmake/bin:/opt/protobuf/bin:$PATH >>/etc/profile
echo export CPLUS_INCLUDE_PATH=/opt/protobuf/include >>/etc/profile
echo export C_INCLUDE_PATH=/opt/protobuf/include >>/etc/profile
echo export LIBRARY_PATH=/opt/protobuf/lib >>/etc/profile
echo export LD_LIBRARY_PATH=/opt/protobuf/lib >>/etc/profile
echo export PROTOBUF_LIBRARY >>/etc/profile
echo export PROTOBUF_INCLUDE_DIR >>/etc/profile

Thank you,
Miklos Szegedi


On Thu, May 17, 2018 at 8:00 AM, Kihwal Lee  wrote:

> Simple commit builds are failing often
>
> https://builds.apache.org/job/Hadoop-trunk-Commit/
>
> Many trunk builds are failing on H19.
> "protoc version is 'libprotoc 2.6.1', expected version is '2.5.0' "
>
> On H4, a cmake version problem was seen.
>
> The commit builds don't seem to be running in a docker container (i.e.
> non-yetus env), so the env is not consistent. What will it take to fix
> this?
>
> Kihwal
>

Re: [DISCUSS] Branch Proposal: HADOOP 15407: ABFS

2018-05-17 Thread Steve Loughran 
Now, what's next? I know we have the normal vote process to merge a branch back 
in...what about the branch creation + giving people branch commit rights?

-steve


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Broken build env

2018-05-17 Thread Kihwal Lee 
Simple commit builds are failing often

https://builds.apache.org/job/Hadoop-trunk-Commit/

Many trunk builds are failing on H19.
"protoc version is 'libprotoc 2.6.1', expected version is '2.5.0' "

On H4, a cmake version problem was seen.

The commit builds don't seem to be running in a docker container (i.e.
non-yetus env), so the env is not consistent. What will it take to fix
this?

Kihwal

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2018-05-17 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/783/

[May 15, 2018 6:31:28 AM] (sunilg) YARN-8266. [UI2] Clicking on application 
from cluster view should
[May 15, 2018 6:43:04 AM] (sunilg) YARN-8166. [UI2] Service page header links 
are broken. Contributed by
[May 15, 2018 6:47:35 AM] (sunilg) YARN-8236. Invalid kerberos principal file 
name cause NPE in native
[May 15, 2018 9:28:19 AM] (wwei) YARN-8278. DistributedScheduling is not 
working in HA. Contributed by
[May 15, 2018 3:13:56 PM] (stevel) HADOOP-15442. 
ITestS3AMetrics.testMetricsRegister can't know metrics
[May 15, 2018 3:19:03 PM] (stevel) HADOOP-15466. Correct units in 
adl.http.timeout. Contributed by Sean
[May 15, 2018 5:21:42 PM] (inigoiri) HDFS-13551. 
TestMiniDFSCluster#testClusterSetStorageCapacity does not
[May 15, 2018 5:27:36 PM] (inigoiri) HDFS-11700. 
TestHDFSServerPorts#testBackupNodePorts doesn't pass on
[May 15, 2018 6:20:32 PM] (inigoiri) HDFS-13548. 
TestResolveHdfsSymlink#testFcResolveAfs fails on Windows.
[May 15, 2018 10:34:54 PM] (inigoiri) HDFS-13567.
[May 16, 2018 12:40:39 AM] (eyang) YARN-8081.  Add support to upgrade a 
component. Contributed
[May 16, 2018 8:25:31 AM] (aajisaka) YARN-8123. Skip compiling old hamlet 
package when the Java version is 10
[May 16, 2018 2:23:49 PM] (jlowe) YARN-8284. get_docker_command refactoring. 
Contributed by Eric Badger
[May 16, 2018 4:31:46 PM] (inigoiri) HDFS-13557. TestDFSAdmin#testListOpenFiles 
fails on Windows. Contributed
[May 16, 2018 4:38:26 PM] (eyang) YARN-8300.  Fixed NPE in 
DefaultUpgradeComponentsFinder.
[May 16, 2018 5:08:49 PM] (inigoiri) 
HDFS-13550.TestDebugAdmin#testComputeMetaCommand fails on Windows.
[May 16, 2018 6:28:39 PM] (arp) HDFS-13512. WebHdfs getFileStatus doesn't 
return ecPolicy. Contributed
[May 16, 2018 8:00:01 PM] (haibochen) YARN-7933. [atsv2 read acls] Add 
TimelineWriter#writeDomain. (Rohith
[May 16, 2018 9:17:28 PM] (jlowe) YARN-8071. Add ability to specify nodemanager 
environment variables




-1 overall


The following subsystems voted -1:
asflicense findbugs unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

FindBugs :

   module:hadoop-hdds/common 
   Useless control flow in 
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$CloseContainerRequestProto$Builder.maybeForceBuilderInitialization()
 At ContainerProtos.java: At ContainerProtos.java:[line 18039] 
   Useless control flow in 
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$CloseContainerResponseProto$Builder.maybeForceBuilderInitialization()
 At ContainerProtos.java: At ContainerProtos.java:[line 18601] 
   Useless control flow in 
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$CopyContainerRequestProto$Builder.maybeForceBuilderInitialization()
 At ContainerProtos.java: At ContainerProtos.java:[line 35184] 
   Useless control flow in 
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$CopyContainerResponseProto$Builder.maybeForceBuilderInitialization()
 At ContainerProtos.java: At ContainerProtos.java:[line 36053] 
   Useless control flow in 
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$CreateContainerResponseProto$Builder.maybeForceBuilderInitialization()
 At ContainerProtos.java: At ContainerProtos.java:[line 13089] 
   Useless control flow in 
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$DatanodeBlockID$Builder.maybeForceBuilderInitialization()
 At ContainerProtos.java: At ContainerProtos.java:[line 1126] 
   Useless control flow in 
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$DeleteChunkResponseProto$Builder.maybeForceBuilderInitialization()
 At ContainerProtos.java: At ContainerProtos.java:[line 30491] 
   Useless control flow in 
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$DeleteContainerRequestProto$Builder.maybeForceBuilderInitialization()
 At ContainerProtos.java: At ContainerProtos.java:[line 15748] 
   Useless control flow in 
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$DeleteContainerResponseProto$Builder.maybeForceBuilderInitialization()
 At ContainerProtos.java: At ContainerProtos.java:[line 16224] 
   Useless control flow in 
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$DeleteKeyResponseProto$Builder.maybeForceBuilderInitialization()
 At ContainerProtos.java: At ContainerProtos.java:[line 23421] 
   Useless control flow in 
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos$KeyValue$Builder.maybeForceBuilderInitialization()
 At ContainerProtos.java: At ContainerProtos.java:[line 1767] 
   Useless control flow in

[jira] [Created] (HADOOP-15474) Rename properties introduced for

2018-05-17 Thread Nanda kumar (JIRA) 
Nanda kumar created HADOOP-15474:


 Summary: Rename properties introduced for 
 Key: HADOOP-15474
 URL: https://issues.apache.org/jira/browse/HADOOP-15474
 Project: Hadoop Common
  Issue Type: Bug
  Components: conf
Affects Versions: 3.1.0
Reporter: Nanda kumar


HADOOP-15007 introduces the following two properties for tagging configuration 
properties
* hadoop.system.tags
* hadoop.custom.tags

This sounds like {{tags}} fall under {{hadoop.system}} and {{hadoop.custom}} 
related properties, but what we really want to achieve here is to have two 
sub-division of {{tags}} namely {{system}} and {{custom}}

For better readability, we can rename them as
* hadoop.tags.system
* hadoop.tags.custom





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org