Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

[Apache Jenkins Server]

For more details, see https://builds.apache.org/job/hadoop-trunk-win/515/

[Jul 1, 2018 11:41:46 AM] (nanda) HADOOP-15574: Suppress build error if there 
are no docs after excluding




-1 overall


The following subsystems voted -1:
compile mvninstall pathlen unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h 00m 00s)
unit


Specific tests:

Failed junit tests :

   hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec 
   hadoop.fs.contract.rawlocal.TestRawlocalContractAppend 
   hadoop.fs.TestFileUtil 
   hadoop.fs.TestFsShellCopy 
   hadoop.fs.TestFsShellList 
   hadoop.http.TestHttpServer 
   hadoop.http.TestHttpServerLogs 
   hadoop.io.nativeio.TestNativeIO 
   hadoop.ipc.TestCallQueueManager 
   hadoop.ipc.TestSocketFactory 
   hadoop.metrics2.impl.TestStatsDMetrics 
   hadoop.security.TestSecurityUtil 
   hadoop.security.TestShellBasedUnixGroupsMapping 
   hadoop.security.token.TestDtUtilShell 
   hadoop.util.TestDiskCheckerWithDiskIo 
   hadoop.util.TestNativeCodeLoader 
   hadoop.hdfs.qjournal.server.TestJournalNode 
   hadoop.hdfs.qjournal.server.TestJournalNodeSync 
   hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestProvidedImpl 
   hadoop.hdfs.server.datanode.TestBlockPoolSliceStorage 
   hadoop.hdfs.server.datanode.TestBlockScanner 
   hadoop.hdfs.server.datanode.TestDataNodeFaultInjector 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure 
   hadoop.hdfs.server.datanode.TestDirectoryScanner 
   hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport 
   hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand 
   hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC 
   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA 
   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA 
   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics 
   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport 
   hadoop.hdfs.server.namenode.TestNameNodeMXBean 
   hadoop.hdfs.server.namenode.TestReencryptionWithKMS 
   hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs 
   hadoop.hdfs.TestDFSShell 
   hadoop.hdfs.TestDFSStripedInputStream 
   hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy 
   hadoop.hdfs.TestDFSUpgradeFromImage 
   hadoop.hdfs.TestFetchImage 
   hadoop.hdfs.TestFileConcurrentReader 
   hadoop.hdfs.TestHDFSFileSystemContract 
   hadoop.hdfs.TestLeaseRecovery2 
   hadoop.hdfs.TestPread 
   hadoop.hdfs.TestReconstructStripedFile 
   hadoop.hdfs.TestSecureEncryptionZoneWithKMS 
   hadoop.hdfs.TestTrashWithSecureEncryptionZones 
   hadoop.hdfs.tools.TestDFSAdmin 
   hadoop.hdfs.web.TestWebHDFS 
   hadoop.hdfs.web.TestWebHdfsUrl 
   hadoop.fs.http.server.TestHttpFSServerWebServer 
   
hadoop.yarn.logaggregation.filecontroller.ifile.TestLogAggregationIndexFileController
 
   
hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch 
   hadoop.yarn.server.nodemanager.containermanager.TestAuxServices 
   hadoop.yarn.server.nodemanager.containermanager.TestContainerManager 
   hadoop.yarn.server.nodemanager.recovery.TestNMLeveldbStateStoreService 
   hadoop.yarn.server.nodemanager.TestContainerExecutor 
   hadoop.yarn.server.nodemanager.TestNodeManagerResync 
   hadoop.yarn.server.webproxy.amfilter.TestAmFilter 
   
hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer 
   
hadoop.yarn.server.timeline.security.TestTimelineAuthenticationFilterForV1 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestFSSchedulerConfigurationStore
 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestLeveldbConfigurationStore
 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
 
   
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementProcessor 
   
hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService
 
   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer 
   hadoop.yarn.server.resourcemanager.TestResourceTrackerService 
   hadoop.yarn.client.api.impl.TestNMClient 
   hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore 
   
hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage
 
   

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/828/

No changes


[Error replacing 'FILE' - Workspace is not accessible]

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

[Apache Jenkins Server]

For more details, see https://builds.apache.org/job/hadoop-trunk-win/514/

[Jun 29, 2018 4:56:13 PM] (inigoiri) HDFS-13707. [PROVIDED Storage] Fix failing 
integration tests in
[Jun 29, 2018 5:02:53 PM] (sunilg) YARN-8455. Add basic ACL check for all ATS 
v2 REST APIs. Contributed by
[Jun 29, 2018 5:21:25 PM] (rohithsharmaks) YARN-8469. [UI2] URL needs to be 
trimmed to handle index.html
[Jun 29, 2018 6:06:28 PM] (jlowe) YARN-8451. Multiple NM heartbeat thread 
created when a slow NM resync
[Jun 29, 2018 6:18:32 PM] (ericp) HADOOP-15548: Randomize local dirs. 
Contributed by Jim Brennan.
[Jun 29, 2018 6:47:30 PM] (gifuma) YARN-8481. AMRMProxyPolicies should accept 
heartbeat response from




-1 overall


The following subsystems voted -1:
compile mvninstall pathlen unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h 00m 00s)
unit


Specific tests:

Failed junit tests :

   hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec 
   hadoop.fs.contract.rawlocal.TestRawlocalContractAppend 
   hadoop.fs.TestFileUtil 
   hadoop.fs.TestFsShellCopy 
   hadoop.fs.TestFsShellList 
   hadoop.http.TestHttpServer 
   hadoop.http.TestHttpServerLogs 
   hadoop.io.nativeio.TestNativeIO 
   hadoop.ipc.TestSocketFactory 
   hadoop.metrics2.impl.TestStatsDMetrics 
   hadoop.security.TestSecurityUtil 
   hadoop.security.TestShellBasedUnixGroupsMapping 
   hadoop.security.token.TestDtUtilShell 
   hadoop.util.TestDiskCheckerWithDiskIo 
   hadoop.util.TestNativeCodeLoader 
   hadoop.hdfs.qjournal.server.TestJournalNode 
   hadoop.hdfs.qjournal.server.TestJournalNodeSync 
   hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages 
   hadoop.hdfs.server.datanode.fsdataset.impl.TestProvidedImpl 
   hadoop.hdfs.server.datanode.TestBlockPoolSliceStorage 
   hadoop.hdfs.server.datanode.TestBlockScanner 
   hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics 
   hadoop.hdfs.server.datanode.TestDataNodeFaultInjector 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure 
   hadoop.hdfs.server.datanode.TestDirectoryScanner 
   hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport 
   hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC 
   hadoop.hdfs.server.mover.TestMover 
   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA 
   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA 
   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics 
   hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs 
   hadoop.hdfs.TestDFSShell 
   hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy 
   hadoop.hdfs.TestDFSStripedOutputStream 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy 
   hadoop.hdfs.TestDFSUpgradeFromImage 
   hadoop.hdfs.TestFetchImage 
   hadoop.hdfs.TestHDFSFileSystemContract 
   hadoop.hdfs.TestLeaseRecovery2 
   hadoop.hdfs.TestPread 
   hadoop.hdfs.TestSecureEncryptionZoneWithKMS 
   hadoop.hdfs.TestTrashWithSecureEncryptionZones 
   hadoop.hdfs.tools.TestDFSAdmin 
   hadoop.hdfs.web.TestWebHDFS 
   hadoop.hdfs.web.TestWebHdfsUrl 
   hadoop.fs.http.server.TestHttpFSServerWebServer 
   
hadoop.yarn.logaggregation.filecontroller.ifile.TestLogAggregationIndexFileController
 
   
hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch 
   hadoop.yarn.server.nodemanager.containermanager.TestAuxServices 
   hadoop.yarn.server.nodemanager.containermanager.TestContainerManager 
   hadoop.yarn.server.nodemanager.recovery.TestNMLeveldbStateStoreService 
   hadoop.yarn.server.nodemanager.TestContainerExecutor 
   hadoop.yarn.server.nodemanager.TestNodeManagerResync 
   hadoop.yarn.server.webproxy.amfilter.TestAmFilter 
   
hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer 
   
hadoop.yarn.server.timeline.security.TestTimelineAuthenticationFilterForV1 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestFSSchedulerConfigurationStore
 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestLeveldbConfigurationStore
 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing 
   
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementProcessor 
   
hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService
 
   hadoop.yarn.server.resourcemanager.TestCapacitySchedulerMetrics 
   hadoop.yarn.server.resourcemanager.TestResourceTrackerService 
   hadoop.yarn.client.api.impl.TestAMRMProxy 
   

Re: 答复: [DISCUSS] Merge Storage Policy Satisfier (SPS) [HDFS-10285] feature branch to trunk

[Uma Maheswara Rao G]

Hi Yiqun,
Thanks for your email.

On Wed, Jun 27, 2018 at 7:19 PM, Lin,Yiqun(vip.com)  wrote:

> Hi Uma,
>
> Seeing the discussion under JIRA HDFS-10285, external SPS running will be
> a recommend way for the users, right?

[Uma]Right now we are providing this option in this phase.

> As you also mentioned, there are some additional works need to test and be
> integrated. One more question from me, that maybe also cared by other
> developers, what's the major differences between internal SPS and external
> SPS? Just not to affect NN running?
> Could you describe a little more for this?

[Uma]You are right. When process running outside, it will not impact NN.
For internal, since we access internal data structures, we wanted to be
careful and we have new thoughts to have that option runs rightly with
Replication monitor etc and that needs some more time for analysis and
testing etc.

>


> Thanks
> Yiqun
>
> Regards,
Uma

> -邮件原件-
> 发件人: Uma Maheswara Rao G [mailto:hadoop@gmail.com]
> 发送时间: 2018年6月28日 6:22
> 收件人: hdfs-dev@hadoop.apache.org
> 主题: Re: [DISCUSS] Merge Storage Policy Satisfier (SPS) [HDFS-10285]
> feature branch to trunk
>
> Hi All,
>
>   After long discussions(offline and on JIRA) on SPS, we came to a
> conclusion on JIRA(HDFS-10285) that, we will go ahead with External SPS
> merge in first phase. In this phase process will not be running inside
> Namenode.
>   We will continue discussion on Internal SPS. Current code base supports
> both internal and external option. We have review comments for Internal
> which needs some additional works for analysis and testing etc. We will
> move Internal SPS work to under HDFS-12226 (Follow-on work for SPS in NN)
> We are working on cleanup task HDFS-13076 for the merge. .
> For more clarity on Internal and External SPS proposal thoughts, please
> refer to JIRA HDFS-10285.
>
> If there are no objections with this, I will go ahead for voting soon.
>
> Regards,
> Uma
>
> On Fri, Nov 17, 2017 at 3:16 PM, Uma Maheswara Rao G  >
> wrote:
>
> > Update: We worked on the review comments and additional JIRAs above
> > mentioned.
> >
> > >1. After the feedbacks from Andrew, Eddy, Xiao in JIRA reviews, we
> > planned to take up the support for recursive API support. HDFS-12291<
> > https://issues.apache.org/jira/browse/HDFS-12291>
> >
> > We provided the recursive API support now.
> >
> > >2. Xattr optimizations HDFS-12225 > he.org/jira/browse/HDFS-12225>
> > Improved this portion as well
> >
> > >3. Few other review comments already fixed and committed HDFS-12214<
> > https://issues.apache.org/jira/browse/HDFS-12214>
> > Fixed the comments.
> >
> > We are continuing to test the feature and working so far well. Also we
> > uploaded a combined patch and got the good QA report.
> >
> > If there are no further objections, we would like to go for merge vote
> > tomorrow. Please by default this feature will be disabled.
> >
> > Regards,
> > Uma
> >
> > On Fri, Aug 18, 2017 at 11:27 PM, Gangumalla, Uma <
> > uma.ganguma...@intel.com> wrote:
> >
> >> Hi Andrew,
> >>
> >> >Great to hear. It'd be nice to define which use cases are met by the
> >> current version of SPS, and which will be handled after the merge.
> >> After the discussions in JIRA, we planned to support recursive API as
> >> well. The primary use cases we planned was for Hbase. Please check
> >> next point for use case details.
> >>
> >> >A bit more detail in the design doc on how HBase would use this
> >> >feature
> >> would also be helpful. Is there an HBase JIRA already?
> >> Please find the usecase details at this comment in JIRA:
> >> https://issues.apache.org/jira/browse/HDFS-10285?focusedComm
> >> entId=16120227=com.atlassian.jira.plugin.system.issueta
> >> bpanels:comment-tabpanel#comment-16120227
> >>
> >> >I also spent some more time with the design doc and posted a few
> >> questions on the JIRA.
> >> Thank you for the reviews.
> >>
> >> To summarize the discussions in JIRA:
> >> 1. After the feedbacks from Andrew, Eddy, Xiao in JIRA reviews, we
> >> planned to take up the support for recursive API support. HDFS-12291<
> >> https://issues.apache.org/jira/browse/HDFS-12291> (Rakesh started the
> >> work on it) 2. Xattr optimizations HDFS-12225 >> he.org/jira/browse/HDFS-12225> (Patch available) 3. Few other review
> >> comments already fixed and committed HDFS-12214<
> >> https://issues.apache.org/jira/browse/HDFS-12214>
> >>
> >> For tracking the follow-up tasks we filed JIRA HDFS-12226, they
> >> should not be critical for merge.
> >>
> >> Regards,
> >> Uma
> >>
> >> From: Andrew Wang  >> andrew.w...@cloudera.com>>
> >> Date: Friday, July 28, 2017 at 11:33 AM
> >> To: Uma Gangumalla  >> uma.ganguma...@intel.com>>
> >> Cc: "hdfs-dev@hadoop.apache.org" <
> >> hdfs-dev@hadoop.apache.org>
> >> Subject: Re: [DISCUSS] Merge Storage Policy Satisfier (SPS)
> >> [HDFS-10285] feature