Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64
For more details, see https://builds.apache.org/job/hadoop-trunk-win/513/ [Jun 28, 2018 4:21:56 PM] (Bharat) HDFS-13705:The native ISA-L library loading failure should be made [Jun 28, 2018 4:39:49 PM] (eyang) YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService. [Jun 28, 2018 5:23:31 PM] (sunilg) YARN-8379. Improve balancing resources in already satisfied queues by [Jun 28, 2018 10:41:39 PM] (nanda) HDDS-185: TestCloseContainerByPipeline#testCloseContainerViaRatis fail [Jun 28, 2018 11:07:16 PM] (nanda) HDDS-178: DN should update transactionId on block delete. Contributed by -1 overall The following subsystems voted -1: compile mvninstall pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 00m 00s) unit Specific tests: Failed junit tests : hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec hadoop.fs.contract.rawlocal.TestRawlocalContractAppend hadoop.fs.TestFileUtil hadoop.fs.TestFsShellCopy hadoop.fs.TestFsShellList hadoop.http.TestHttpServer hadoop.http.TestHttpServerLogs hadoop.io.nativeio.TestNativeIO hadoop.ipc.TestSocketFactory hadoop.metrics2.impl.TestStatsDMetrics hadoop.security.TestSecurityUtil hadoop.security.TestShellBasedUnixGroupsMapping hadoop.security.token.TestDtUtilShell hadoop.util.TestDiskCheckerWithDiskIo hadoop.util.TestNativeCodeLoader hadoop.hdfs.qjournal.server.TestJournalNode hadoop.hdfs.qjournal.server.TestJournalNodeSync hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages hadoop.hdfs.server.datanode.fsdataset.impl.TestProvidedImpl hadoop.hdfs.server.datanode.TestBlockPoolSliceStorage hadoop.hdfs.server.datanode.TestBlockScanner hadoop.hdfs.server.datanode.TestDataNodeFaultInjector hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure hadoop.hdfs.server.datanode.TestDirectoryScanner hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC hadoop.hdfs.server.mover.TestMover hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs hadoop.hdfs.TestDFSShell hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy hadoop.hdfs.TestDFSStripedOutputStream hadoop.hdfs.TestDFSStripedOutputStreamWithFailure hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy hadoop.hdfs.TestDFSUpgradeFromImage hadoop.hdfs.TestDistributedFileSystem hadoop.hdfs.TestFetchImage hadoop.hdfs.TestHDFSFileSystemContract hadoop.hdfs.TestPread hadoop.hdfs.TestReconstructStripedFile hadoop.hdfs.TestSecureEncryptionZoneWithKMS hadoop.hdfs.TestTrashWithSecureEncryptionZones hadoop.hdfs.tools.TestDFSAdmin hadoop.hdfs.web.TestWebHDFS hadoop.hdfs.web.TestWebHdfsUrl hadoop.fs.http.server.TestHttpFSServerWebServer hadoop.yarn.logaggregation.filecontroller.ifile.TestLogAggregationIndexFileController hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch hadoop.yarn.server.nodemanager.containermanager.TestAuxServices hadoop.yarn.server.nodemanager.containermanager.TestContainerManager hadoop.yarn.server.nodemanager.recovery.TestNMLeveldbStateStoreService hadoop.yarn.server.nodemanager.TestContainerExecutor hadoop.yarn.server.nodemanager.TestNodeManagerResync hadoop.yarn.server.webproxy.amfilter.TestAmFilter hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer hadoop.yarn.server.timeline.security.TestTimelineAuthenticationFilterForV1 hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestFSSchedulerConfigurationStore hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestLeveldbConfigurationStore hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementProcessor hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService hadoop.yarn.server.resourcemanager.TestResourceTrackerService hadoop.yarn.client.api.impl.TestAMRMProxy hadoop.yarn.client.api.impl.TestNMClient hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore
[jira] [Resolved] (YARN-8478) The capacity scheduler logs too frequently seriously affecting performance
[ https://issues.apache.org/jira/browse/YARN-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved YARN-8478. -- Resolution: Duplicate > The capacity scheduler logs too frequently seriously affecting performance > -- > > Key: YARN-8478 > URL: https://issues.apache.org/jira/browse/YARN-8478 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler, capacityscheduler >Reporter: YunFan Zhou >Assignee: YunFan Zhou >Priority: Critical > Attachments: image-2018-06-29-14-08-50-981.png > > > The capacity scheduler logs too frequently, seriously affecting performance. > As a result of our test that the scheduling speed of capacity scheduler is > difficult to reach 5000/s in the production scenario. > And it will soon reach the log bottleneck. > My current work is to change many log levels from INFO to DEBUG level. > [~wangda] [~leftnoteasy] Any suggestion? > !image-2018-06-29-14-08-50-981.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8484) Fix NPE during ServiceStop in Router classes
Giovanni Matteo Fumarola created YARN-8484: -- Summary: Fix NPE during ServiceStop in Router classes Key: YARN-8484 URL: https://issues.apache.org/jira/browse/YARN-8484 Project: Hadoop YARN Issue Type: Bug Reporter: Giovanni Matteo Fumarola -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8483) Logs in FederationInterceptorREST should give some information on the exception
Íñigo Goiri created YARN-8483: - Summary: Logs in FederationInterceptorREST should give some information on the exception Key: YARN-8483 URL: https://issues.apache.org/jira/browse/YARN-8483 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.9.1 Reporter: Íñigo Goiri Assignee: Íñigo Goiri When there is an error collecting the data from the subclusters, there is not details: [FederationInterceptorREST #3] org.apache.hadoop.yarn.server.router.webapp.FederationInterceptorREST: Subcluster 1 failed to return nodesInfo. [FederationInterceptorREST #2] org.apache.hadoop.yarn.server.router.webapp.FederationInterceptorREST: Subcluster 0 failed to return nodesInfo. We should add the message for the exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8482) [Router] Add cache service for fast answers to getApps
Giovanni Matteo Fumarola created YARN-8482: -- Summary: [Router] Add cache service for fast answers to getApps Key: YARN-8482 URL: https://issues.apache.org/jira/browse/YARN-8482 Project: Hadoop YARN Issue Type: Sub-task Reporter: Giovanni Matteo Fumarola -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8481) AMRMProxyPolicies should accept heartbeat response from new/unknown subclusters
Botong Huang created YARN-8481: -- Summary: AMRMProxyPolicies should accept heartbeat response from new/unknown subclusters Key: YARN-8481 URL: https://issues.apache.org/jira/browse/YARN-8481 Project: Hadoop YARN Issue Type: Bug Reporter: Botong Huang Assignee: Botong Huang Currently BroadcastAMRMProxyPolicy assumes that we only span the application to the sub-clusters instructed by itself via _splitResourceRequests_. However, with AMRMProxy HA, second attempts of the application might come up with multiple sub-clusters initially without consulting the AMRMProxyPolicy at all. This leads to exceptions in _notifyOfResponse._ It should simply allow the new/unknown sub-cluster heartbeat responses. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/ [Jun 28, 2018 5:37:22 AM] (aajisaka) HADOOP-15495. Upgrade commons-lang version to 3.7 in [Jun 28, 2018 5:58:40 AM] (aajisaka) HADOOP-14313. Replace/improve Hadoop's byte[] comparator. Contributed by [Jun 28, 2018 6:39:33 AM] (aengineer) HDDS-195. Create generic CommandWatcher utility. Contributed by Elek, [Jun 28, 2018 4:21:56 PM] (Bharat) HDFS-13705:The native ISA-L library loading failure should be made [Jun 28, 2018 4:39:49 PM] (eyang) YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService. [Jun 28, 2018 5:23:31 PM] (sunilg) YARN-8379. Improve balancing resources in already satisfied queues by [Jun 28, 2018 10:41:39 PM] (nanda) HDDS-185: TestCloseContainerByPipeline#testCloseContainerViaRatis fail [Jun 28, 2018 11:07:16 PM] (nanda) HDDS-178: DN should update transactionId on block delete. Contributed by -1 overall The following subsystems voted -1: asflicense findbugs pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed CTEST tests : test_test_libhdfs_threaded_hdfs_static test_libhdfs_threaded_hdfspp_test_shim_static Failed junit tests : hadoop.util.TestDiskChecker hadoop.util.TestBasicDiskValidator hadoop.hdfs.server.balancer.TestBalancer hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageEntities hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowActivity hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageApps hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageSchema hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageDomain hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.mapred.TestMRTimelineEventHandling cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/diff-compile-javac-root.txt [360K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/diff-checkstyle-root.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/diff-patch-shelldocs.txt [16K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/whitespace-eol.txt [9.4M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/whitespace-tabs.txt [1.1M] xml: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/xml.txt [4.0K] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/branch-findbugs-hadoop-hdds_client.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt [48K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/branch-findbugs-hadoop-hdds_tools.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/branch-findbugs-hadoop-ozone_client.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/branch-findbugs-hadoop-ozone_common.txt [24K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/826/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt [4.0K]
[jira] [Created] (YARN-8480) Add boolean option for resources
Daniel Templeton created YARN-8480: -- Summary: Add boolean option for resources Key: YARN-8480 URL: https://issues.apache.org/jira/browse/YARN-8480 Project: Hadoop YARN Issue Type: Improvement Reporter: Daniel Templeton Assignee: Szilard Nemeth Make it possible to define a resource with a boolean value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8479) The capacity scheduler logs too frequently seriously affecting performance
YunFan Zhou created YARN-8479: - Summary: The capacity scheduler logs too frequently seriously affecting performance Key: YARN-8479 URL: https://issues.apache.org/jira/browse/YARN-8479 Project: Hadoop YARN Issue Type: Sub-task Components: capacity scheduler, capacityscheduler Reporter: YunFan Zhou -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8478) The capacity scheduler logs too frequently, seriously affecting performance
YunFan Zhou created YARN-8478: - Summary: The capacity scheduler logs too frequently, seriously affecting performance Key: YARN-8478 URL: https://issues.apache.org/jira/browse/YARN-8478 Project: Hadoop YARN Issue Type: Bug Components: capacity scheduler, capacityscheduler Reporter: YunFan Zhou -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
