The failing Azure tests are being tracked at HADOOP-17325 https://issues.apache.org/jira/browse/HADOOP-17325
On Mon, 9 Nov 2020 at 23:02, Ahmed Hussein <a...@ahussein.me> wrote: > I created new Jiras for HDFS failures. Please consider doing the same for > Yarn and Azure. > For convenience, the list of failures in the qbt report is as follows: > > Test Result > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/ > > > (50 > failures / -7) > > - > > > org.apache.hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination.testGetCachedDatanodeReport > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.hdfs.server.federation.router/TestRouterRpcMultiDestination/testGetCachedDatanodeReport/ > > > - > > > org.apache.hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination.testNamenodeMetrics > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.hdfs.server.federation.router/TestRouterRpcMultiDestination/testNamenodeMetrics/ > > > - > > > org.apache.hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination.testErasureCoding > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.hdfs.server.federation.router/TestRouterRpcMultiDestination/testErasureCoding/ > > > - > > > org.apache.hadoop.hdfs.server.datanode.TestBPOfferService.testMissBlocksWhenReregister > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.hdfs.server.datanode/TestBPOfferService/testMissBlocksWhenReregister/ > > > - > org.apache.hadoop.yarn.sls.TestReservationSystemInvariants.testSimulatorRunning[Testing > with: SYNTH, > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler, > (nodeFile null)] > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.yarn.sls/TestReservationSystemInvariants/testSimulatorRunning_Testing_with__SYNTH__org_apache_hadoop_yarn_server_resourcemanager_scheduler_fair_FairScheduler___nodeFile_null__/ > > > - > org.apache.hadoop.yarn.sls.appmaster.TestAMSimulator.testAMSimulator[1] > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.yarn.sls.appmaster/TestAMSimulator/testAMSimulator_1_/ > > > - > > > org.apache.hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer.testTokenThreadTimeout > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.yarn.server.resourcemanager.security/TestDelegationTokenRenewer/testTokenThreadTimeout/ > > > - > > > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithOpportunisticContainers > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.yarn.applications.distributedshell/TestDistributedShell/testDSShellWithOpportunisticContainers/ > > > - > > > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithEnforceExecutionType > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.yarn.applications.distributedshell/TestDistributedShell/testDSShellWithEnforceExecutionType/ > > > - org.apache.hadoop.fs.azure.TestBlobMetadata.testFolderMetadata > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestBlobMetadata/testFolderMetadata/ > > > - > > org.apache.hadoop.fs.azure.TestBlobMetadata.testFirstContainerVersionMetadata > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestBlobMetadata/testFirstContainerVersionMetadata/ > > > - org.apache.hadoop.fs.azure.TestBlobMetadata.testPermissionMetadata > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestBlobMetadata/testPermissionMetadata/ > > > - org.apache.hadoop.fs.azure.TestBlobMetadata.testOldPermissionMetadata > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestBlobMetadata/testOldPermissionMetadata/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemConcurrency.testNoTempBlobsVisible > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemConcurrency/testNoTempBlobsVisible/ > > > - > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemConcurrency.testLinkBlobs > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemConcurrency/testLinkBlobs/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testListStatusRootDir > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testListStatusRootDir/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testRenameDirectoryMoveToExistingDirectory > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testRenameDirectoryMoveToExistingDirectory/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testListStatus > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testListStatus/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testRenameDirectoryAsExistingDirectory > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testRenameDirectoryAsExistingDirectory/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testRenameToDirWithSamePrefixAllowed > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testRenameToDirWithSamePrefixAllowed/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testLSRootDir > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testLSRootDir/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testDeleteRecursively > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testDeleteRecursively/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck.testWasbFsck > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemFileNameCheck/testWasbFsck/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testChineseCharactersFolderRename > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testChineseCharactersFolderRename/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRedoRenameFolderInFolderListingWithZeroByteRenameMetadata > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRedoRenameFolderInFolderListingWithZeroByteRenameMetadata/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRedoRenameFolderInFolderListing > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRedoRenameFolderInFolderListing/ > > > - > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testUriEncoding > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testUriEncoding/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testDeepFileCreation > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testDeepFileCreation/ > > > - > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testListDirectory > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testListDirectory/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRedoRenameFolderRenameInProgress > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRedoRenameFolderRenameInProgress/ > > > - > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRenameFolder > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRenameFolder/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRenameImplicitFolder > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRenameImplicitFolder/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRedoRenameFolder > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRedoRenameFolder/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testStoreDeleteFolder > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testStoreDeleteFolder/ > > > - org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRename > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRename/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testListStatus > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testListStatus/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testRenameDirectoryAsEmptyDirectory > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testRenameDirectoryAsEmptyDirectory/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testListStatusFilterWithSomeMatches > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testListStatusFilterWithSomeMatches/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testRenameDirectoryAsNonExistentDirectory > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testRenameDirectoryAsNonExistentDirectory/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testGlobStatusSomeMatchesInDirectories > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testGlobStatusSomeMatchesInDirectories/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testGlobStatusWithMultipleWildCardMatches > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testGlobStatusWithMultipleWildCardMatches/ > > > - > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testDeleteRecursively > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testDeleteRecursively/ > > > - > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testImplicitFolderDeleted > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testImplicitFolderDeleted/ > > > - > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testFileAndImplicitFolderSameName > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testFileAndImplicitFolderSameName/ > > > - > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testSetOwnerOnImplicitFolder > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testSetOwnerOnImplicitFolder/ > > > - > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testFileInImplicitFolderDeleted > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testFileInImplicitFolderDeleted/ > > > - > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testImplicitFolderListed > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testImplicitFolderListed/ > > > - > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testCreatingDeepFileCreatesExplicitFolder > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testCreatingDeepFileCreatesExplicitFolder/ > > > - > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testSetPermissionOnImplicitFolder > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testSetPermissionOnImplicitFolder/ > > > - org.apache.hadoop.fs.azure.TestWasbFsck.testDelete > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestWasbFsck/testDelete/ > > > > > On Mon, Nov 9, 2020 at 11:24 AM Ahmed Hussein <a...@ahussein.me> wrote: > > > Hello folks, > > > > Over the last month, there has been concern about the stability of > Hadoop. > > Looking at the latest QBT report (Nov 8th, 2020 1:39 AM) > > < > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/#showFailuresLink > >, > > there were 50 failing tests, 41 of which are in "hadoop-azure" module. > > Thanks to the effort of the community, the yetus qbt report looks better > > by miles. However, it will be highly appreciated if some developers > > volunteer some time to take a look at the hadoop-azure. > > > > If tests in fs.azure are irrelevant to active contributors, then please > > consider disabling those tests to save resources and avoid side effects > of > > those failures on the other modules (busy CPUs, memory release, ports > > listening..etc). > > > > Thank you. > > > > -- > > Best Regards, > > > > *Ahmed Hussein, PhD* > > > > > -- > Best Regards, > > *Ahmed Hussein, PhD* >
