I'll look at these tomorrow -I'd rather fix whatever problem there is rather than roll back a JAR change which is inevitably going to come back.
As extra motivation: no commits -> hadoop-azure until this is fixed. On Mon, 9 Nov 2020 at 21:19, Ayush Saxena <ayush...@gmail.com> wrote: > The failing Azure tests are being tracked at HADOOP-17325 > > https://issues.apache.org/jira/browse/HADOOP-17325 > > On Mon, 9 Nov 2020 at 23:02, Ahmed Hussein <a...@ahussein.me> wrote: > > > I created new Jiras for HDFS failures. Please consider doing the same for > > Yarn and Azure. > > For convenience, the list of failures in the qbt report is as follows: > > > > Test Result > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/ > > > > > (50 > > failures / -7) > > > > - > > > > > org.apache.hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination.testGetCachedDatanodeReport > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.hdfs.server.federation.router/TestRouterRpcMultiDestination/testGetCachedDatanodeReport/ > > > > > - > > > > > org.apache.hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination.testNamenodeMetrics > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.hdfs.server.federation.router/TestRouterRpcMultiDestination/testNamenodeMetrics/ > > > > > - > > > > > org.apache.hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination.testErasureCoding > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.hdfs.server.federation.router/TestRouterRpcMultiDestination/testErasureCoding/ > > > > > - > > > > > org.apache.hadoop.hdfs.server.datanode.TestBPOfferService.testMissBlocksWhenReregister > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.hdfs.server.datanode/TestBPOfferService/testMissBlocksWhenReregister/ > > > > > - > > > org.apache.hadoop.yarn.sls.TestReservationSystemInvariants.testSimulatorRunning[Testing > > with: SYNTH, > > > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler, > > (nodeFile null)] > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.yarn.sls/TestReservationSystemInvariants/testSimulatorRunning_Testing_with__SYNTH__org_apache_hadoop_yarn_server_resourcemanager_scheduler_fair_FairScheduler___nodeFile_null__/ > > > > > - > > org.apache.hadoop.yarn.sls.appmaster.TestAMSimulator.testAMSimulator[1] > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.yarn.sls.appmaster/TestAMSimulator/testAMSimulator_1_/ > > > > > - > > > > > org.apache.hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer.testTokenThreadTimeout > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.yarn.server.resourcemanager.security/TestDelegationTokenRenewer/testTokenThreadTimeout/ > > > > > - > > > > > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithOpportunisticContainers > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.yarn.applications.distributedshell/TestDistributedShell/testDSShellWithOpportunisticContainers/ > > > > > - > > > > > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithEnforceExecutionType > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.yarn.applications.distributedshell/TestDistributedShell/testDSShellWithEnforceExecutionType/ > > > > > - org.apache.hadoop.fs.azure.TestBlobMetadata.testFolderMetadata > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestBlobMetadata/testFolderMetadata/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestBlobMetadata.testFirstContainerVersionMetadata > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestBlobMetadata/testFirstContainerVersionMetadata/ > > > > > - org.apache.hadoop.fs.azure.TestBlobMetadata.testPermissionMetadata > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestBlobMetadata/testPermissionMetadata/ > > > > > - > org.apache.hadoop.fs.azure.TestBlobMetadata.testOldPermissionMetadata > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestBlobMetadata/testOldPermissionMetadata/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemConcurrency.testNoTempBlobsVisible > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemConcurrency/testNoTempBlobsVisible/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemConcurrency.testLinkBlobs > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemConcurrency/testLinkBlobs/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testListStatusRootDir > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testListStatusRootDir/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testRenameDirectoryMoveToExistingDirectory > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testRenameDirectoryMoveToExistingDirectory/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testListStatus > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testListStatus/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testRenameDirectoryAsExistingDirectory > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testRenameDirectoryAsExistingDirectory/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testRenameToDirWithSamePrefixAllowed > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testRenameToDirWithSamePrefixAllowed/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testLSRootDir > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testLSRootDir/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked.testDeleteRecursively > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemContractMocked/testDeleteRecursively/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck.testWasbFsck > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemFileNameCheck/testWasbFsck/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testChineseCharactersFolderRename > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testChineseCharactersFolderRename/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRedoRenameFolderInFolderListingWithZeroByteRenameMetadata > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRedoRenameFolderInFolderListingWithZeroByteRenameMetadata/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRedoRenameFolderInFolderListing > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRedoRenameFolderInFolderListing/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testUriEncoding > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testUriEncoding/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testDeepFileCreation > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testDeepFileCreation/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testListDirectory > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testListDirectory/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRedoRenameFolderRenameInProgress > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRedoRenameFolderRenameInProgress/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRenameFolder > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRenameFolder/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRenameImplicitFolder > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRenameImplicitFolder/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRedoRenameFolder > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRedoRenameFolder/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testStoreDeleteFolder > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testStoreDeleteFolder/ > > > > > - > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked.testRename > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemMocked/testRename/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testListStatus > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testListStatus/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testRenameDirectoryAsEmptyDirectory > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testRenameDirectoryAsEmptyDirectory/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testListStatusFilterWithSomeMatches > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testListStatusFilterWithSomeMatches/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testRenameDirectoryAsNonExistentDirectory > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testRenameDirectoryAsNonExistentDirectory/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testGlobStatusSomeMatchesInDirectories > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testGlobStatusSomeMatchesInDirectories/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testGlobStatusWithMultipleWildCardMatches > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testGlobStatusWithMultipleWildCardMatches/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked.testDeleteRecursively > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestNativeAzureFileSystemOperationsMocked/testDeleteRecursively/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testImplicitFolderDeleted > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testImplicitFolderDeleted/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testFileAndImplicitFolderSameName > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testFileAndImplicitFolderSameName/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testSetOwnerOnImplicitFolder > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testSetOwnerOnImplicitFolder/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testFileInImplicitFolderDeleted > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testFileInImplicitFolderDeleted/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testImplicitFolderListed > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testImplicitFolderListed/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testCreatingDeepFileCreatesExplicitFolder > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testCreatingDeepFileCreatesExplicitFolder/ > > > > > - > > > > > org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations.testSetPermissionOnImplicitFolder > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestOutOfBandAzureBlobOperations/testSetPermissionOnImplicitFolder/ > > > > > - org.apache.hadoop.fs.azure.TestWasbFsck.testDelete > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/testReport/junit/org.apache.hadoop.fs.azure/TestWasbFsck/testDelete/ > > > > > > > > > On Mon, Nov 9, 2020 at 11:24 AM Ahmed Hussein <a...@ahussein.me> wrote: > > > > > Hello folks, > > > > > > Over the last month, there has been concern about the stability of > > Hadoop. > > > Looking at the latest QBT report (Nov 8th, 2020 1:39 AM) > > > < > > > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/319/#showFailuresLink > > >, > > > there were 50 failing tests, 41 of which are in "hadoop-azure" module. > > > Thanks to the effort of the community, the yetus qbt report looks > better > > > by miles. However, it will be highly appreciated if some developers > > > volunteer some time to take a look at the hadoop-azure. > > > > > > If tests in fs.azure are irrelevant to active contributors, then please > > > consider disabling those tests to save resources and avoid side effects > > of > > > those failures on the other modules (busy CPUs, memory release, ports > > > listening..etc). > > > > > > Thank you. > > > > > > -- > > > Best Regards, > > > > > > *Ahmed Hussein, PhD* > > > > > > > > > -- > > Best Regards, > > > > *Ahmed Hussein, PhD* > > >
