[
https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15710216#comment-15710216
]
Aaron Fabbri commented on HADOOP-13793:
---------------------------------------
All s3a integration tests passed with default endpoint in US West 2.
Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate
Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractDelete
Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractDistCp
Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractGetFileStatus
Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractMkdir
Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractOpen
Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename
Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractRootDir
Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractSeek
Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContext
Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextCreateMkdir
Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextMainOperations
Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextStatistics
Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextURI
Running org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextUtil
Running org.apache.hadoop.fs.s3a.ITestBlockingThreadPoolExecutorService
Running org.apache.hadoop.fs.s3a.ITestS3AAWSCredentialsProvider
Running org.apache.hadoop.fs.s3a.ITestS3ABlockOutputArray
Running org.apache.hadoop.fs.s3a.ITestS3ABlockOutputByteBuffer
Running org.apache.hadoop.fs.s3a.ITestS3ABlockOutputDisk
Running org.apache.hadoop.fs.s3a.ITestS3ABlocksize
Running org.apache.hadoop.fs.s3a.ITestS3AConfiguration
Running org.apache.hadoop.fs.s3a.ITestS3ACredentialsInURL
Running org.apache.hadoop.fs.s3a.ITestS3AEncryption
Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionAlgorithmPropagation
Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionBlockOutputStream
Running org.apache.hadoop.fs.s3a.ITestS3AFailureHandling
Running org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost
Running org.apache.hadoop.fs.s3a.ITestS3AFileSystemContract
Running org.apache.hadoop.fs.s3a.ITestS3AMiscOperations
Running org.apache.hadoop.fs.s3a.ITestS3ATemporaryCredentials
Running org.apache.hadoop.fs.s3a.ITestS3ATestUtils
Running org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency
Running org.apache.hadoop.fs.s3a.scale.ITestS3ADeleteFilesOneByOne
Running org.apache.hadoop.fs.s3a.scale.ITestS3ADeleteManyFiles
Running org.apache.hadoop.fs.s3a.scale.ITestS3ADirectoryPerformance
Running org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesArrayBlocks
Running org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesByteBufferBlocks
Running org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesClassicOutput
Running org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesDiskBlocks
Running org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance
Running org.apache.hadoop.fs.s3a.yarn.ITestS3A
Running org.apache.hadoop.fs.s3a.yarn.ITestS3AMiniYarnCluster
Running org.apache.hadoop.fs.s3native.ITestInMemoryNativeS3FileSystemContract
...
Failed tests:
ITestJets3tNativeS3FileSystemContract>NativeS3FileSystemContractBaseTest.testListStatusForRoot:66
Root directory is not empty; expected:<0> but was:<4>
Tests in error:
ITestJets3tNativeS3FileSystemContract>FileSystemContractBaseTest.testLSRootDir:727->FileSystemContractBaseTest.assertListFilesFinds:742
ยป FileNotFound
Tests run: 492, Failures: 1, Errors: 1, Skipped: 103
> s3guard: add inconsistency injection, integration tests
> -------------------------------------------------------
>
> Key: HADOOP-13793
> URL: https://issues.apache.org/jira/browse/HADOOP-13793
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Aaron Fabbri
> Assignee: Aaron Fabbri
> Attachments: HADOOP-13793-HADOOP-13345.001.patch,
> HADOOP-13793-HADOOP-13345.002.patch
>
>
> Many of us share concerns that testing the consistency features of S3Guard
> will be difficult if we depend on the rare and unpredictable occurrence of
> actual inconsistency in S3 to exercise those code paths.
> I think we should have a mechanism for injecting failure to force exercising
> of the consistency codepaths in S3Guard.
> Requirements:
> - Integration tests that cause S3A to see the types of inconsistency we
> address with S3Guard.
> - These are deterministic integration tests.
> Unit tests are possible as well, if we were to stub out the S3Client. That
> may be less bang for the buck, though.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
