[
https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15635880#comment-15635880
]
Steve Loughran edited comment on HADOOP-13793 at 11/4/16 10:05 AM:
-------------------------------------------------------------------
If you look at what's gone in HADOOP-13560, I moved all the s3 client
operations out of the new output stream, and into S3aFileSystem. That is: no
direct access/use of the AWS client lib.
if that is also done for s3guard calls, then inconsistency can be mocked by a
subclass of S3AFileSystem whose low-level list call is overridden to return
inconsistent data. It may also help future maintenance
was (Author: [email protected]):
If you look at what's gone in HADOOP-13560, I moved all the s3 client
operations out of the new output stream, and into S3aFileSystem. That is: no
direct access/use of the AWS client lib.
if that is also done for s3guard calls, then inconsitency can be mocked by a
subclass of S3AFileSystem whose low-level list call is overridden to return
inconsistent data
> s3guard: add inconsistency injection, integration tests
> -------------------------------------------------------
>
> Key: HADOOP-13793
> URL: https://issues.apache.org/jira/browse/HADOOP-13793
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Aaron Fabbri
>
> Many of us share concerns that testing the consistency features of S3Guard
> will be difficult if we depend on the rare and unpredictable occurrence of
> actual inconsistency in S3 to exercise those code paths.
> I think we should have a mechanism for injecting failure to force exercising
> of the consistency codepaths in S3Guard.
> Requirements:
> - Integration tests that cause S3A to see the types of inconsistency we
> address with S3Guard.
> - These are deterministic integration tests.
> Unit tests are possible as well, if we were to stub out the S3Client. That
> may be less bang for the buck, though.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
