[
https://issues.apache.org/jira/browse/HADOOP-17409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331287#comment-17331287
]
Steve Loughran edited comment on HADOOP-17409 at 6/2/21, 1:49 PM:
------------------------------------------------------------------
IT's all in 3.3.1 just not needed any more. I plan to worry about how to cut it
from the code after the auditing PR HADOOP-17511 is in
Keeping in s3guard complicates our life -too many branches in the code of
with/without s3guard, and doubles the amount of test runs needed. IT also
complicates any attempt to do maintenance refactoring (see
https://github.com/steveloughran/engineering-proposals/blob/trunk/refactoring-s3a.md
)
It's going to take care to remove. I'm thinking of having a separate apache
branch where we remove different pieces one by one
# dynamodb (just have local for testing)
# the various tools (import, prune, fsck and their tests)
# Reading from s3guard in list/getFileStatus etc. Will need to tweak tests
which fail when s3guard is on-or just disable them
# writing to it.
# all S3guard only tests
# All parameterized tests with guarded/raw phases
was (Author: [email protected]):
IT's all in 3.3.1 just not needed any more. I plan to worry about how to cut it
from the code after the auditing PR HADOOP-17511 is in
Keeping in s3guard complicates our life -too many branches in the code of
with/without s3guard, and doubles the amount of test runs needed. IT also
complicates any attempt to do maintenance refactoring (see
https://github.com/steveloughran/engineering-proposals/blob/trunk/refactoring-s3a.md
)
It's going to take care to remove. I'm thinking of having a separate apache
branch where we remove different pieces one by one
# dynamodb (just have local for testing)
# the various tools (import, prune, fsck and their tests)
# Reading from s3guard in list/getFileStatus etc. Will need to tweak tests
which fail when s3guard is on-or just disable them
# writing to it.
# all S3guard only tests
# All parameterized tests with guarded/raw phases
# u
> Remove S3Guard - no longer needed
> ---------------------------------
>
> Key: HADOOP-17409
> URL: https://issues.apache.org/jira/browse/HADOOP-17409
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Priority: Major
>
> With Consistent S3, S3Guard is superfluous.
> stop developing it and wean people off it as soon as they can.
> Then we can worry about what to do in the code. It has gradually insinuated
> its way through the layers, especially things like multi-object delete
> handling (see HADOOP-17244). Things would be a lot simpler without it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
