[
https://issues.apache.org/jira/browse/HADOOP-17409?focusedWorklogId=683354&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-683354
]
ASF GitHub Bot logged work on HADOOP-17409:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 18/Nov/21 14:58
Start Date: 18/Nov/21 14:58
Worklog Time Spent: 10m
Work Description: mehakmeet commented on a change in pull request #3534:
URL: https://github.com/apache/hadoop/pull/3534#discussion_r752291332
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/InconsistentAmazonS3Client.java
##########
@@ -60,12 +55,13 @@
import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability;
import org.apache.hadoop.conf.Configuration;
-import org.apache.hadoop.fs.Path;
/**
* A wrapper around {@link com.amazonaws.services.s3.AmazonS3} that injects
- * inconsistency and/or errors. Used for testing S3Guard.
- * Currently only delays listing visibility, not affecting GET.
+ * failures.
+ * It used to also inject inconsistency, but this was removed with S3Guard;
+ * what is retained is the ability to throttle AWS operations and for the
+ * input stream to be inconsistent.
*/
@InterfaceAudience.Private
@InterfaceStability.Unstable
Review comment:
Should we remove some methods like `setFailureInjectionPolicy()` and
`clearInconsistency()` which are explicitly trying to create S3 client for
"s3guard"
##########
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFailureHandling.java
##########
@@ -171,14 +143,10 @@ public void testMultiObjectDeleteMissingEntriesSucceeds()
throws Throwable {
List<DeleteObjectsRequest.KeyVersion> keys = keysToDelete(
Review comment:
Ref of `TestPartialDeleteFailures` and `MultiObjectDeleteSupport` which
are removed now.
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java
##########
@@ -1014,6 +931,7 @@ private Constants() {
"2s";
Review comment:
Should mark S3GUARD_CONSISTENCY_RETRY_INTERVAL configs as Deprecated as
well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 683354)
Time Spent: 3h 10m (was: 3h)
> Remove S3Guard - no longer needed
> ---------------------------------
>
> Key: HADOOP-17409
> URL: https://issues.apache.org/jira/browse/HADOOP-17409
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> With Consistent S3, S3Guard is superfluous.
> stop developing it and wean people off it as soon as they can.
> Then we can worry about what to do in the code. It has gradually insinuated
> its way through the layers, especially things like multi-object delete
> handling (see HADOOP-17244). Things would be a lot simpler without it
> This work is being done in the feature branch HADOOP-17409-remove-s3guard
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
