[
https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17745161#comment-17745161
]
Steve Loughran edited comment on HADOOP-18073 at 7/20/23 4:21 PM:
------------------------------------------------------------------
rebase branches hightlight spotbugs and javadoc issues, both minor, plus style
{code}
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/ProgressableProgressListener.java:80:
warning: no @param for upload
{code}
and something that needs review, probably a spotbugs disable if we are happy
its a false alarm
{code}
Code Warning
IS Inconsistent synchronization of
org.apache.hadoop.fs.s3a.S3AFileSystem.s3AsyncClient; locked 60% of time
Bug type IS2_INCONSISTENT_SYNC (click for details)
In class org.apache.hadoop.fs.s3a.S3AFileSystem
Field org.apache.hadoop.fs.s3a.S3AFileSystem.s3AsyncClient
Synchronized 60% of the time
Unsynchronized access at S3AFileSystem.java:[line 1764]
Unsynchronized access at S3AFileSystem.java:[line 989]
Synchronized access at S3AFileSystem.java:[line 4179]
Synchronized access at S3AFileSystem.java:[line 4184]
Synchronized access at S3AFileSystem.java:[line 1002]
{code}
{code}
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/impl/ActiveAuditManagerS3A.java:413:
//
https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/interceptor/ExecutionInterceptor.html:
Line is longer than 100 characters (found 115). [LineLength]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java:128:
private <BuilderT extends S3BaseClientBuilder<BuilderT, ClientT>, ClientT>
BuilderT configureClientBuilder(: Line is longer than 100 characters (found
109). [LineLength]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSHeaders.java:24:public
interface AWSHeaders {: interfaces should describe a type and hence have
methods. [InterfaceIsType]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSHeaders.java:46:
/** S3's version ID header */: First sentence should end with a period.
[JavadocStyle]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSHeaders.java:49:
/** Header describing what class of storage a user wants */: First sentence
should end with a period. [JavadocStyle]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSHeaders.java:52:
/** Header describing what archive tier the object is in, if any */: First
sentence should end with a period. [JavadocStyle]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSHeaders.java:55:
/** Header for optional server-side encryption algorithm */: First sentence
should end with a period. [JavadocStyle]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSHeaders.java:58:
/** Range header for the get object request */: First sentence should end
with a period. [JavadocStyle]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSHeaders.java:68:
/** JSON-encoded description of encryption materials used during encryption
*/: First sentence should end with a period. [JavadocStyle]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSHeaders.java:71:
/** Header for the optional restore information of an object */: First
sentence should end with a period. [JavadocStyle]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/InconsistentS3ClientFactory.java:68:
FailureInjectionInterceptor(FailureInjectionPolicy policy) {: 'ctor def
modifier' has incorrect indentation level 3, expected level should be 4.
[Indentation]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/select/BlockingEnumeration.java:57:
private final Signal<T> END_SIGNAL = new Signal<>((Throwable)null);:27: Name
'END_SIGNAL' must match pattern '^[a-z][a-zA-Z0-9]*$'. [MemberName]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/AbstractS3AMockTest.java:57:
protected S3Client s3;:22: Variable 's3' must be private and have accessor
methods. [VisibilityModifier]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/audit/AbstractAuditingTest.java:28:import
java.util.function.Consumer;:8: Unused import - java.util.function.Consumer.
[UnusedImports]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/StreamPublisher.java:38:
public StreamPublisher(Stream<T> data, Executor executor) {:3: Redundant
'public' modifier. [RedundantModifier]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/StreamPublisher.java:43:
public StreamPublisher(Stream<T> data) {:3: Redundant 'public' modifier.
[RedundantModifier]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/TestSelectEventStreamPublisher.java:159:
.map(e -> { throw SdkException.create("error!", null); }));:19: '{' at
column 19 should have line break after. [LeftCurly]
{code}
was (Author: [email protected]):
rebase branches hightlight spotbugs and javadoc issues, both minor
{code}
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/ProgressableProgressListener.java:80:
warning: no @param for upload
{code}
and something that needs review, probably a spotbugs disable if we are happy
its a false alarm
{code}
Code Warning
IS Inconsistent synchronization of
org.apache.hadoop.fs.s3a.S3AFileSystem.s3AsyncClient; locked 60% of time
Bug type IS2_INCONSISTENT_SYNC (click for details)
In class org.apache.hadoop.fs.s3a.S3AFileSystem
Field org.apache.hadoop.fs.s3a.S3AFileSystem.s3AsyncClient
Synchronized 60% of the time
Unsynchronized access at S3AFileSystem.java:[line 1764]
Unsynchronized access at S3AFileSystem.java:[line 989]
Synchronized access at S3AFileSystem.java:[line 4179]
Synchronized access at S3AFileSystem.java:[line 4184]
Synchronized access at S3AFileSystem.java:[line 1002]
{code}
> Upgrade AWS SDK to v2
> ---------------------
>
> Key: HADOOP-18073
> URL: https://issues.apache.org/jira/browse/HADOOP-18073
> Project: Hadoop Common
> Issue Type: Task
> Components: auth, fs/s3
> Affects Versions: 3.3.1
> Reporter: xiaowei sun
> Assignee: Ahmar Suhail
> Priority: Major
> Labels: pull-request-available
> Attachments: Upgrading S3A to SDKV2.pdf
>
>
> This task tracks upgrading Hadoop's AWS connector S3A from AWS SDK for Java
> V1 to AWS SDK for Java V2.
> Original use case:
> {quote}We would like to access s3 with AWS SSO, which is supported in
> software.amazon.awssdk:sdk-core:2.*.
> In particular, from
> [https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html],
> when to set 'fs.s3a.aws.credentials.provider', it must be
> "com.amazonaws.auth.AWSCredentialsProvider". We would like to support
> "software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider" which
> supports AWS SSO, so users only need to authenticate once.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
