[
https://issues.apache.org/jira/browse/HADOOP-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18857:
------------------------------------
Description:
if the v2 signer is requested on an AWS SDKv2 build, you get an NPE.
proposed: special recognition of fs.s3a.signing-algorithm=S3SignerType and
fail with a message that it isn't supported
{code}
> bin/hadoop jar $CLOUDSTORE storediag -D fs.s3a.signing-algorithm=S3SignerType
> -w s3a://stevel-london/
2023-08-21 14:36:41,229 [main] INFO diag.StoreDiag
(StoreDurationInfo.java:close(137)) - Duration of Creating filesystem for
s3a://stevel-london/: 0:00:357
java.lang.NullPointerException
at
org.apache.hadoop.fs.s3a.auth.SignerFactory.createSigner(SignerFactory.java:104)
at
org.apache.hadoop.fs.s3a.impl.AWSClientConfig.createClientConfigBuilder(AWSClientConfig.java:90)
at
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.createClientOverrideConfiguration(DefaultS3ClientFactory.java:163)
at
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.configureClientBuilder(DefaultS3ClientFactory.java:147)
at
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.createS3Client(DefaultS3ClientFactory.java:89)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.bindAWSClient(S3AFileSystem.java:989)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:610)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3601)
{code}
was:
if the v2 signer is requested on an AWS SDKv2 build, you get an NPE.
proposed: special recogni
{code}
2023-08-21 14:36:41,229 [main] INFO diag.StoreDiag
(StoreDurationInfo.java:close(137)) - Duration of Creating filesystem for
s3a://stevel-london/: 0:00:357
java.lang.NullPointerException
at
org.apache.hadoop.fs.s3a.auth.SignerFactory.createSigner(SignerFactory.java:104)
at
org.apache.hadoop.fs.s3a.impl.AWSClientConfig.createClientConfigBuilder(AWSClientConfig.java:90)
at
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.createClientOverrideConfiguration(DefaultS3ClientFactory.java:163)
at
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.configureClientBuilder(DefaultS3ClientFactory.java:147)
at
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.createS3Client(DefaultS3ClientFactory.java:89)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.bindAWSClient(S3AFileSystem.java:989)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:610)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3601)
{code}
> AWS v2 SDK: fail meaningfully if legacy v2 signing requested
> ------------------------------------------------------------
>
> Key: HADOOP-18857
> URL: https://issues.apache.org/jira/browse/HADOOP-18857
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Priority: Major
>
> if the v2 signer is requested on an AWS SDKv2 build, you get an NPE.
> proposed: special recognition of fs.s3a.signing-algorithm=S3SignerType and
> fail with a message that it isn't supported
> {code}
> > bin/hadoop jar $CLOUDSTORE storediag -D
> > fs.s3a.signing-algorithm=S3SignerType -w s3a://stevel-london/
> 2023-08-21 14:36:41,229 [main] INFO diag.StoreDiag
> (StoreDurationInfo.java:close(137)) - Duration of Creating filesystem for
> s3a://stevel-london/: 0:00:357
> java.lang.NullPointerException
> at
> org.apache.hadoop.fs.s3a.auth.SignerFactory.createSigner(SignerFactory.java:104)
> at
> org.apache.hadoop.fs.s3a.impl.AWSClientConfig.createClientConfigBuilder(AWSClientConfig.java:90)
> at
> org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.createClientOverrideConfiguration(DefaultS3ClientFactory.java:163)
> at
> org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.configureClientBuilder(DefaultS3ClientFactory.java:147)
> at
> org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.createS3Client(DefaultS3ClientFactory.java:89)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.bindAWSClient(S3AFileSystem.java:989)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:610)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3601)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
