[
https://issues.apache.org/jira/browse/HADOOP-18975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17809271#comment-17809271
]
Viraj Jasani commented on HADOOP-18975:
---------------------------------------
{code:java}
<property>
<name>fs.s3a.bucket.landsat-pds.endpoint.fips</name>
<value>true</value>
<description>Use the fips endpoint</description>
</property> {code}
[[email protected]] [~ahmar] do we really need fips enabled for landsat in
hadoop-tools/hadoop-aws/src/test/resources/core-site.xml ?
This is breaking several tests from full suite that i am running against
us-west-2 for PR [https://github.com/apache/hadoop/pull/6479]
e.g.
{code:java}
[ERROR]
testSelectOddRecordsIgnoreHeaderV1(org.apache.hadoop.fs.s3a.select.ITestS3Select)
Time elapsed: 2.917 s <<< ERROR!
java.lang.IllegalArgumentException: An endpoint cannot set when
fs.s3a.endpoint.fips is true : https://s3-us-west-2.amazonaws.com
at
org.apache.hadoop.util.Preconditions.checkArgument(Preconditions.java:213)
at
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.configureEndpointAndRegion(DefaultS3ClientFactory.java:292)
at
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.configureClientBuilder(DefaultS3ClientFactory.java:179)
at
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.createS3Client(DefaultS3ClientFactory.java:126)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.bindAWSClient(S3AFileSystem.java:1063)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:677)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3601)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:171)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3702)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3653)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:555)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:366)
at
org.apache.hadoop.fs.s3a.select.AbstractS3SelectTest.setup(AbstractS3SelectTest.java:304)
at
org.apache.hadoop.fs.s3a.select.ITestS3Select.setup(ITestS3Select.java:112)
{code}
> AWS SDK v2: extend support for FIPS endpoints
> ----------------------------------------------
>
> Key: HADOOP-18975
> URL: https://issues.apache.org/jira/browse/HADOOP-18975
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
>
> v1 SDK supported FIPS just by changing the endpoint.
> Now we have a new builder setting to use.
> * add new fs.s3a.endpoint.fips option
> * pass it down
> * test
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
