[
https://issues.apache.org/jira/browse/HADOOP-9680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13697089#comment-13697089
]
Robert Gibbon commented on HADOOP-9680:
---------------------------------------
I took a look at your patch in HADOOP-9623. Some comments:
* Bucket keyspace listings running over a s3-native fs will be broken by your
patch, they make use of the method
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieveMetadata(String
key) when querying S3 for the given URI; if the URI does not correspond to a
single key in a bucket (ie a single object) an exception is thrown. In the
above mentioned method, the exception's Message property was being parsed for
the string "ResponseCode=404" to interpret that the URI is not a single key. If
the condition is met, it returns null. Its a horrible piece of code and a very
poorly defined contract with the calling party. It is also broken by jets3t
0.9.0, which doesn't pass back that message anymore in that situation. I
adapted it to look at the ResponseCode property for the integer 404 instead,
but someone who knows that code better than me would do a good deed to fix it
more sustainably.
* I needed to upgrade jets3t to 0.9.0 because I need support for AWS IAM
federated access tokens (temporary, time limited access credentials, tied to a
session ticket). I don't see any support for that in the patch in HADOOP-9623,
for me its of no value unless it supports temp security tokens.
I think we're aligned on the need for an uprev of the jets3t implementation in
hadoop.
HTH
> Extend S3FS and S3NativeFS to work with AWS IAM Temporary Security Credentials
> ------------------------------------------------------------------------------
>
> Key: HADOOP-9680
> URL: https://issues.apache.org/jira/browse/HADOOP-9680
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
> Reporter: Robert Gibbon
> Priority: Minor
> Attachments: s3fs-temp-iam-creds.diff.patch
>
>
> Here is a patch in unified diff format to enable Amazon Web Services IAM
> Temporary Security Credentials secured interactions with S3 from Hadoop.
> It bumps the JetS3t release version up to 0.9.0.
> To use a temporary security credential set, you need to provide the following
> properties, depending on the implementation (s3 or s3native):
> fs.s3.awsAccessKeyId or fs.s3n.awsAccessKeyId - the temporary access key id
> issued by AWS IAM
> fs.s3.awsSecretAccessKey or fs.s3n.awsSecretAccessKey - the temporary secret
> access key issued by AWS IAM
> fs.s3.awsSessionToken or fs.s3n.awsSessionToken - the session ticket issued
> by AWS IAM along with the temporary key
> fs.s3.awsTokenFriendlyName or fs.s3n.awsTokenFriendlyName - any string
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
