[
https://issues.apache.org/jira/browse/HADOOP-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jagane Sundar updated HADOOP-8136:
----------------------------------
Target Version/s: 1.0.2
Status: Patch Available (was: Open)
Here's my first cut at a patch for this jira. Some notes:
1. Most of the changes are on account of the jets3t-0.8.1 library throwing a
ServiceException, instead of a S3ServiceException.
2. There is also a piece of brittle code that was comparing an exception string
with 'ResponseCode=404' to decide whether the S3 server threw a 404 error. I
replaced that with a more rational test of response code.
3. I turned down the logging by setting
org.apache.hadoop.fs.s3native=ERROR
org.jets3t.service.impl.rest.httpclient.RestStorageService=ERROR
in log4j.properties
> Enhance hadoop to use a newer version (0.8.1) of the jets3t library
> -------------------------------------------------------------------
>
> Key: HADOOP-8136
> URL: https://issues.apache.org/jira/browse/HADOOP-8136
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
> Affects Versions: 1.0.0, 0.22.0, 0.23.3
> Environment: Ubuntu 11.04, 64 bit, JDK 1.6.0_30
> Reporter: Jagane Sundar
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Hadoop is built against, and includes, an older version of the Jets3t library
> - version 0.6.1.
> The current version of the Jets3t library(as of March 2012) is 0.8.1. This
> new version includes many improvements such as support for "Requester-Pays"
> buckets.
> Since hadoop includes a copy of the version 0.6.1 jets3t library, and since
> this version ends up early in the CLASSPATH, any Map Reduce application that
> wants to use the jets3t library ends up getting version 0.6.1 of the jets3t
> library. The MR application fails, usually with an error stating that the
> method signature of some method in the Jets3t library does not match.
> It would be useful to enhance Jets3tNativeFileSystemStore.java to use the API
> published by the 0.8.1 version of the jets3t library.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
