[
https://issues.apache.org/jira/browse/HADOOP-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659090#action_12659090
]
Steve Loughran commented on HADOOP-4888:
----------------------------------------
I use HttpClient a lot in other projects; It does understand HTTP better than
java.net. But if you have implemented both ends of the communication and have
no IIS-based proxy in between, its needs are less urgent.
If you do use it, here are my ivy.xml settings used to drop all the logging
and junit dependencies that the POMs imply are mandatory. You just need to run
with commons-logging JAR on the classpath, nothing else
<dependency org="commons-httpclient"
name="commons-httpclient"
rev="${commons-httpclient.version}"
conf="compile->master;httpclient->default">
<exclude org="commons-logging"/>
<exclude org="junit"/>
</dependency>
HttpcCient does work with S3, because Restlet uses HttpClient, and restlet
talks to S3. There is no built in support in HttpClient for AWS authentication,
though no doubt this feature would be welcomed by many.
> Use Apache HttpClient for fetching map outputs
> ----------------------------------------------
>
> Key: HADOOP-4888
> URL: https://issues.apache.org/jira/browse/HADOOP-4888
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Attachments: 4888-0.patch
>
>
> It's worth experimenting with the
> [HttpClient|http://hc.apache.org/httpclient-3.x/] library to speed up the
> shuffle.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
