[
https://issues.apache.org/jira/browse/HADOOP-10203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13884015#comment-13884015
]
Hudson commented on HADOOP-10203:
---------------------------------
FAILURE: Integrated in Hadoop-Yarn-trunk #464 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/464/])
HADOOP-10203. Connection leak in Jets3tNativeFileSystemStore#retrieveMetadata.
Contributed by Andrei Savu. (atm:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1561720)
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/Jets3tNativeFileSystemStore.java
> Connection leak in Jets3tNativeFileSystemStore#retrieveMetadata
> ----------------------------------------------------------------
>
> Key: HADOOP-10203
> URL: https://issues.apache.org/jira/browse/HADOOP-10203
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Environment: CDH 2.0.0-cdh4.5.0
> (30821ec616ee7a21ee8447949b7c6208a8f1e7d8)
> Reporter: Andrei Savu
> Assignee: Andrei Savu
> Fix For: 2.4.0
>
> Attachments: HADOOP-10203-trunk.patch, HADOOP-10203.patch
>
>
> Jets3tNativeFileSystemStore#retrieveMetadata is leaking connections.
> This affects any client that tries to read many small files very quickly
> (e.g. distcp from s3 to hdfs with small files blocks due to connection pool
> starvation).
> This is not a problem for larger files because when the GC runs any
> connection that's out of scope will be released in #finalize().
> We are seeing the following log messages as a symptom of this problem:
> {noformat}
> 13/12/26 13:40:01 WARN httpclient.HttpMethodReleaseInputStream: Attempting to
> release HttpMethod in finalize() as its response data stream has gone out of
> scope. This attempt will not always succeed and cannot be relied upon! Please
> ensure response data streams are always fully consumed or closed to avoid
> HTTP connection starvation.
> 13/12/26 13:40:01 WARN httpclient.HttpMethodReleaseInputStream: Successfully
> released HttpMethod in finalize(). You were lucky this time... Please ensure
> response data streams are always fully consumed or closed.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
