[
https://issues.apache.org/jira/browse/HDFS-11846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022171#comment-16022171
]
Weiwei Yang edited comment on HDFS-11846 at 5/24/17 1:51 AM:
-------------------------------------------------------------
Thanks [~xyao] for reviewing this. I uploaded v2 patch which has following
changes
bq. OzoneClientUtils.java: Do we need a configure for HTTP Connection Timeout?
Added another property {{OZONE_CLIENT_CONNECTION_TIMEOUT_MS}} for the
connection timeout.
bq. OzoneBucket.java: we should ensure the outPutStream is closed properly in
the final section as well.
I have fixed this line, and I have reviewed all streams in these classes and
make sure they are closed in final statement as well.
bq. OzoneClient.java: unnecessary change.
Fixed
bq. OzoneClient.java: httppost-> httpPost?
I have fixed all such names in {{OzoneClient}}, {{OzoneVolume}} and
{{OzoneBucket}} to make sure they all be consistent.
bq. OzoneClient.java: I think we should use PoolingHttpClientConnectionManager
instead of creating a new connection for each request.
I tried to use PoolingHttpClientConnectionManager but I found it doesn't work
in current code base. The problem was on the netty server side. It doesn't
serve a reused http connection well, I have created HDFS-11873 for that.
Thank you.
was (Author: cheersyang):
Thanks [~xyao] for reviewing this. I uploaded v2 patch which has following
changes
bq. OzoneClientUtils.java: Do we need a configure for HTTP Connection Timeout?
Added another property {{OZONE_CLIENT_CONNECTION_TIMEOUT_MS}} for the
connection timeout.
bq. OzoneBucket.java: we should ensure the outPutStream is closed properly in
the final section as well.
I have fixed this line, and I have reviewed all streams in these classes and
make sure they are closed in final statement as well.
bq. OzoneClient.java: unnecessary change.
Fixed
bq. OzoneClient.java: httppost-> httpPost?
I have fixed all such names in {{OzoneClient}}, {{OzoneVolume}} and
{{OzoneBucket}} to make sure they all be consistent.
bq. OzoneClient.java: I think we should use PoolingHttpClientConnectionManager
instead of creating a new connection for each request.
I tried to use PoolingHttpClientConnectionManager but I found it doesn't work
in current code base. The problem was on the netty server side. It doesn't
serve a reused http connection well, I think we need a new JIRA to fix that.
Thank you.
> Ozone: Potential http connection leaks in ozone clients
> -------------------------------------------------------
>
> Key: HDFS-11846
> URL: https://issues.apache.org/jira/browse/HDFS-11846
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Attachments: HDFS-11846-HDFS-7240.001.patch,
> HDFS-11846-HDFS-7240.002.patch
>
>
> There are several problems
> # Http clients in {{OzoneVolume}}, {{OzoneBucket}} and {{OzoneClient}} are
> created per request, per [Reuse of HttpClient
> instance|http://hc.apache.org/httpclient-3.x/performance.html#Reuse_of_HttpClient_instance]
> doc, proposed to reuse the http client instance to reduce the over head.
> # Some resources in these classes were not properly cleaned up. E.g the http
> connection, HttpGet/HttpPost requests.
>
> This jira's purpose is to fix these issues and investigate how we can improve
> the client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
