[
https://issues.apache.org/jira/browse/HDFS-11887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036933#comment-16036933
]
Mukul Kumar Singh commented on HDFS-11887:
------------------------------------------
Thanks for your comments [~cheersyang], I really appreciate them. I thought
about your comments and my reply is as follows.
1) A client is leaked if and only if it is not closed. So as you pointed out,
if the client with active reference is evicted from cache, then it will not be
closed on eviction. However please note that this client will be closed as part
of {{releaseClient}} on this client, which will decrement the reference count
and will call {{cleanup}}
2) For the second part of the comment, the existing client connection will not
be closed because there is a reference count on the client and hence the "older
instance" will not get connection refused error because of this reference count.
3) To answer that question that the cache should keep all the active clients in
the cache. I feel that it is impossible for a cache with finite size to
guarantee that, for example in the attached patch, the cache max size is 256
connections(configurable). If all of these connections are active and a new
client is created, that will definitely cause eviction of one of the existing
connections even when it has a reference count on it. So the best cache
behavior can either provide a MFU/LRU cache behavior to cache these elements.
4) This patch ensures that the client is closed when both it has been evicted
from cache as well as there no active users of the client. Hence ensuring that
there are no leaks.
Again I really appreciate all the comments and please point me to any issues
which I may overlook or have missed.
We can also setup a discussion to discuss this in detail. Thanks for your
feedback :)
> XceiverClientManager should close XceiverClient on eviction from cache
> ----------------------------------------------------------------------
>
> Key: HDFS-11887
> URL: https://issues.apache.org/jira/browse/HDFS-11887
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Mukul Kumar Singh
> Assignee: Mukul Kumar Singh
> Attachments: HDFS-11887-HDFS-7240.001.patch,
> HDFS-11887-HDFS-7240.002.patch
>
>
> XceiverClientManager doesn't close client on eviction which can leak
> resources.
> {code}
> public XceiverClientManager(Configuration conf) {
> .
> .
> .
> public void onRemoval(
> RemovalNotification<String, XceiverClientWithAccessInfo>
> removalNotification) {
> // If the reference count is not 0, this xceiver client should
> not
> // be evicted, add it back to the cache.
> WithAccessInfo info = removalNotification.getValue();
> if (info.hasRefence()) {
> synchronized (XceiverClientManager.this.openClient) {
> XceiverClientManager.this
> .openClient.put(removalNotification.getKey(), info);
> }
> }
> {code}
> Also a stack overflow can be triggered because of putting the element back in
> the cache on eviction.
> {code}
> synchronized (XceiverClientManager.this.openClient) {
> XceiverClientManager.this
> .openClient.put(removalNotification.getKey(), info);
> }
> {code}
> This bug will try to fix both of these cases.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
