[
https://issues.apache.org/jira/browse/HDFS-16876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simbarashe Dzinamarira updated HDFS-16876:
------------------------------------------
Description:
An element in RouterStateIdContext#namespaceIdMap is deleted when there is no
connectionPool referencing the namespace. This is done by a thread in
ConnectionManager that cleans up stale connectionPools as well.
I propose a less aggressive approach, that is, cleaning up an entry when the
router cannot resolve a namenode belonging to the namespace.
The benefit of this approach is that even when there are no active connection,
the router still track a recent state of the namenode. This will be beneficial
for debugging.
was:
When cleaning up stale connection pools, the ConnectionManager also removes the
associated namespace to stateId mapping from the shared RouterStateIdContext. A
mapping is only removed when there is no other connectionPool referencing it.
When a new connection is created, the stateId mapping gets recreated.
Since the RouterStateIdContext is shared across connections, it is cleaner to
drive its cleanup using the list of namespaces the router references instead of
closely tracking the active connections.
> Garbage collect map entries in shared RouterStateIdContext using information
> from namenodeResolver instead of the map of active connectionPools.
> ------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-16876
> URL: https://issues.apache.org/jira/browse/HDFS-16876
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
> Reporter: Simbarashe Dzinamarira
> Assignee: Simbarashe Dzinamarira
> Priority: Critical
> Labels: pull-request-available
>
> An element in RouterStateIdContext#namespaceIdMap is deleted when there is no
> connectionPool referencing the namespace. This is done by a thread in
> ConnectionManager that cleans up stale connectionPools as well.
> I propose a less aggressive approach, that is, cleaning up an entry when the
> router cannot resolve a namenode belonging to the namespace.
> The benefit of this approach is that even when there are no active
> connection, the router still track a recent state of the namenode. This will
> be beneficial for debugging.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
