[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
----------------------------
Description:
The cache has ttl and can close KeyProvider when cache entry is expired but
when DFSClient is closed, we also need to make sure the underlying KeyProvider
used by DFSClient is closed as well. The cache has a removeListener hook which
is called when cache entry is removed.
{code:java}
org.apache.hadoop.hdfs.KeyProviderCache
Class KeyProviderCache
...
public KeyProviderCache(long expiryMs) {
cache = CacheBuilder.newBuilder()
.expireAfterAccess(expiryMs, TimeUnit.MILLISECONDS)
.removalListener(new RemovalListener<URI, KeyProvider>() {
@Override
public void onRemoval(
@Nonnull RemovalNotification<URI, KeyProvider> notification) {
try {
assert notification.getValue() != null;
notification.getValue().close();
} catch (Throwable e) {
LOG.error(
"Error closing KeyProvider with uri ["
+ notification.getKey() + "]", e);
}
}
})
.build();
}{code}
was:
The cache has ttl and can close KeyProvider when cache entry is expired but
when DFSClient is closed, we also need to make sure the underlying KeyProvider
used by DFSClient is closed as well. The cache has a removeListener hook which
is called when cache entry is removed.
{code:java}
org.apache.hadoop.hdfs.KeyProviderCache
public KeyProviderCache(long expiryMs) {
cache = CacheBuilder.newBuilder()
.expireAfterAccess(expiryMs, TimeUnit.MILLISECONDS)
.removalListener(new RemovalListener<URI, KeyProvider>() {
@Override
public void onRemoval(
@Nonnull RemovalNotification<URI, KeyProvider> notification) {
try {
assert notification.getValue() != null;
notification.getValue().close();
} catch (Throwable e) {
LOG.error(
"Error closing KeyProvider with uri ["
+ notification.getKey() + "]", e);
}
}
})
.build();
}{code}
> Cached KeyProvider in KeyProviderCache does not get closed when DFSClient is
> closed
> ------------------------------------------------------------------------------------
>
> Key: HDFS-16518
> URL: https://issues.apache.org/jira/browse/HDFS-16518
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
> Affects Versions: 2.10.0
> Reporter: Lei Yang
> Priority: Major
>
> The cache has ttl and can close KeyProvider when cache entry is expired but
> when DFSClient is closed, we also need to make sure the underlying
> KeyProvider used by DFSClient is closed as well. The cache has a
> removeListener hook which is called when cache entry is removed.
> {code:java}
> org.apache.hadoop.hdfs.KeyProviderCache
> Class KeyProviderCache
> ...
> public KeyProviderCache(long expiryMs) {
> cache = CacheBuilder.newBuilder()
> .expireAfterAccess(expiryMs, TimeUnit.MILLISECONDS)
> .removalListener(new RemovalListener<URI, KeyProvider>() {
> @Override
> public void onRemoval(
> @Nonnull RemovalNotification<URI, KeyProvider> notification) {
> try {
> assert notification.getValue() != null;
> notification.getValue().close();
> } catch (Throwable e) {
> LOG.error(
> "Error closing KeyProvider with uri ["
> + notification.getKey() + "]", e);
> }
> }
> })
> .build();
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
