[
https://issues.apache.org/jira/browse/HDFS-16518?focusedWorklogId=746838&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-746838
]
ASF GitHub Bot logged work on HDFS-16518:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 23/Mar/22 19:49
Start Date: 23/Mar/22 19:49
Worklog Time Spent: 10m
Work Description: li-leyang commented on a change in pull request #4100:
URL: https://github.com/apache/hadoop/pull/4100#discussion_r833665038
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/KeyProviderCache.java
##########
@@ -85,6 +90,26 @@ public KeyProvider call() throws Exception {
}
}
+ public static final int SHUTDOWN_HOOK_PRIORITY =
FileSystem.SHUTDOWN_HOOK_PRIORITY - 1;
+
+ private class KeyProviderCacheFinalizer implements Runnable {
+ @Override
+ public synchronized void run() {
+ invalidateCache();
+ }
+ }
+
+ /**
+ * Invalidate cache and auto close KeyProviders in the cache
+ */
+ @VisibleForTesting
+ synchronized void invalidateCache() {
+ LOG.debug("Invalidating all cached KeyProviders in ShutdownHookManager.");
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 746838)
Time Spent: 50m (was: 40m)
> Cached KeyProvider in KeyProviderCache should be closed with
> ShutdownHookManager
> --------------------------------------------------------------------------------
>
> Key: HDFS-16518
> URL: https://issues.apache.org/jira/browse/HDFS-16518
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
> Affects Versions: 2.10.0
> Reporter: Lei Yang
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We need to make sure the underlying KeyProvider used by multiple DFSClient
> instances is closed at one shot during jvm shutdown. Within the shutdownhook,
> we invalidate the cache and make sure they are all closed. The cache has a
> removeListener hook which is called when cache entry is invalidated.
> {code:java}
> Class KeyProviderCache
> ...
> public KeyProviderCache(long expiryMs) {
> cache = CacheBuilder.newBuilder()
> .expireAfterAccess(expiryMs, TimeUnit.MILLISECONDS)
> .removalListener(new RemovalListener<URI, KeyProvider>() {
> @Override
> public void onRemoval(
> @Nonnull RemovalNotification<URI, KeyProvider> notification) {
> try {
> assert notification.getValue() != null;
> notification.getValue().close();
> } catch (Throwable e) {
> LOG.error(
> "Error closing KeyProvider with uri ["
> + notification.getKey() + "]", e);
> }
> }
> })
> .build();
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
