[
https://issues.apache.org/jira/browse/HDFS-10757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423519#comment-15423519
]
Xiaoyu Yao commented on HDFS-10757:
-----------------------------------
Thanks [~xiaochen], [~asuresh] and [~jnp] for the discussion. The original
issue leaking thread in the title of HDFS-7718 has been fixed with HADOOP-11368
before HDFS-7718 is resolved. HDFS-7718 introduced KeyProviderCache to the
ClientContext. Maybe we should revisit the goal of KeyProviderCache, which
seems to be one of the sources of the problem. KeyProviderCache contains a map
with key based on KMS URI. When combining with KMSClientProvider that caches
UGI(actualUgi), wrong context may be used as the example [~jnp] mentioned above.
HADOOP-13381 changed the KMSClientProvider#createConnection() by checking if
the currentUGI contains kms-dt but only for non-proxy currentUGI. Correct me if
I'm wrong: when the currentUGI is a new proxy user with kms-dt, I don't think
we should use the stale actualUGI here. Also, we have a few KMS operations
(such as add, and renew/cancel delegation token from HADOOP-13155) that don't
go through KMSClientProvider#createConnection() but use the cached actualUGI.
It will cause similar issue when using with KeyProviderCache enabled.
> KMSClientProvider combined with KeyProviderCache can result in wrong UGI
> being used
> -----------------------------------------------------------------------------------
>
> Key: HDFS-10757
> URL: https://issues.apache.org/jira/browse/HDFS-10757
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Priority: Critical
>
> ClientContext::get gets the context from CACHE via a config setting based
> name, then KeyProviderCache stored in ClientContext gets the key provider
> cached by URI from the configuration, too. These would return the same
> KeyProvider regardless of current UGI.
> KMSClientProvider caches the UGI (actualUgi) in ctor; that means in
> particular that all the users of DFS with KMSClientProvider in a process will
> get the KMS token (along with other credentials) of the first user, via the
> above cache.
> Either KMSClientProvider shouldn't store the UGI, or one of the caches should
> be UGI-aware, like the FS object cache.
> Side note: the comment in createConnection that purports to handle the
> different UGI doesn't seem to cover what it says it covers. In our case, we
> have two unrelated UGIs with no auth (createRemoteUser) with bunch of tokens,
> including a KMS token, added.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
