[
https://issues.apache.org/jira/browse/HADOOP-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083500#comment-16083500
]
Xiao Chen commented on HADOOP-14563:
------------------------------------
Thanks Rushabh for the contribution and Wei-Chiu for review, and sorry for
coming late on this.
bq. Sounds like LBKMSCP should throw an exception when all providers throw
exception?
I'm not sure if this is the correct behavior. Say NN is configured with a
LBKMSCP with 3 underlying KMSCPs. Now on warmup, 2 of the KMSCPs failed and the
3rd succeeded. When NN is generating the next edek, it has 1/3 chance to fetch
one from local cache, and 2/3 chance to make a call to KMS.
Is this acceptable? I'm leaning on 'fail' instead of 'continue' in Rushabh's
original question.
[~asuresh], what's your thoughts as the initial author?
> LoadBalancingKMSClientProvider#warmUpEncryptedKeys swallows IOException
> -----------------------------------------------------------------------
>
> Key: HADOOP-14563
> URL: https://issues.apache.org/jira/browse/HADOOP-14563
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.8.1
> Reporter: Rushabh S Shah
> Assignee: Rushabh S Shah
> Fix For: 2.9.0, 3.0.0-beta1, 2.8.2
>
> Attachments: HADOOP-14563-1.patch, HADOOP-14563-2.patch,
> HADOOP-14563.patch
>
>
> TestAclsEndToEnd is failing consistently in HADOOP-14521.
> The reason behind it is LoadBalancingKMSClientProvider#warmUpEncryptedKeys
> swallows IOException while KMSClientProvider#warmUpEncryptedKeys throws all
> the way back to createEncryptionZone and creation of EZ fails.
> Following are the relevant piece of code snippets.
> {code:title=KMSClientProvider.java|borderStyle=solid}
> @Override
> public void warmUpEncryptedKeys(String... keyNames)
> throws IOException {
> try {
> encKeyVersionQueue.initializeQueuesForKeys(keyNames);
> } catch (ExecutionException e) {
> throw new IOException(e);
> }
> }
> {code}
> {code:title=LoadBalancingKMSClientProvider.java|borderStyle=solid}
> // This request is sent to all providers in the load-balancing group
> @Override
> public void warmUpEncryptedKeys(String... keyNames) throws IOException {
> for (KMSClientProvider provider : providers) {
> try {
> provider.warmUpEncryptedKeys(keyNames);
> } catch (IOException ioe) {
> LOG.error(
> "Error warming up keys for provider with url"
> + "[" + provider.getKMSUrl() + "]", ioe);
> }
> }
> }
> {code}
> In HADOOP-14521, I intend to always instantiate
> LoadBalancingKMSClientProvider even if there is only one provider so that the
> retries can applied at only one place.
> We need to decide whether we want to fail in both the case or continue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
