[
https://issues.apache.org/jira/browse/HDFS-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16039943#comment-16039943
]
Andrew Wang commented on HDFS-11885:
------------------------------------
Thanks for reviewing Rushabh. New patch attached, with comments:
bq. 1. We can skip reading EDEKCacheLoader related conf properties in
FSNamesystem. We can just pass the conf object to EDEKCacheLoader class and let
it read in EDEKCacheLoader.
One issue with this is that I'd like to do fail-fast with the precondition
checks. If we defer the config reading to EDEKCacheLoader's constructor, it
won't fail until the NN starts active services. I left this as is.
bq. 2. Why to call edekCacheLoader.warmUpEdekCache(0) after
FSNamesystem#createEncryptionZone ?
We'd still like to warm up in the background when a new EZ key is seen. This
way it does not have to block during the first file write; we get consistent
performance.
bq. 3. The patch applies almost cleanly in branch-2.8 but compilation is
failing with the following error.
Thanks, changed the test a bit and checked compilation on branch-2.
bq. The above chunk of code is redundant as
KeyProviderCryptoExtension#warmUpEncryptedKeys is never called since that is
only being called at namenode startup and since we are not restarting namenode
in the test case, it will never be called.
I did this as a guard against future changes to prod code. Some future
programmer might change createFile to calling warmUpEncryptedKeys, and it'd be
caught by this test case.
> createEncryptionZone should not block on initializing EDEK cache
> ----------------------------------------------------------------
>
> Key: HDFS-11885
> URL: https://issues.apache.org/jira/browse/HDFS-11885
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: encryption
> Affects Versions: 2.6.5
> Reporter: Andrew Wang
> Assignee: Andrew Wang
> Priority: Critical
> Attachments: HDFS-11885.001.patch, HDFS-11885.002.patch,
> HDFS-11885.003.patch
>
>
> When creating an encryption zone, we call {{ensureKeyIsInitialized}}, which
> calls {{provider.warmUpEncryptedKeys(keyName)}}. This is a blocking call,
> which attempts to fill the key cache up to the low watermark.
> If the KMS is down or slow, this can take a very long time, and cause the
> createZone RPC to fail with a timeout.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
