[
https://issues.apache.org/jira/browse/HDFS-13603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850487#comment-17850487
]
ASF GitHub Bot commented on HDFS-13603:
---------------------------------------
simbadzina commented on code in PR #6774:
URL: https://github.com/apache/hadoop/pull/6774#discussion_r1597113868
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/ValueQueue.java:
##########
@@ -269,12 +269,23 @@ public ValueQueue(final int numValues, final float
lowWaterMark, long expiry,
* Initializes the Value Queues for the provided keys by calling the
* fill Method with "numInitValues" values
* @param keyNames Array of key Names
- * @throws ExecutionException executionException.
+ * @throws IOException if no successful initialization for any key
*/
- public void initializeQueuesForKeys(String... keyNames)
- throws ExecutionException {
+ public void initializeQueuesForKeys(String... keyNames) throws IOException {
+ int successfulInitializations = 0;
+ ExecutionException lastException = null;
+
for (String keyName : keyNames) {
- keyQueues.get(keyName);
+ try {
+ keyQueues.get(keyName);
+ successfulInitializations++;
+ } catch (ExecutionException e) {
+ lastException = e;
+ }
+ }
+
+ if (keyNames.length > 0 && successfulInitializations == 0) {
+ throw new IOException("Failed to initialize any queue for the provided
keys.", lastException);
Review Comment:
It seems you've made warm up a best effort operation. If so, there should be
no need to through an exception here. Just logging a warning should be enough.
> Warmup NameNode EDEK thread retries continuously if there's an invalid key
> ---------------------------------------------------------------------------
>
> Key: HDFS-13603
> URL: https://issues.apache.org/jira/browse/HDFS-13603
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: encryption, namenode
> Affects Versions: 2.8.0
> Reporter: Antony Jay
> Priority: Major
> Labels: pull-request-available
>
> https://issues.apache.org/jira/browse/HDFS-9405 adds a background thread to
> pre-warm EDEK cache.
> However this fails and retries continuously if key retrieval fails for one
> encryption zone. In our usecase, we have temporarily removed keys for certain
> encryption zones. Currently namenode and kms log is filled up with errors
> related to background thread retrying warmup for ever .
> The pre-warm thread should
> * Continue to refresh other encryption zones even if it fails for one
> * Should retry only if it fails for all encryption zones, which will be the
> case when kms is down.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
