[
https://issues.apache.org/jira/browse/HADOOP-12699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111838#comment-15111838
]
Xiao Chen commented on HADOOP-12699:
------------------------------------
Zhe talked to me offline today regarding the sequence to reproduce the failure.
Below is the order of the events happened to result in an old key being
retrieved. Note that the race is only in the
{{EagerKeyGeneratorKeyProviderCryptoExtension}} in {{KMSWebApp}}, so steps are
only focused on the {{ValueQueue}} here.
# Caller {{generateEncryptedKey}}, resulting in the server side KMSProvider's
{{ValueQueue}} fired up an async filling task - a {{NamedRunnable}} is added
into the {{UniqueKeyBlockingQueue}}(queue).
# The {{NamedRunnable}} in step 1 is executed, filling in the local
{{retEdeks}}, but not yet {{keyQueue.addAll}}. Code
[here|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/EagerKeyGeneratorKeyProviderCryptoExtension.java#L83].
# Caller {{rollNewVersion}}, {{ValueQueue}} is drained. That is, the queue of
the being rolled name in {{LoadingCache}}(keyQueues) is cleared.
# The {{NamedRunnable}} continues to run, adding the old keys from local
variable into {{keyQueue}}.
# Caller side {{generateEncryptedKey}}, poll from the keyQueue, get old key.
As a side note, simply not using the local variable wouldn't help, because even
if
> TestKMS#testKMSProvider intermittently fails during 'test rollover draining'
> ----------------------------------------------------------------------------
>
> Key: HADOOP-12699
> URL: https://issues.apache.org/jira/browse/HADOOP-12699
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Xiao Chen
> Assignee: Xiao Chen
> Attachments: HADOOP-12699.01.patch, HADOOP-12699.02.patch,
> HADOOP-12699.03.patch, HADOOP-12699.04.patch, HADOOP-12699.06.patch,
> HADOOP-12699.07.patch, HADOOP-12699.repro.2, HADOOP-12699.repro.patch
>
>
> I've seen several failures of testKMSProvider, all failed in the following
> snippet:
> {code}
> // test rollover draining
> KeyProviderCryptoExtension kpce = KeyProviderCryptoExtension.
> createKeyProviderCryptoExtension(kp);
> .....
> EncryptedKeyVersion ekv1 = kpce.generateEncryptedKey("k6");
> kpce.rollNewVersion("k6");
> EncryptedKeyVersion ekv2 = kpce.generateEncryptedKey("k6");
> Assert.assertNotEquals(ekv1.getEncryptionKeyVersionName(),
> ekv2.getEncryptionKeyVersionName());
> {code}
> with error message
> {quote}Values should be different. Actual: k6@0{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
