[jira] [Commented] (HDFS-6971) Bounded staleness of EDEK caches on the NN
[
https://issues.apache.org/jira/browse/HDFS-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14525845#comment-14525845
]
Hadoop QA commented on HDFS-6971:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 29s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 27s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 33s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 4s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 41s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | common tests | 23m 47s | Tests passed in
hadoop-common. |
| | | 60m 33s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12672067/HDFS-6971-20140930-v1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a319771 |
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10747/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10747/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10747/console |
This message was automatically generated.
Bounded staleness of EDEK caches on the NN
--
Key: HDFS-6971
URL: https://issues.apache.org/jira/browse/HDFS-6971
Project: Hadoop HDFS
Issue Type: Sub-task
Components: encryption
Reporter: Andrew Wang
Assignee: Zhe Zhang
Attachments: HDFS-6971-20140930-v1.patch
The EDEK cache on the NN can hold onto keys after the admin has rolled the
key. It'd be good to time-bound the caches, perhaps also providing an
explicit flush command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6971) Bounded staleness of EDEK caches on the NN
[
https://issues.apache.org/jira/browse/HDFS-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14525753#comment-14525753
]
Hadoop QA commented on HDFS-6971:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 35s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 29s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 41s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 5s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 40s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | common tests | 22m 40s | Tests passed in
hadoop-common. |
| | | 59m 42s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12672067/HDFS-6971-20140930-v1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a319771 |
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10739/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10739/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10739/console |
This message was automatically generated.
Bounded staleness of EDEK caches on the NN
--
Key: HDFS-6971
URL: https://issues.apache.org/jira/browse/HDFS-6971
Project: Hadoop HDFS
Issue Type: Sub-task
Components: encryption
Reporter: Andrew Wang
Assignee: Zhe Zhang
Attachments: HDFS-6971-20140930-v1.patch
The EDEK cache on the NN can hold onto keys after the admin has rolled the
key. It'd be good to time-bound the caches, perhaps also providing an
explicit flush command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6971) Bounded staleness of EDEK caches on the NN
[
https://issues.apache.org/jira/browse/HDFS-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153486#comment-14153486
]
Hadoop QA commented on HDFS-6971:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672067/HDFS-6971-20140930-v1.patch
against trunk revision b915869.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/8270//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8270//console
This message is automatically generated.
Bounded staleness of EDEK caches on the NN
--
Key: HDFS-6971
URL: https://issues.apache.org/jira/browse/HDFS-6971
Project: Hadoop HDFS
Issue Type: Sub-task
Components: encryption
Reporter: Andrew Wang
Assignee: Zhe Zhang
Attachments: HDFS-6971-20140930-v1.patch
The EDEK cache on the NN can hold onto keys after the admin has rolled the
key. It'd be good to time-bound the caches, perhaps also providing an
explicit flush command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6971) Bounded staleness of EDEK caches on the NN
[
https://issues.apache.org/jira/browse/HDFS-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152013#comment-14152013
]
Andrew Wang commented on HDFS-6971:
---
Yea, what we'd need here is {{expireAfterWrite}} as you say. Thanks Zhe.
Bounded staleness of EDEK caches on the NN
--
Key: HDFS-6971
URL: https://issues.apache.org/jira/browse/HDFS-6971
Project: Hadoop HDFS
Issue Type: Sub-task
Components: encryption
Affects Versions: 2.5.0
Reporter: Andrew Wang
Assignee: Zhe Zhang
The EDEK cache on the NN can hold onto keys after the admin has rolled the
key. It'd be good to time-bound the caches, perhaps also providing an
explicit flush command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6971) Bounded staleness of EDEK caches on the NN
[
https://issues.apache.org/jira/browse/HDFS-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148068#comment-14148068
]
Zhe Zhang commented on HDFS-6971:
-
[~andrew.wang] What level of time-boundedness do we need? Is
{{expireAfterAccess}} sufficient or we need {{expireAfterWrite}}? I think the
latter is necessary right?
Bounded staleness of EDEK caches on the NN
--
Key: HDFS-6971
URL: https://issues.apache.org/jira/browse/HDFS-6971
Project: Hadoop HDFS
Issue Type: Sub-task
Components: encryption
Affects Versions: 2.5.0
Reporter: Andrew Wang
Assignee: Zhe Zhang
The EDEK cache on the NN can hold onto keys after the admin has rolled the
key. It'd be good to time-bound the caches, perhaps also providing an
explicit flush command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6971) Bounded staleness of EDEK caches on the NN
[
https://issues.apache.org/jira/browse/HDFS-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14145094#comment-14145094
]
Zhe Zhang commented on HDFS-6971:
-
The {{ValueQueue}} ({{encKeyVersionQueue}}) uses an underlying cache
{{keyQueues}}, which is constructed with {{expireAfterAccess}} too. I'll add a
test in {{TestKMS}} to verify the time-boundedness.
Bounded staleness of EDEK caches on the NN
--
Key: HDFS-6971
URL: https://issues.apache.org/jira/browse/HDFS-6971
Project: Hadoop HDFS
Issue Type: Sub-task
Components: encryption
Affects Versions: 2.5.0
Reporter: Andrew Wang
Assignee: Zhe Zhang
The EDEK cache on the NN can hold onto keys after the admin has rolled the
key. It'd be good to time-bound the caches, perhaps also providing an
explicit flush command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6971) Bounded staleness of EDEK caches on the NN
[ https://issues.apache.org/jira/browse/HDFS-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140249#comment-14140249 ] Andrew Wang commented on HDFS-6971: --- So we actually have caches in a number of places :) The CachingKeyProvider is actually on the KMS. On the NN side, we're interacting with the KMSClientProvider. If you look at KMSClientProvider#generateEncryptedKey, you'll see it interacting with a ValueQueue, which is the cache I was thinking about. I don't think this is time-bounded. If you could, you can also double check that CachingKeyProvider drops its cached encrypted keys at the right times, i.e. on modify ops like deleteKey and rollNewVersion. As long as it does so, I think we should be up-to-date. Bounded staleness of EDEK caches on the NN -- Key: HDFS-6971 URL: https://issues.apache.org/jira/browse/HDFS-6971 Project: Hadoop HDFS Issue Type: Sub-task Components: encryption Affects Versions: 2.5.0 Reporter: Andrew Wang Assignee: Zhe Zhang The EDEK cache on the NN can hold onto keys after the admin has rolled the key. It'd be good to time-bound the caches, perhaps also providing an explicit flush command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6971) Bounded staleness of EDEK caches on the NN
[
https://issues.apache.org/jira/browse/HDFS-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134735#comment-14134735
]
Zhe Zhang commented on HDFS-6971:
-
{{currentKeyCache}} in {{CachingKeyProvidet}} is already created with
{{expireAfterAccess}}. Is this JIRA about another cache? Regardless, a flush
function is still needed.
Bounded staleness of EDEK caches on the NN
--
Key: HDFS-6971
URL: https://issues.apache.org/jira/browse/HDFS-6971
Project: Hadoop HDFS
Issue Type: Sub-task
Components: encryption
Affects Versions: 2.5.0
Reporter: Andrew Wang
Assignee: Zhe Zhang
The EDEK cache on the NN can hold onto keys after the admin has rolled the
key. It'd be good to time-bound the caches, perhaps also providing an
explicit flush command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
