[jira] [Commented] (MAPREDUCE-7141) Allow KMS generated spill encryption keys
[ https://issues.apache.org/jira/browse/MAPREDUCE-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356485#comment-17356485 ] Jim Brennan commented on MAPREDUCE-7141: [~ahussein] the current PR does not apply to trunk. Can you please update it? > Allow KMS generated spill encryption keys > - > > Key: MAPREDUCE-7141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7141 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Kuhu Shukla >Assignee: Ahmed Hussein >Priority: Major > Labels: pull-request-available > Attachments: MAPREDUCE-7141-001.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Currently the only way an encryption key for task spills is generated is by > the AM's key generator. This JIRA tracks the work required to add KMS support > to this key's generation allowing fault tolerance to AM failures/re-runs and > also give another option to the client on how it wants the keys to be created. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7141) Allow KMS generated spill encryption keys
[
https://issues.apache.org/jira/browse/MAPREDUCE-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17276589#comment-17276589
]
Hadoop QA commented on MAPREDUCE-7141:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s{color}
| {color:red}{color} | {color:red} MAPREDUCE-7141 does not apply to trunk.
Rebase required? Wrong Branch? See
https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | MAPREDUCE-7141 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954487/MAPREDUCE-7141-001.patch
|
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build/52/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Allow KMS generated spill encryption keys
> -
>
> Key: MAPREDUCE-7141
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7141
> Project: Hadoop Map/Reduce
> Issue Type: Task
>Reporter: Kuhu Shukla
>Assignee: Kuhu Shukla
>Priority: Major
> Attachments: MAPREDUCE-7141-001.patch
>
>
> Currently the only way an encryption key for task spills is generated is by
> the AM's key generator. This JIRA tracks the work required to add KMS support
> to this key's generation allowing fault tolerance to AM failures/re-runs and
> also give another option to the client on how it wants the keys to be created.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7141) Allow KMS generated spill encryption keys
[ https://issues.apache.org/jira/browse/MAPREDUCE-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17276587#comment-17276587 ] Ahmed Hussein commented on MAPREDUCE-7141: -- Thanks [~kshukla] for uploading the patch. I will rebase the patch and try to see how this can be pushed forward. > Allow KMS generated spill encryption keys > - > > Key: MAPREDUCE-7141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7141 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Kuhu Shukla >Assignee: Kuhu Shukla >Priority: Major > Attachments: MAPREDUCE-7141-001.patch > > > Currently the only way an encryption key for task spills is generated is by > the AM's key generator. This JIRA tracks the work required to add KMS support > to this key's generation allowing fault tolerance to AM failures/re-runs and > also give another option to the client on how it wants the keys to be created. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7141) Allow KMS generated spill encryption keys
[
https://issues.apache.org/jira/browse/MAPREDUCE-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951141#comment-16951141
]
Hadoop QA commented on MAPREDUCE-7141:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red} MAPREDUCE-7141 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | MAPREDUCE-7141 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954487/MAPREDUCE-7141-001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7673/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Allow KMS generated spill encryption keys
> -
>
> Key: MAPREDUCE-7141
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7141
> Project: Hadoop Map/Reduce
> Issue Type: Task
>Reporter: Kuhu Shukla
>Assignee: Kuhu Shukla
>Priority: Major
> Attachments: MAPREDUCE-7141-001.patch
>
>
> Currently the only way an encryption key for task spills is generated is by
> the AM's key generator. This JIRA tracks the work required to add KMS support
> to this key's generation allowing fault tolerance to AM failures/re-runs and
> also give another option to the client on how it wants the keys to be created.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7141) Allow KMS generated spill encryption keys
[
https://issues.apache.org/jira/browse/MAPREDUCE-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745460#comment-16745460
]
Hadoop QA commented on MAPREDUCE-7141:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 52s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 41s{color} | {color:orange}
hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 35 new +
805 unchanged - 1 fixed = 840 total (was 806) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 3s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
6s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
50s{color} | {color:green} hadoop-mapreduce-client-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
26s{color} | {color:green} hadoop-mapreduce-client-app in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}132m 24s{color}
| {color:red} hadoop-mapreduce-client-jobclient in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
27s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}210m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | MAPREDUCE-7141 |
| JIRA Patch URL |
[jira] [Commented] (MAPREDUCE-7141) Allow KMS generated spill encryption keys
[ https://issues.apache.org/jira/browse/MAPREDUCE-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745410#comment-16745410 ] Kuhu Shukla commented on MAPREDUCE-7141: CC: [~eepayne], [~jeagles], [~Jim_Brennan] > Allow KMS generated spill encryption keys > - > > Key: MAPREDUCE-7141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7141 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Kuhu Shukla >Assignee: Kuhu Shukla >Priority: Major > Attachments: MAPREDUCE-7141-001.patch > > > Currently the only way an encryption key for task spills is generated is by > the AM's key generator. This JIRA tracks the work required to add KMS support > to this key's generation allowing fault tolerance to AM failures/re-runs and > also give another option to the client on how it wants the keys to be created. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7141) Allow KMS generated spill encryption keys
[
https://issues.apache.org/jira/browse/MAPREDUCE-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739792#comment-16739792
]
Kuhu Shukla commented on MAPREDUCE-7141:
This patch is not perfect and has one big challenge. How do we decouple
ourselves for KMSSpillKeyProvider to not use the DistributedFileSystem for
getting the key provider shown below. What does it take for the RM to provide
an independent key provider.
{code}KeyProvider keyProvider =
((DistributedFileSystem)FileSystem.get(getConf())).getKeyProvider();
{code}
I appreciate any and all comments on the same.
> Allow KMS generated spill encryption keys
> -
>
> Key: MAPREDUCE-7141
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7141
> Project: Hadoop Map/Reduce
> Issue Type: Task
>Reporter: Kuhu Shukla
>Assignee: Kuhu Shukla
>Priority: Major
> Attachments: MAPREDUCE-7141-001.patch
>
>
> Currently the only way an encryption key for task spills is generated is by
> the AM's key generator. This JIRA tracks the work required to add KMS support
> to this key's generation allowing fault tolerance to AM failures/re-runs and
> also give another option to the client on how it wants the keys to be created.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
