[
https://issues.apache.org/jira/browse/MAPREDUCE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15498224#comment-15498224
]
Haibo Chen commented on MAPREDUCE-6638:
---------------------------------------
Thanks for your comments, Karthik! Uploading a new patch to address your
suggestions, which does make the logging much more user-friendly.
bq. One could store the encrypted key in KMS. Once stored, we could do one of
the following: 1) Tasks have a way to fetch this key directly 2) Leave the
tasks as is, but augment the AM to recover this key as part of recovery.
I think option 2 is advantageous in that the change needed is minimal and only
one query for each job attempt to the safe store is needed instead of one for
each task.
Will create a follow up jira once this is done.
> Do not attempt to recover jobs if encrypted spill is enabled
> ------------------------------------------------------------
>
> Key: MAPREDUCE-6638
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6638
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
> Affects Versions: 2.7.2
> Reporter: Karthik Kambatla
> Assignee: Haibo Chen
> Attachments: mapreduce6638.001.patch, mapreduce6638.002.patch,
> mapreduce6638.003.patch
>
>
> Post the fix to CVE-2015-1776, jobs with ecrypted spills enabled cannot be
> recovered if the AM fails. We should store the key some place safe so they
> can actually be recovered. If there is no "safe" place, at least we should
> restart the job by re-running all mappers/reducers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
