[
https://issues.apache.org/jira/browse/MAPREDUCE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15470532#comment-15470532
]
Haibo Chen commented on MAPREDUCE-6638:
---------------------------------------
[[email protected]] Thanks for your reviews! I have left this jira to do 1),
so I will create another jira that does 2) after this is done. But as I said in
my previous comment, MR does not seem to have any safe store to persist the
encryption key across job attempts (Not sure how much efforts it is to
introduce one).
bq. Can the boolean condition be simplified to numReduceTasks <= 0 ||
shuffleKeyValidForRecovery || !spillEncrypted?
Not quite sure about when to make something like this incline or a tiny method.
I was thinking of using the method name as comment sort of. Any guideline on
this?
> Jobs with encrypted spills don't recover if the AM goes down
> ------------------------------------------------------------
>
> Key: MAPREDUCE-6638
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6638
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 2.7.2
> Reporter: Karthik Kambatla
> Assignee: Haibo Chen
> Priority: Critical
> Attachments: mapreduce6638.001.patch, mapreduce6638.002.patch
>
>
> Post the fix to CVE-2015-1776, jobs with ecrypted spills enabled cannot be
> recovered if the AM fails. We should store the key some place safe so they
> can actually be recovered. If there is no "safe" place, at least we should
> restart the job by re-running all mappers/reducers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
