Haibo Chen commented on MAPREDUCE-6638:

Thanks for your comments, Karthik!  Uploading a new patch to address your 
suggestions, which does make the logging much more user-friendly. 

bq. One could store the encrypted key in KMS. Once stored, we could do one of 
the following: 1) Tasks have a way to fetch this key directly    2) Leave the 
tasks as is, but augment the AM to recover this key as part of recovery.
I think option 2 is advantageous in that the change needed is minimal and only 
one query for each job attempt to the safe store is needed instead of one for 
each task. 

Will create a follow up jira once this is done.

> Do not attempt to recover jobs if encrypted spill is enabled
> ------------------------------------------------------------
>                 Key: MAPREDUCE-6638
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6638
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: applicationmaster
>    Affects Versions: 2.7.2
>            Reporter: Karthik Kambatla
>            Assignee: Haibo Chen
>         Attachments: mapreduce6638.001.patch, mapreduce6638.002.patch, 
> mapreduce6638.003.patch
> Post the fix to CVE-2015-1776, jobs with ecrypted spills enabled cannot be 
> recovered if the AM fails. We should store the key some place safe so they 
> can actually be recovered. If there is no "safe" place, at least we should 
> restart the job by re-running all mappers/reducers. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

Reply via email to