[jira] [Commented] (MAPREDUCE-5047) keep.failed.task.files=true causes job failure on secure clusters
[ https://issues.apache.org/jira/browse/MAPREDUCE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599133#comment-13599133 ] Sandy Ryza commented on MAPREDUCE-5047: --- Attempted to make sure that IsolationRunner still works, and it appears to be broken for other reasons, on both secure and non-secure clusters. keep.failed.task.files=true causes job failure on secure clusters - Key: MAPREDUCE-5047 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047 Project: Hadoop Map/Reduce Issue Type: Bug Components: task, tasktracker Affects Versions: 1.1.1 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: MAPREDUCE-5047.patch To support IsolationRunner, split info is written to local directories. This occurs inside MapTask#localizeConfiguration, which is called both tasktracker and by the child JVM. On a secure cluster, the tasktacker's attempt to write it fails, because the tasktracker does not have permission to write to the user's directory. It is likely that the call to localizeConfiguration in the tasktracker can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5047) keep.failed.task.files=true causes job failure on secure clusters
[ https://issues.apache.org/jira/browse/MAPREDUCE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599134#comment-13599134 ] Alejandro Abdelnur commented on MAPREDUCE-5047: --- +1. keep.failed.task.files=true causes job failure on secure clusters - Key: MAPREDUCE-5047 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047 Project: Hadoop Map/Reduce Issue Type: Bug Components: task, tasktracker Affects Versions: 1.1.1 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: MAPREDUCE-5047.patch To support IsolationRunner, split info is written to local directories. This occurs inside MapTask#localizeConfiguration, which is called both tasktracker and by the child JVM. On a secure cluster, the tasktacker's attempt to write it fails, because the tasktracker does not have permission to write to the user's directory. It is likely that the call to localizeConfiguration in the tasktracker can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5047) keep.failed.task.files=true causes job failure on secure clusters
[ https://issues.apache.org/jira/browse/MAPREDUCE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599295#comment-13599295 ] Sandy Ryza commented on MAPREDUCE-5047: --- To elaborate on this a little further, IsolationRunner throws a NPE whenever it starts due to not reading in mapred.local.dir from the job conf before trying to use it. IsolationRunner has been broken for a long time and was removed in MR2 (MAPREDUCE-2606). While this patch doesn't fix IsolationRunner, keep.failed.task.files still has utility without it, and this fixes the bug that was causing jobs to fail every time it's turned on. keep.failed.task.files=true causes job failure on secure clusters - Key: MAPREDUCE-5047 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047 Project: Hadoop Map/Reduce Issue Type: Bug Components: task, tasktracker Affects Versions: 1.1.1 Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 1.3.0 Attachments: MAPREDUCE-5047.patch To support IsolationRunner, split info is written to local directories. This occurs inside MapTask#localizeConfiguration, which is called both tasktracker and by the child JVM. On a secure cluster, the tasktacker's attempt to write it fails, because the tasktracker does not have permission to write to the user's directory. It is likely that the call to localizeConfiguration in the tasktracker can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5047) keep.failed.task.files=true causes job failure on secure clusters
[
https://issues.apache.org/jira/browse/MAPREDUCE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13594253#comment-13594253
]
Hadoop QA commented on MAPREDUCE-5047:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12572230/MAPREDUCE-5047.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3386//console
This message is automatically generated.
keep.failed.task.files=true causes job failure on secure clusters
-
Key: MAPREDUCE-5047
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: task, tasktracker
Affects Versions: 1.1.1
Reporter: Sandy Ryza
Assignee: Sandy Ryza
Attachments: MAPREDUCE-5047.patch
To support IsolationRunner, split info is written to local directories. This
occurs inside MapTask#localizeConfiguration, which is called both tasktracker
and by the child JVM. On a secure cluster, the tasktacker's attempt to write
it fails, because the tasktracker does not have permission to write to the
user's directory. It is likely that the call to localizeConfiguration in the
tasktracker can be removed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5047) keep.failed.task.files=true causes job failure on secure clusters
[ https://issues.apache.org/jira/browse/MAPREDUCE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13594251#comment-13594251 ] Sandy Ryza commented on MAPREDUCE-5047: --- localizeConfiguration is needed in the tasktracker in order to set task-specific configuration options, but split.info does not need to be created at that time. The patch moves the action of writing out split.info into a new writeFilesRequiredForRerun method. This method is called by the Child, but not by the tasktracker. Tested on a pseudo distributed cluster and on a secure distributed cluster that the permissions error no longer shows up and that split.info is still written out to the correct location. keep.failed.task.files=true causes job failure on secure clusters - Key: MAPREDUCE-5047 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047 Project: Hadoop Map/Reduce Issue Type: Bug Components: task, tasktracker Affects Versions: 1.1.1 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: MAPREDUCE-5047.patch To support IsolationRunner, split info is written to local directories. This occurs inside MapTask#localizeConfiguration, which is called both tasktracker and by the child JVM. On a secure cluster, the tasktacker's attempt to write it fails, because the tasktracker does not have permission to write to the user's directory. It is likely that the call to localizeConfiguration in the tasktracker can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
