[
https://issues.apache.org/jira/browse/HADOOP-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
arkady borkovsky updated HADOOP-1857:
-------------------------------------
For streaming, it would be good to have some default failure handling.
It would be nice if the default task failure handling covers the following:
First of all: the message in the UI should state that the streaming command has
failed
Second: Runping' suggestion about the current record and tasks stats is most
useful.
Add to this
ls -l for the current directory and
getenv
Third: a few regexp patterns can capture most of the typical failures:
* shell error message (command not found, wrong permissions, specific
messages from awk, grep, sed, etc.)
* Perl and Python stack traces from the stderr
* if a core file is present -- print its stack.
This will cover most of the problems.
Also: certain type of errors -- command not find, syntax error in a script, etc
-- should kill the job without retries.
If an error of this kind happened in one task (== re pattern matched), it will
happen in all tasks.
> Ability to run a script when a task fails to capture stack traces
> -----------------------------------------------------------------
>
> Key: HADOOP-1857
> URL: https://issues.apache.org/jira/browse/HADOOP-1857
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.14.0
> Reporter: Amareshwari Sri Ramadasu
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.15.0
>
>
> This basically is for providing a better user interface for debugging failed
> jobs. Today we see stack traces for failed tasks on the job ui if the job
> happened to be a Java MR job. For non-Java jobs like Streaming, Pipes, the
> diagnostic info on the job UI is not helpful enough to debug what might have
> gone wrong. They are usually framework traces and not app traces.
> We want to be able to provide a facility, via user-provided scripts, for doing
> post-processing on task logs, input, output, etc. There should be some default
> scripts like running core dumps under gdb for locating illegal instructions,
> the last few lines from stderr, etc. These outputs could be sent to the
> tasktracker and in turn to the jobtracker which would then display it on the
> job UI on demand.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
