Child task debugging on Hadoop
------------------------------
Key: HADOOP-1699
URL: https://issues.apache.org/jira/browse/HADOOP-1699
Project: Hadoop
Issue Type: Improvement
Components: mapred
Environment: Any hadoop installation
Reporter: Srikanth Kakani
Priority: Critical
Recently I discovered debugging a map/reduce task on hadoop can be simplified
with couple of lines code in TaskRunner enabling us to run jdb on the child
tasks.
TaskRunner.java: 282
if(conf.getBoolean("mapred.debug.child.task", false)) {
Random r=new Random();
int debugPort=8000 + r.nextInt(1000);
vargs.add("-Xdebug");
vargs.add("-Xrunjdwp:transport=dt_socket,address="+ debugPort
+",server=y,suspend=n");
LOG.info("Running debug server for task "+ t.getTaskId() + " at
port" + debugPort);
}
This code runs the child tasks with debug enabled after getting a configuration
variable.
Connecting to a child task is as simple as running jdb -attach
<hostname>:<debugPort> from anywhere. Additionally authentication information
could be included in the jobConf.
I believe it will greatly reduce the development/debug time on hadoop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
