[
https://issues.apache.org/jira/browse/MAPREDUCE-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292072#comment-13292072
]
Bikas Saha commented on MAPREDUCE-4322:
---------------------------------------
Sorry. I forgot the review comments themselves.
1) TaskLog.java - Any special reasons to perform the command line length check
multiple times instead of once at the end of buildCommandLine()?
2) the advantage with the -classpath argument was isolation of the classpath to
the specific spawned JVM. But by changing the classpath env var we risk
changing it for every spawned process too. Maybe thats not much of a problem.
3) What if CLASSPATH is already set on the machine? Will this append to it or
override it? From the code it looks like generating the classpath list will
pick up the parent classpath. So if CLASSPATH env var is already set then it
will be part of classpath list via the parent jvm (TaskTracket jvm). So even if
the taskjvm.cmd sets the CLASSPATH it will be a superset of any existing
CLASSPATH env var. Can you please verify this by having a pre-existing
CLASSPATH set?
> Fix command-line length abort issues on Windows
> -----------------------------------------------
>
> Key: MAPREDUCE-4322
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4322
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Environment: Windows, downstream applications with long aggregate
> classpaths
> Reporter: John Gordon
> Assignee: Ivan Mitic
> Attachments: MAPREDUCE-4322-branch-1-win.patch
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> When a task is started on the tasktracker, it creates a small batch file to
> invoke java and runs that batch. Within the batch file, the invocation of
> Java currently has -classpath ${CLASSPATH} inline to the command. That line
> often exceeds 8000 characters. This is ok for most linux distributions
> because the line limit env variable is often set much higher than this.
> However, for Windows this cause cmd to abort execution. This surfaces in
> Hadoop as an unknown failure mode for the task.
> I think the easiest and most natural way to fix this is to push the
> -classpath option into a config file to take the longest variable part of the
> line and put it somewhere that scales better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
