[
https://issues.apache.org/jira/browse/MAPREDUCE-4369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402792#comment-13402792
]
Ivan Mitic commented on MAPREDUCE-4369:
---------------------------------------
Thanks for the change Bikas!
A few questions/suggestions:
1. In {{WindowsResourceCalculatorPlugin#getProcResourceValues()}} you mention
that some tests use JVM_PID. Do you happen to have a list of these tests?
2. Can you please refactor
{{ResourceCalculatorPlugin#getResourceCalculatorPlugin()}} to accept
processPid, and update call sites to pass the appropriate value (I see only 3
call sites). The cause of this bug in the first place is not having all call
sites set the processPid accordingly. And then, if the passed-in processPid is
null, you can fallback to {{System.getenv().get("JVM_PID")}}. Make sense? If
I'm seeing things correctly, this way you might be able to clean up some of the
newly introduced code.
> Fix streaming job failures with WindowsResourceCalculatorPlugin
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4369
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4369
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: MAPREDUCE-4369.branch-1-win.1.patch
>
>
> Some streaming jobs use local mode job runs that do not start tasks trackers.
> In these cases, the jvm context is not setup and hence local mode execution
> causes the code to crash.
> Fix is to not not use ResourceCalculatorPlugin in such cases or make the
> local job run creating dummy jvm contexts. Choosing the first option because
> thats the current implicit behavior in Linux. The ProcfsBasedProcessTree
> (used inside the LinuxResourceCalculatorPlugin) does no real work when the
> process pid is not setup correctly. This is what happens when local job mode
> runs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
