Sangjin Lee created MAPREDUCE-6588:
--------------------------------------
Summary: default values for PATH for tasks are not quite correct
Key: MAPREDUCE-6588
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6588
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
There are several issues with the way PATH are handled for MR tasks and AMs on
Windows.
- it does not get the PWD as part of the path, unlike the non-Windows case (see
MAPREDUCE-6021)
- the MR AM doesn't get the Hadoop's bin directory in the path explicitly
through mapred configuration; rather, it implicitly inherits it from the node
manager (see MAPREDUCE-6577)
- the default value for the MR task PATH uses "%...%" macros which may be
expanded on the client side prematurely; this is probably OK for tasks (as it's
done by the AM), but not OK for AMs; the late-binding cross-platform macros
should be used
I took a stab at addressing these as part of MAPREDUCE-6577, but I realized
that it would be a bigger issue than the fix for MAPREDUCE-6577 itself. Thus,
I'm filing a separate JIRA to address the Windows side of things.
On a related note, there are quite a few brittle tests on testing these
(TestMiniMRChildTask, TestMiniMRClientCluster, etc.) which are built in such a
way that the tests pass against the current behavior but don't provide a lot of
robust test value. We should also update those tests to make them more robust.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
