Robert Joseph Evans created MAPREDUCE-4549:
----------------------------------------------
Summary: Distributed cache conflicts breaks backwards compatability
Key: MAPREDUCE-4549
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4549
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.3, 2.1.0-alpha, 3.0.0, 2.2.0-alpha
Reporter: Robert Joseph Evans
Assignee: Robert Joseph Evans
Priority: Critical
I recently put in MAPREDUCE-4503 which went a bit too far, and broke backwards
compatibility with 1.0 in distribtued cache entries. This is to change the
behavior of the distributed cache to more closely match that of 1.0.
In 1.0 when adding in a cache archive link the first link would win (be the one
that was created), not the last one as is the current behavior, when there were
conflicts then all of the others were ignored and just did not get a symlink
created, and finally no symlink was created for archives that had did not have
a fragment in the URL.
To simulate this behavior after we parse the cache files and cache archives
configuration we should walk through all conflicting links and pick the first
link that has a fragment to win. If no link has a fragment then it is just the
first link wins. All other conflicting links will have a warning an the name
of the link will be changed to include a UUID. If the same file is both in the
distributed cache as a cache file and a cache archive we will throw an
exception, for backwards compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
