[
https://issues.apache.org/jira/browse/MAPREDUCE-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972587#action_12972587
]
David Rosenstrauch commented on MAPREDUCE-1938:
-----------------------------------------------
As per Mahadev's comment ("I have usually seens users wanting to override one
of the jars in the classpath.") That's my use case and, I'd think, the major
use case re: this bug.
To clarify my issue: the version of Jackson that ships with Hadoop is old.
But Avro, which I use in several of my M/R jobs, requires a newer version.
Even if I supply the newer Jackson jar using -libjars, the job still picks up
the old one (which has priority on the classpath) and so my job fails.
There is no easy workaround for this. What we wound up doing to fix this issue
in our cluster was to go into every single node, rename the jackson.jar out of
the way, and then restart the daemons. Very manual and error prone process.
(Not to mention a very inelegant fix.)
So what's needed here is some way to tell Hadoop (without too much muss and
fuss): for execution of this M/R job only, *my* version of the jackson.jar
takes precedence over the one that comes shipped.
> Ability for having user's classes take precedence over the system classes for
> tasks' classpath
> ----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1938
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1938
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: job submission, task, tasktracker
> Reporter: Devaraj Das
> Assignee: Krishna Ramachandran
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: mapred-1938-1, mapred-1938-2.patch, mapred-1938-3.patch,
> mr-1938-bp20.1.patch, mr-1938-bp20.patch
>
>
> It would be nice to have the ability in MapReduce to allow users to specify
> for their jobs alternate implementations of classes that are already defined
> in the MapReduce libraries. For example, an alternate implementation for
> CombineFileInputFormat.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
