[
https://issues.apache.org/jira/browse/MAPREDUCE-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972018#action_12972018
]
Alejandro Abdelnur commented on MAPREDUCE-1938:
-----------------------------------------------
IMO this is not the right way of approaching this issue as it will lead to have
multiple implementations of a library in the classpath and that sooner or later
breaks things in very odd ways. Following an example of an issue I've run into
few years ago:
a-1.jar contains classes: A, B (with B using A)
a-2.jar contains classes: A, C (with C using A)
A classes from a-1.jar and a-2.jar are different.
Kernel 'sometimes' uses class B from a-1.jar
user-app 'sometimes' uses class C from a-2.jar
All this will compile, but will create very odd errors at runtime.
Because of this, I wouldn't provide a mechanism to allow users to do this.
Instead, I think we should implement a task-classloader that only exposes to
the application the MR API and the user is complete control of all the JARs in
the app classpath, including input/output formats, etc, etc.
I understand this can be a bit tricky because the line of API vs implementation
classes is not that clear as in the servlet API (to bring an example of APIs
with implementations doing this).
> Ability for having user's classes take precedence over the system classes for
> tasks' classpath
> ----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1938
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1938
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: job submission, task, tasktracker
> Reporter: Devaraj Das
> Assignee: Krishna Ramachandran
> Fix For: 0.22.0
>
> Attachments: mapred-1938-1, mapred-1938-2.patch, mapred-1938-3.patch,
> mr-1938-bp20.1.patch, mr-1938-bp20.patch
>
>
> It would be nice to have the ability in MapReduce to allow users to specify
> for their jobs alternate implementations of classes that are already defined
> in the MapReduce libraries. For example, an alternate implementation for
> CombineFileInputFormat.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
