[
https://issues.apache.org/jira/browse/MAPREDUCE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983334#action_12983334
]
Scott Carey commented on MAPREDUCE-1478:
----------------------------------------
This approach will work.
I see another possibility that is simpler, but has some serious drawbacks.
Instead of separating the source directories and classpaths at build time, file
set includes/excludes could be used at jar creation time to create two jar
files.
Benefit: simpler, requires no code changes:
Drawback: does not actually disentangle the code, which is more brittle and
could lead to confusion depending on what changes are in the user lib jars.
That is a big drawback.
The above approach is more complicated but does enforce true separation.
In the long run, tackling the disentangling of the lib from the system is a
requirement but the extra complexity requires more careful review. It also is
the bulk of the work required to make the lib a truly separate library -- for
maven or OSGi.
I'm not an expert on the details of the API and code changes needed for clean
separation as a consequence of this. The callback changes seem like an
improvement. Duplication of wrappers and reflective access are usually signs
that there is a better way, but that better way almost always requires an API
break.
Overall, I like this approach because it is moving in the right direction for
separating the core execution from more volatile or user-extensible library
code.
> Separate the mapred.lib and mapreduce.lib classes to a different jar and
> include the user jar ahead of the lib jar.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1478
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1478
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: task
> Reporter: Owen O'Malley
> Attachments: MAPREDUCE-1478.patch, move.sh
>
>
> Currently the user can't include updated library jars as part of their job.
> By pulling out the lib classes we can include the classes (eg.
> TextInputFormat) in the user's jar and get their version and not the system
> installed one.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
