[
https://issues.apache.org/jira/browse/PIG-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160191#comment-13160191
]
Thomas Weise commented on PIG-2388:
-----------------------------------
Even though the common code (outside shim) can be compiled against either of
the Hadoop MR versions, it needs to run against the version it was compiled
against (due to changes from class to interface in several cases). Whenever we
have something like jobcontext.getConfiguration() etc., the bytecode for
the method call will be different depending on whether jobcontext is a class or
interface (compile time fine, runtime not). Other places like
somemethod(JobContext context) don't have that problem. Could get it to work
for basic illustrate, but as soon as MR comes into the picture, there are many
many places in the common code that are affected and it is not reasonably
possible to shim all those.
Our solution will be an installer that contains set of jar files compiled
against both versions and resolve the dependency at startup/install time.
> Make shim for Hadoop 0.20 and 0.23 support dynamic
> --------------------------------------------------
>
> Key: PIG-2388
> URL: https://issues.apache.org/jira/browse/PIG-2388
> Project: Pig
> Issue Type: Improvement
> Affects Versions: 0.10, 0.9.2
> Reporter: Thomas Weise
> Assignee: Thomas Weise
> Fix For: 0.10, 0.9.2
>
> Attachments: PIG-2388_branch-0.9.patch
>
>
> We need a single Pig installation that works with both Hadoop versions. The
> current shim implementation assumes different builds for each version. We can
> solve this statically through internal build/installation system or by making
> the shim dynamic so that pig.jar will work on both version with runtime
> detection. Attached patch is to convert the static shims into a shim
> interface with 2 implementations, each of which will be compiled against the
> respective Hadoop version and included into single pig.jar (similar to what
> Hive does).
> The default build behavior remains unchanged, only the shim for
> ${hadoopversion} will be compiled. Both shims can be built via: ant
> -Dbuild-all-shims=true
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
