[
https://issues.apache.org/jira/browse/PIG-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13596588#comment-13596588
]
Prashant Kommireddi commented on PIG-3194:
------------------------------------------
Dmitriy, yes. Here is what I have done after the fix:
*1. compile pig with 1.4*
1a. start up pig with hadoop 0.20.2 - runs [expected]
1b. start up pig with hadoop 1 - runs [expected]
1c. test-commit and few other tests mentioned in my previous comment all pass.
1d. run sample scripts against 1a and 1b - works good.
*2. compile pig with 1.3*
2a. start up pig with hadoop 0.20.2 - runs [expected]
2b. start up pig with hadoop-1 - runs [expected]
2c. test-commit and other tests pass but TestMRCompiler fails - *[not expected]*
2d. run sample scripts against 2a and 2b - works good
The issue with 2c (TestMRCompiler.testMergeJoin failing against 1.3) is that
the gold file being used contains a serialized string that was generated using
1.4. There seem to be a few differences between 1.3 and 1.4 with encode/decode
behavior - please see https://issues.apache.org/jira/browse/CODEC-89,
https://issues.apache.org/jira/browse/CODEC-91
Considering our tests run against hadoop-1 and all pass I am not sure if we
should be spending time in making the test case codec 1.3 aware?
> Changes to ObjectSerializer.java break compatibility with Hadoop 0.20.2
> -----------------------------------------------------------------------
>
> Key: PIG-3194
> URL: https://issues.apache.org/jira/browse/PIG-3194
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.11
> Reporter: Kai Londenberg
> Assignee: Prashant Kommireddi
> Fix For: 0.11.1
>
> Attachments: PIG-3194.patch
>
>
> The changes to ObjectSerializer.java in the following commit
> http://svn.apache.org/viewvc?view=revision&revision=1403934 break
> compatibility with Hadoop 0.20.2 Clusters.
> The reason is, that the code uses methods from Apache Commons Codec 1.4 -
> which are not available in Apache Commons Codec 1.3 which is shipping with
> Hadoop 0.20.2.
> The offending methods are Base64.decodeBase64(String) and
> Base64.encodeBase64URLSafeString(byte[])
> If I revert these changes, Pig 0.11.0 candidate 2 works well with our Hadoop
> 0.20.2 Clusters.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
