[
https://issues.apache.org/jira/browse/PIG-2484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430431#comment-13430431
]
Rohini Palaniswamy commented on PIG-2484:
-----------------------------------------
Did some debugging to see why the float precision is different between 20 and
23 to ensure that is not a cause for concern. Summing up of doubles in java has
known issues
(http://www.velocityreviews.com/forums/t139008-java-double-precision.html). The
precision differs based on the order in which the numbers are summed up. That
is the reason for precision differing between H20 and H23. The order in which
key and values come to the output.collect from the map are same, but when the
reduce of the combiner is called the order of values in Iterable<values> is
different in 20 and 23. There must be some algo change somewhere for grouping
of elements for the combiner. Did not take time and dig deeper to see what the
actual change in mapred is.
We should be able to safely ignore the float precision change in the e2e
tests.
> Fix several e2e test failures/aborts for 23
> -------------------------------------------
>
> Key: PIG-2484
> URL: https://issues.apache.org/jira/browse/PIG-2484
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.2, 0.10.0, 0.11
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.10.0, 0.11
>
> Attachments: PIG-2484-1.patch, PIG-2484-2.patch, PIG-2484-3.patch,
> PIG-2484-4-branch0.9.patch
>
>
> There are still a couple of e2e test aborts/failures for hadoop23. Most of
> them are due to test infrastructure, minor backward incompatibility change in
> 23, or recent changes in Pig. Here is a list:
> Scripting_1/Scripting_2: MAPREDUCE-3700
> Native_3: 23 test need a hadoop23-steaming.jar
> MonitoredUDF_1: Seems related to guava upgrade (PIG-2460), Pig's guava is
> newer than hadoop23's
> UdfException_1, UdfException_2, UdfException_3, UdfException_4: Error message
> change
> Checkin_2, GroupAggFunc_7, GroupAggFunc_9, GroupAggFunc_12, GroupAggFunc_13,
> Types_6, Scalar_1: float precision
> Limit_2: The specific output records change, test infrastructure should allow
> this
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
