Anup Ahire created PIG-4023:
-------------------------------
Summary: BigDec/Int sort is broken
Key: PIG-4023
URL: https://issues.apache.org/jira/browse/PIG-4023
Project: Pig
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Anup Ahire
BigDec/Int sort seems broken. I am getting following errors.
java.lang.ClassCastException: java.math.BigInteger incompatible with
java.lang.Double
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigBigIntegerRawComparator.compare(PigBigIntegerRawComparator.java:99)
at java.util.Arrays.binarySearch(Arrays.java:700)
at java.util.Arrays.binarySearch(Arrays.java:305)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:81)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:60)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:689)
at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:122)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368)
----------------------------------------------------------------------------
java.lang.ClassCastException: java.math.BigDecimal incompatible with
java.lang.Double
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigBigDecimalRawComparator.compare(PigBigDecimalRawComparator.java:99)
at java.util.Arrays.binarySearch(Arrays.java:700)
at java.util.Arrays.binarySearch(Arrays.java:305)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:81)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:60)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:689)
at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:122)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368)
PigBig(Decimal/Intger)RawComparator wrongly casts BigInteger and
BigDecimal to Double.
Also, Big(Decimal/Integer)Writable classes are not using buffer offset and
length while initializing ByteArrayInputStream in compare method
--
This message was sent by Atlassian JIRA
(v6.2#6252)
