[
https://issues.apache.org/jira/browse/PIG-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763234#action_12763234
]
Thejas M Nair commented on PIG-999:
-----------------------------------
{code}
l = load 'st_attr2.bin' using BinStorage();
f = foreach l generate $1, $4#'origin'; -- $4#'origin is stored as chararray
o = order f by $2;
dump o;
{code}
It results in map-reduce failure with error -
java.lang.ClassCastException: org.apache.pig.impl.io.NullableText cannot be
cast to
org.apache.pig.impl.io.NullableBytesWritable
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigBytesRawComparator.compare(PigBytesRawComparator.java:94)
at java.util.Arrays.binarySearch0(Arrays.java:2105)
at java.util.Arrays.binarySearch(Arrays.java:2043)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:64)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:53)
at
org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:466)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:108)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:251)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:240)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:93)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
> sorting on map-value fails if map-value is not of bytearray type
> ----------------------------------------------------------------
>
> Key: PIG-999
> URL: https://issues.apache.org/jira/browse/PIG-999
> Project: Pig
> Issue Type: Sub-task
> Reporter: Thejas M Nair
>
> When query execution plan is created by pig, it assumes the type to be
> bytearray because there is no schema information associated with map fields.
> But at run time, the loader might return the actual type. This results in a
> ClassCastException.
> This issue points to the larger issue of the way pig is handling types for
> map-value.
> This issue should be fixed in the context of revisiting the frontend logic
> and pig-latin semantics.
> This is related to PIG-880 . The patch in PIG-880 changed PigStorage to
> always return bytearray for map values to work around this, but other loaders
> like BinStorage can return the actual type causing this issue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
