[
https://issues.apache.org/jira/browse/HIVE-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13943524#comment-13943524
]
Eugene Koifman commented on HIVE-6724:
--------------------------------------
The use case is Hive-Pig-Hive.
HCatLoader automatically sets hcat.data.tiny.small.int.promotion=true as that
is required by Pig. HCatStorer did the opposite for no good reason. Since Pig
doesn't evaluate each statement separately, the Storer action clobbered Loader
action (the context that contains the configuration is shared). I changed
Storer not to do that.
> HCatStorer throws ClassCastException while storing tinyint/smallint data
> ------------------------------------------------------------------------
>
> Key: HIVE-6724
> URL: https://issues.apache.org/jira/browse/HIVE-6724
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
> Affects Versions: 0.12.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
>
> given Hive tables:
> 1) create table pig_hcatalog_1 (si smallint) STORED AS TEXTFILE;
> 2) create table all100k (si smallint, ti tinyint) STORED ....;
> the following sequence of steps (assuming there is data in all100k)
> {noformat}
> a=load 'all100k' using org.apache.hive.hcatalog.pig.HCatLoader();
> b = foreach a generate si;
> store b into 'pig_hcatalog_1' using org.apache.hive.hcatalog.pig.HCatStorer();
> {noformat}
> produces
> {noformat}
> org.apache.hadoop.mapred.YarnChild: Exception running child :
> java.lang.ClassCastException: java.lang.Short cannot be cast to
> java.lang.Integer
> at
> org.apache.hive.hcatalog.pig.HCatBaseStorer.getJavaObj(HCatBaseStorer.java:372)
> at
> org.apache.hive.hcatalog.pig.HCatBaseStorer.putNext(HCatBaseStorer.java:306)
> at org.apache.hive.hcatalog.pig.HCatStorer.putNext(HCatStorer.java:61)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:635)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
