[
https://issues.apache.org/jira/browse/HCATALOG-365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250175#comment-13250175
]
Rohini Palaniswamy commented on HCATALOG-365:
---------------------------------------------
+1. Looks good. But I am thinking in addition to this patch, it would it be
better if we passed the JobConf which has the jobProperties copied to
initialize the serde instead of storageHandler.getConf() in
HCatBaseInputFormat.
InternalUtil.initializeInputSerDe(serde, jobConf,
partitionInfo.getTableInfo(),partitionInfo.getPartitionSchema());
instead of
Configuration conf = storageHandler.getConf();
InternalUtil.initializeInputSerDe(serde, conf,
partitionInfo.getTableInfo(),partitionInfo.getPartitionSchema());
Sushanth/Alan,
Thoughts?
> HCatUtil.getStorageHandler should set the configuration
> -------------------------------------------------------
>
> Key: HCATALOG-365
> URL: https://issues.apache.org/jira/browse/HCATALOG-365
> Project: HCatalog
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Attachments: HCATALOG-365.diff
>
>
> {{HCatUtil.getStorageHandler}} creates a {{FosterStorageHandler}} when no
> existing storage handler is defined. The configuration is not set though.
> Without this patch the configuration passed to your serde {{initialize}} is
> null.
> {code}
> java.lang.RuntimeException: java.io.IOException: Unable to create
> objectInspector for serde class
> com.twitter.elephantbird.hive.serde.ThriftSerDe
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:237)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.<init>(PigRecordReader.java:109)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:119)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:614)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> Caused by: java.io.IOException: Unable to create objectInspector for serde
> class com.twitter.elephantbird.hive.serde.ThriftSerDe
> at
> org.apache.hcatalog.mapreduce.HCatBaseInputFormat.createRecordReader(HCatBaseInputFormat.java:212)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:228)
> ... 5 more
> Caused by: java.lang.NullPointerException
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187)
> at
> com.twitter.elephantbird.hive.serde.ThriftSerDe.initialize(ThriftSerDe.java:28)
> at
> org.apache.hcatalog.mapreduce.InternalUtil.initializeSerDe(InternalUtil.java:164)
> at
> org.apache.hcatalog.mapreduce.InternalUtil.initializeInputSerDe(InternalUtil.java:144)
> at
> org.apache.hcatalog.mapreduce.HCatBaseInputFormat.createRecordReader(HCatBaseInputFormat.java:208)
> ... 6 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
