[
https://issues.apache.org/jira/browse/HCATALOG-365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251980#comment-13251980
]
Travis Crawford commented on HCATALOG-365:
------------------------------------------
@sushanth - Sounds like we should make two tweaks in this patch. One is setting
the FosterStorageHandler configuration, which I think makes sense because its
incompletely setup and might cause issues elsewhere. Additionally, we can
simplify as follows:
{code:title=Current code}
Configuration conf = storageHandler.getConf();
InternalUtil.initializeInputSerDe(serde, conf,
partitionInfo.getTableInfo(),partitionInfo.getPartitionSchema());
{code}
{code:title=Proposed new code}
InternalUtil.initializeInputSerDe(serde, jobContext.getConfiguration(),
partitionInfo.getTableInfo(),partitionInfo.getPartitionSchema());
{code}
If this sounds good I'll update the patch.
> HCatUtil.getStorageHandler should set the configuration
> -------------------------------------------------------
>
> Key: HCATALOG-365
> URL: https://issues.apache.org/jira/browse/HCATALOG-365
> Project: HCatalog
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Attachments: HCATALOG-365.diff
>
>
> {{HCatUtil.getStorageHandler}} creates a {{FosterStorageHandler}} when no
> existing storage handler is defined. The configuration is not set though.
> Without this patch the configuration passed to your serde {{initialize}} is
> null.
> {code}
> java.lang.RuntimeException: java.io.IOException: Unable to create
> objectInspector for serde class
> com.twitter.elephantbird.hive.serde.ThriftSerDe
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:237)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.<init>(PigRecordReader.java:109)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:119)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:614)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> Caused by: java.io.IOException: Unable to create objectInspector for serde
> class com.twitter.elephantbird.hive.serde.ThriftSerDe
> at
> org.apache.hcatalog.mapreduce.HCatBaseInputFormat.createRecordReader(HCatBaseInputFormat.java:212)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:228)
> ... 5 more
> Caused by: java.lang.NullPointerException
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187)
> at
> com.twitter.elephantbird.hive.serde.ThriftSerDe.initialize(ThriftSerDe.java:28)
> at
> org.apache.hcatalog.mapreduce.InternalUtil.initializeSerDe(InternalUtil.java:164)
> at
> org.apache.hcatalog.mapreduce.InternalUtil.initializeInputSerDe(InternalUtil.java:144)
> at
> org.apache.hcatalog.mapreduce.HCatBaseInputFormat.createRecordReader(HCatBaseInputFormat.java:208)
> ... 6 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
