[
https://issues.apache.org/jira/browse/HCATALOG-365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251960#comment-13251960
]
Sushanth Sowmyan commented on HCATALOG-365:
-------------------------------------------
+1 on the patch itself - agreed that this is needed.
@Rohini : You're right in that we need that, but looking at Travis' change, I
think it should achieve exactly that in the cases where we were missing it. The
problem was that the conf being sent in to the serde was being picked up from
the storagehandler.getConf, but we'd not called setConf on it in the case of a
FosterStorageHandler, so there was an issue. In the case of a custom
StorageHandler, .setConf is already being called on it from having instantiated
it using ReflectionUtils, so it should work.
> HCatUtil.getStorageHandler should set the configuration
> -------------------------------------------------------
>
> Key: HCATALOG-365
> URL: https://issues.apache.org/jira/browse/HCATALOG-365
> Project: HCatalog
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Attachments: HCATALOG-365.diff
>
>
> {{HCatUtil.getStorageHandler}} creates a {{FosterStorageHandler}} when no
> existing storage handler is defined. The configuration is not set though.
> Without this patch the configuration passed to your serde {{initialize}} is
> null.
> {code}
> java.lang.RuntimeException: java.io.IOException: Unable to create
> objectInspector for serde class
> com.twitter.elephantbird.hive.serde.ThriftSerDe
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:237)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.<init>(PigRecordReader.java:109)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:119)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:614)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> Caused by: java.io.IOException: Unable to create objectInspector for serde
> class com.twitter.elephantbird.hive.serde.ThriftSerDe
> at
> org.apache.hcatalog.mapreduce.HCatBaseInputFormat.createRecordReader(HCatBaseInputFormat.java:212)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:228)
> ... 5 more
> Caused by: java.lang.NullPointerException
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187)
> at
> com.twitter.elephantbird.hive.serde.ThriftSerDe.initialize(ThriftSerDe.java:28)
> at
> org.apache.hcatalog.mapreduce.InternalUtil.initializeSerDe(InternalUtil.java:164)
> at
> org.apache.hcatalog.mapreduce.InternalUtil.initializeInputSerDe(InternalUtil.java:144)
> at
> org.apache.hcatalog.mapreduce.HCatBaseInputFormat.createRecordReader(HCatBaseInputFormat.java:208)
> ... 6 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
