[
https://issues.apache.org/jira/browse/HIVE-6147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588536#comment-16588536
]
Vamsi Subhash Achanta commented on HIVE-6147:
---------------------------------------------
[~swarnim]
Hi,
Is there any work currently going on to make the serialization work at a column
level? If I have 2 columns with different avro schema for each column, the
current code is not working. Ex:
{{CREATE EXTERNAL TABLE txn_store.transactions_single3_1}}
{{ROW FORMAT SERDE "org.apache.hadoop.hive.hbase.HBaseSerDe"}}
{{STORED BY "org.apache.hadoop.hive.hbase.HBaseStorageHandler"}}
{{WITH SERDEPROPERTIES (}}
{{"hbase.columns.mapping" =
":key,nsp:scr_p_1,nsp:scr_m_1,nsp:scr_a_1",}}{{"nsp.scr_p_1.serialization.type"
= "avro",}}
{{"avro.schema.retriever" =
"com.phonepe.hive.schema.PaymentNamespaceAvroSchemaRetriever",}}
{{"nsp.scr_p_1.avro.schema.url" =
"hdfs://namenode:8020/user/admin/schemas/payment_namespace.avsc",}}{{"nsp.scr_m_1.serialization.type"
= "avro",}}
{{"avro.schema.retriever" =
"com.phonepe.hive.schema.MerchantNamespaceAvroSchemaRetriever",}}
{{"nsp.scr_m_1.avro.schema.url" =
"hdfs://namenode.nm1:8020/user/admin/schemas/merchant_namespace.avsc",}}{{"nsp.scr_a_1.serialization.type"
= "avro",}}
{{"avro.schema.retriever" =
"com.phonepe.hive.schema.AccountingNamespaceAvroSchemaRetriever",}}
{{"nsp.scr_a_1.avro.schema.url" =
"hdfs://namenode:8020/user/admin/schemas/accounting_namespace.avsc"}}
{{)}}
{{TBLPROPERTIES (}}
{{"hbase.table.name"="txn_store:transactions_single3_cf",}}
{{"hbase.table.default.storage.type"="binary",}}
{{"hbase.mapred.output.outputtable"="txn_store:transactions_single3_cf",}}
{{"hbase.struct.autogenerate"="true");}}
When executing the select query, it fails with below exception:
Caused by: org.apache.hive.service.cli.HiveSQLException: java.io.IOException:
org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating nsp_scra1
> Support avro data stored in HBase columns
> -----------------------------------------
>
> Key: HIVE-6147
> URL: https://issues.apache.org/jira/browse/HIVE-6147
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
> Affects Versions: 0.12.0, 0.13.0
> Reporter: Swarnim Kulkarni
> Assignee: Swarnim Kulkarni
> Priority: Major
> Fix For: 0.14.0
>
> Attachments: HIVE-6147.1.patch.txt, HIVE-6147.2.patch.txt,
> HIVE-6147.3.patch.txt, HIVE-6147.3.patch.txt, HIVE-6147.4.patch.txt,
> HIVE-6147.5.patch.txt, HIVE-6147.6.patch.txt
>
>
> Presently, the HBase Hive integration supports querying only primitive data
> types in columns. It would be nice to be able to store and query Avro objects
> in HBase columns by making them visible as structs to Hive. This will allow
> Hive to perform ad hoc analysis of HBase data which can be deeply structured.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
