[
https://issues.apache.org/jira/browse/HCATALOG-237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207210#comment-13207210
]
Alan Gates commented on HCATALOG-237:
-------------------------------------
Almost nothing. Pig will work with HCat as before, via HCatLoader and
HCatStorer. The one piece we haven't started working on is a connector to use
existing Pig load/store functions for a given format with HCatalog, as existed
with storage drivers. We need these, but we haven't gotten started on them yet.
> Switch from using StorageDrivers to SerDes to do data (de)serialization
> -----------------------------------------------------------------------
>
> Key: HCATALOG-237
> URL: https://issues.apache.org/jira/browse/HCATALOG-237
> Project: HCatalog
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Alan Gates
> Fix For: 0.4
>
> Attachments: HCat StorageDriver to SerDe Conversion design notes
> v2.pdf, StorageDriver-StorageHandler replacement.xlsx
>
>
> HCatalog started by creating its own classes, InputStorageDriver and
> OutputStorageDriver, to do data conversion between the storage layer
> Input/OutputFormats and the HCatInput/OutputFormats. These provide very
> similar functionality to Hive's SerDe class, though with a much simpler
> interface.
> This usage of separate classes has led to a number of issues for HCatalog.
> One, it cannot make use of existing Hive SerDes. Two, it has led to a need
> to make HCat specific extensions of Hive interfaces (such as the
> StorageHandler) to provide the StorageDescriptors. Three, it means that
> users who already have Hive installed cannot use HCatalog without first
> updating every partition in their metastore with storage driver information.
> I propose we switch to using SerDes for this. To address the issue of the
> more complicated SerDe interface we can provide adaptor classes that make
> writing new SerDes easy in simple cases.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
