[
https://issues.apache.org/jira/browse/HIVE-6897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14266353#comment-14266353
]
Rohan Aletty commented on HIVE-6897:
------------------------------------
Commenting on this since there hasn't been an update for a while now. We're
running an ETL pipeline using Pig/Hive on CDH5 and we tried to dynamically
partition our output data (serialized through Avro) using HCatalog. However, it
was necessary for us to be able to append onto an existing partition.
We are able to store into a new partition and load from it again. However, we
cannot store into that partition again, similar to to the findings on here:
http://alvincjin.blogspot.com/2014/07/hcatalog-doesnt-support-appending.html.
> Allow overwrite/append to external Hive table (with partitions) via HCatStorer
> ------------------------------------------------------------------------------
>
> Key: HIVE-6897
> URL: https://issues.apache.org/jira/browse/HIVE-6897
> Project: Hive
> Issue Type: Improvement
> Components: HCatalog, HiveServer2
> Affects Versions: 0.12.0
> Reporter: Dip Kharod
>
> I'm using HCatStorer to write to external Hive table with partition from Pig
> and have the following different use cases:
> 1) Need to overwrite (aka, refresh) data into table: Currently I end up doing
> this outside (drop partition and delete HDFS folder) of Pig which is very
> painful and error-prone
> 2) Need to append (aka, add new file) data to the Hive external
> table/partition: Again, I end up doing this outside of Pig by copying file in
> appropriate folder
> It would be very productive for the developers to have both options in
> HCatStorer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
