[ 
https://issues.apache.org/jira/browse/HIVE-6897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097067#comment-15097067
 ] 

Dip Kharod commented on HIVE-6897:
----------------------------------

The latest Hive versions now does append by default - so if you write to the 
table via HCatalog by default it will append. This actually made the matter 
worst for us! We learned the lesson hard way - after seeing duplicates.

You need to make the Hive table immutable by setting the table property i.e. 
TBLPROPERTIES ("immutable"="true") so you don't find yourself using an 
unwelcome feature!

As far as I know, you still have to delete and write to "refresh". I would love 
an option at the time of writing. 

> Allow overwrite/append to external Hive table (with partitions) via HCatStorer
> ------------------------------------------------------------------------------
>
>                 Key: HIVE-6897
>                 URL: https://issues.apache.org/jira/browse/HIVE-6897
>             Project: Hive
>          Issue Type: Improvement
>          Components: HCatalog, HiveServer2
>    Affects Versions: 0.12.0
>            Reporter: Dip Kharod
>
> I'm using HCatStorer to write to external Hive table with partition from Pig 
> and have the following different use cases:
> 1) Need to overwrite (aka, refresh) data into table: Currently I end up doing 
> this outside (drop partition and delete HDFS folder) of Pig which is very 
> painful and error-prone
> 2) Need to append (aka, add new file) data to the Hive external 
> table/partition: Again, I end up doing this outside of Pig by copying file in 
> appropriate folder
> It would be very productive for the developers to have both options in 
> HCatStorer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to