[
https://issues.apache.org/jira/browse/PIG-4649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654062#comment-14654062
]
Rohini Palaniswamy commented on PIG-4649:
-----------------------------------------
Filed HIVE-11456 to fix HCatStorer. But also adding support in Pig to not
optimize unions if certain StoreFunc is present. Initially, by default will
turn union optimization off for HCatStorer using a setting.
pig.tez.opt.union.unsupported.storefuncs=org.apache.hcatalog.pig.HCatStorer,org.apache.hive.hcatalog.pig.HCatStorer
Folks with a hive release with HIVE-11456 fixed can override that setting.
> [Pig on Tez] Union followed by HCatStorer misses some data
> ----------------------------------------------------------
>
> Key: PIG-4649
> URL: https://issues.apache.org/jira/browse/PIG-4649
> Project: Pig
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.16.0
>
>
> Script to reproduce:
> {code}
> A = LOAD 'data01.txt' USING PigStorage() as (id:chararray, message:chararray);
> B = LOAD 'data02.txt' USING PigStorage() as (id:chararray, message:chararray);
> C = UNION A, B;
> STORE C INTO 'db.table1' USING org.apache.hive.hcatalog.pig.HCatStorer();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
