[jira] [Updated] (HIVE-22501) Stats reported multiple times during MR execution for UNION queries

[Daniel Voros (Jira)]

 [ 
https://issues.apache.org/jira/browse/HIVE-22501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Voros updated HIVE-22501:

Status: Patch Available  (was: Open)

Attached patch #1 that only let's operators report their stats once.

> Stats reported multiple times during MR execution for UNION queries
> ---
>
> Key: HIVE-22501
> URL: https://issues.apache.org/jira/browse/HIVE-22501
> Project: Hive
>  Issue Type: Bug
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Major
>  Labels: mapreduce
> Attachments: HIVE-22501.1.patch
>
>
> Take the following example:
> {code}
> set hive.execution.engine=mr;
> create table tb(id string) stored as orc;
> insert into tb values('1');
> create table tb2 like tb stored as orc;
> insert into tb2 select * from tb union all select * from tb;
> {code}
> Last insert results in 2 records in the table, but 
> {{TOTAL_TABLE_ROWS_WRITTEN}} statistic (and number of affected rows on the 
> consolse) is 4.
> We seem to traverse the operator graph multiple times starting from every TS 
> operator and increment the counters every time we hit the FS operator. 
> UNION-ing the table 3 times results in 9 TOTAL_TABLE_ROWS_WRITTEN.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22501) Stats reported multiple times during MR execution for UNION queries

[Daniel Voros (Jira)]

 [ 
https://issues.apache.org/jira/browse/HIVE-22501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Voros updated HIVE-22501:

Attachment: HIVE-22501.1.patch

> Stats reported multiple times during MR execution for UNION queries
> ---
>
> Key: HIVE-22501
> URL: https://issues.apache.org/jira/browse/HIVE-22501
> Project: Hive
>  Issue Type: Bug
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Major
>  Labels: mapreduce
> Attachments: HIVE-22501.1.patch
>
>
> Take the following example:
> {code}
> set hive.execution.engine=mr;
> create table tb(id string) stored as orc;
> insert into tb values('1');
> create table tb2 like tb stored as orc;
> insert into tb2 select * from tb union all select * from tb;
> {code}
> Last insert results in 2 records in the table, but 
> {{TOTAL_TABLE_ROWS_WRITTEN}} statistic (and number of affected rows on the 
> consolse) is 4.
> We seem to traverse the operator graph multiple times starting from every TS 
> operator and increment the counters every time we hit the FS operator. 
> UNION-ing the table 3 times results in 9 TOTAL_TABLE_ROWS_WRITTEN.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)