[
https://issues.apache.org/jira/browse/HIVE-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Remus Rusanu updated HIVE-6873:
-------------------------------
Attachment: image004.png
image003.png
image002.png
image001.png
set hive.map.groupby.sorted=true; was the missing part for me…
From: Jitendra Nath Pandey (JIRA) [mailto:[email protected]]
Sent: Friday, April 11, 2014 1:02 AM
To: Remus Rusanu
Subject: [jira] [Commented] (HIVE-6873) DISTINCT clause in aggregates is
handled incorrectly by vectorized execution
[cid:[email protected]]
Jitendra Nath
Pandey<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=jnp>
commented on [Bug] HIVE-6873<https://issues.apache.org/jira/browse/HIVE-6873>
Re: DISTINCT clause in aggregates is handled incorrectly by vectorized
execution<https://issues.apache.org/jira/browse/HIVE-6873>
Here is a scenario where we get incorrect result. It shows up on sorted
bucketed column with hive.map.groupby.sorted=true, and only on group by queries
with no keys.
Here are the steps:
hive> Create table T(a int, b int) clustered by (a) sorted by (a) stored as orc;
load following data:
300 1
300 1
300 1
300 1
300 1
hive> set hive.map.groupby.sorted=true;
hive> select sum(distinct a) from T; // Incorrect result.
hive> select count(distinct a) from T; // This is also incorrect.
[Add Comment]<https://issues.apache.org/jira/browse/HIVE-6873#add-comment>
Add Comment<https://issues.apache.org/jira/browse/HIVE-6873#add-comment>
This message was sent by Atlassian JIRA (v6.2#6252-sha1:aa34325)
[Atlassian logo]
> DISTINCT clause in aggregates is handled incorrectly by vectorized execution
> ----------------------------------------------------------------------------
>
> Key: HIVE-6873
> URL: https://issues.apache.org/jira/browse/HIVE-6873
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.13.0, 0.14.0
> Reporter: Remus Rusanu
> Assignee: Remus Rusanu
> Attachments: HIVE-6873.1.patch, HIVE-6873.2.patch, HIVE-6873.3.patch,
> image001.png, image002.png, image003.png, image004.png
>
>
> The vectorized aggregates ignore the DISTINCT clause. This cause incorrect
> results. Due to how GroupByOperatorDesc adds the DISTINCT keys to the overall
> aggregate keys the vectorized aggregates do account for the extra key, but
> they do not process the data correctly for the key. the reduce side the
> aggregates the input from the vectorized map side to results that are only
> sometimes correct but mostly incorrect. HIVE-4607 tracks the proper fix, but
> meantime I'm filing a bug to disable vectorized execution if DISTINCT is
> present. Fix is trivial.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
