Re: [jira] [Created] (HIVE-20381) Vectorization: Reduce dedup of GroupBy + PTF turns off vectorization

2018-08-13 Thread mathes waran 
Hi team,

I would request  you to unsubscribe my account from these mailing lists.

Thanks,
Matheswaran.S

On Tue, Aug 14, 2018 at 6:00 AM Gopal V (JIRA)  wrote:

> Gopal V created HIVE-20381:
> --
>
>  Summary: Vectorization: Reduce dedup of GroupBy + PTF turns
> off vectorization
>  Key: HIVE-20381
>  URL: https://issues.apache.org/jira/browse/HIVE-20381
>  Project: Hive
>   Issue Type: Bug
>   Components: Vectorization
> Affects Versions: 3.1.0, 4.0.0
> Reporter: Gopal V
>
>
> One of the PTF Reducers in Query51 is not vectorized because there's a
> reduce deduplication which combines a group-by and a windowing shuffle.
>
> {code}
> | Reducer 8  |
> | Execution mode: llap   |
> | Reduce Vectorization:  |
> | enabled: true  |
> | enableConditionsMet:
> hive.vectorized.execution.reduce.enabled IS true, hive.execution.engine tez
> IN [tez, spark] IS true |
> | notVectorizedReason: PTF operator: Only PTF directly
> under reduce-shuffle is supported |
> | vectorized: false
> {code}
>
> It vectorizes all PTF vertices (after HIVE-20367), with {{ set
> hive.optimize.reducededuplication=false;}}
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)
>

[jira] [Created] (HIVE-20381) Vectorization: Reduce dedup of GroupBy + PTF turns off vectorization

2018-08-13 Thread Gopal V (JIRA) 
Gopal V created HIVE-20381:
--

 Summary: Vectorization: Reduce dedup of GroupBy + PTF turns off 
vectorization
 Key: HIVE-20381
 URL: https://issues.apache.org/jira/browse/HIVE-20381
 Project: Hive
  Issue Type: Bug
  Components: Vectorization
Affects Versions: 3.1.0, 4.0.0
Reporter: Gopal V


One of the PTF Reducers in Query51 is not vectorized because there's a reduce 
deduplication which combines a group-by and a windowing shuffle.

{code}
| Reducer 8  |
| Execution mode: llap   |
| Reduce Vectorization:  |
| enabled: true  |
| enableConditionsMet: hive.vectorized.execution.reduce.enabled 
IS true, hive.execution.engine tez IN [tez, spark] IS true |
| notVectorizedReason: PTF operator: Only PTF directly under 
reduce-shuffle is supported |
| vectorized: false   
{code}

It vectorizes all PTF vertices (after HIVE-20367), with {{ set 
hive.optimize.reducededuplication=false;}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)