[
https://issues.apache.org/jira/browse/HIVE-24930?focusedWorklogId=572013&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-572013
]
ASF GitHub Bot logged work on HIVE-24930:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 25/Mar/21 17:22
Start Date: 25/Mar/21 17:22
Worklog Time Spent: 10m
Work Description: abstractdog merged pull request #2113:
URL: https://github.com/apache/hive/pull/2113
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 572013)
Time Spent: 50m (was: 40m)
> Operator.setDone() short-circuit from child op is not used in vectorized
> codepath (if childSize == 1)
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-24930
> URL: https://issues.apache.org/jira/browse/HIVE-24930
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> This looks like a possible performance regression in case of limit,
> considering the following query:
> {code}
> explain vectorization detail select
> ws_item_sk item_sk, d_date,
> sum(ws_sales_price) over (partition by ws_item_sk order by d_date range
> between 10 preceding and current row) cume_sales,
> last_value(ws_sales_price) over (partition by ws_item_sk order by d_date
> range between 10 preceding and current row) last_price
> from web_sales
> ,date_dim
> where ws_sold_date_sk=d_date_sk
> and d_month_seq between 1214 and 1214+11
> and ws_item_sk is not NULL
> group by ws_item_sk, d_date, ws_sales_price
> limit 100;
> {code}
> in case of vectorized ptf (note: the issue is independent of ptf operator
> though), the whole pipeline process all the rows, which leads to serious
> performance regression (note 1439591782 runtime rows for all the operators
> except limit)
> non-vectorized:
> {code}
> set hive.vectorized.execution.ptf.enabled=false;
> ...
> | Select Operator |
> | Statistics: Num rows: 1415172503/1439591782 Data size:
> 248969569264 Basic stats: COMPLETE Column stats: COMPLETE |
> | PTF Operator |
> | Statistics: Num rows: 1415172503/449131 Data size:
> 248969569264 Basic stats: COMPLETE Column stats: COMPLETE |
> | Select Operator |
> | Statistics: Num rows: 1415172503/11526 Data size:
> 565867418560 Basic stats: COMPLETE Column stats: COMPLETE |
> {code}
> vectorized:
> {code}
> set hive.vectorized.execution.ptf.enabled=true;
> ...
> | Select Operator |
> | Statistics: Num rows: 1415172503/1439591782 Data size:
> 248969569264 Basic stats: COMPLETE Column stats: COMPLETE |
> | PTF Operator |
> | Statistics: Num rows: 1415172503/1439591782 Data size:
> 248969569264 Basic stats: COMPLETE Column stats: COMPLETE |
> | Select Operator |
> | Statistics: Num rows: 1415172503/1439591782 Data size:
> 565867418560 Basic stats: COMPLETE Column stats: COMPLETE |
> | File Output Operator |
> | Statistics: Num rows: 100/11300 Data size: 40000
> Basic stats: COMPLETE Column stats: COMPLETE |
> {code}
> this is because this short-circuit is missing if childSize==1 (from
> vectorForward):
> {code}
> // if all children are done, this operator is also done
> if (childrenDone != 0 && childrenDone == childOperatorsArray.length) {
> setDone(true);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
