[jira] [Created] (HIVE-8309) CBO: Fix OB by removing constraining DT, Use external names for col Aliases, Remove unnecessary Selects, Make DT Name counter query specific
Laljo John Pullokkaran created HIVE-8309: Summary: CBO: Fix OB by removing constraining DT, Use external names for col Aliases, Remove unnecessary Selects, Make DT Name counter query specific Key: HIVE-8309 URL: https://issues.apache.org/jira/browse/HIVE-8309 Project: Hive Issue Type: Sub-task Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8309) CBO: Fix OB by removing constraining DT, Use external names for col Aliases, Remove unnecessary Selects, Make DT Name counter query specific
[ https://issues.apache.org/jira/browse/HIVE-8309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8309: - Attachment: HIVE-8309.patch > CBO: Fix OB by removing constraining DT, Use external names for col Aliases, > Remove unnecessary Selects, Make DT Name counter query specific > > > Key: HIVE-8309 > URL: https://issues.apache.org/jira/browse/HIVE-8309 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8309.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8309) CBO: Fix OB by removing constraining DT, Use external names for col Aliases, Remove unnecessary Selects, Make DT Name counter query specific
[ https://issues.apache.org/jira/browse/HIVE-8309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8309: - Status: Open (was: Patch Available) > CBO: Fix OB by removing constraining DT, Use external names for col Aliases, > Remove unnecessary Selects, Make DT Name counter query specific > > > Key: HIVE-8309 > URL: https://issues.apache.org/jira/browse/HIVE-8309 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8309.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8309) CBO: Fix OB by removing constraining DT, Use external names for col Aliases, Remove unnecessary Selects, Make DT Name counter query specific
[ https://issues.apache.org/jira/browse/HIVE-8309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8309: - Status: Patch Available (was: Open) > CBO: Fix OB by removing constraining DT, Use external names for col Aliases, > Remove unnecessary Selects, Make DT Name counter query specific > > > Key: HIVE-8309 > URL: https://issues.apache.org/jira/browse/HIVE-8309 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8309.1.patch, HIVE-8309.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8309) CBO: Fix OB by removing constraining DT, Use external names for col Aliases, Remove unnecessary Selects, Make DT Name counter query specific
[ https://issues.apache.org/jira/browse/HIVE-8309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8309: - Attachment: HIVE-8309.1.patch > CBO: Fix OB by removing constraining DT, Use external names for col Aliases, > Remove unnecessary Selects, Make DT Name counter query specific > > > Key: HIVE-8309 > URL: https://issues.apache.org/jira/browse/HIVE-8309 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8309.1.patch, HIVE-8309.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8395) CBO: enable by default
[ https://issues.apache.org/jira/browse/HIVE-8395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162941#comment-14162941 ] Laljo John Pullokkaran commented on HIVE-8395: -- Xuefu, I understand your concern. However it would be great to know up front if some code change breaks CBO. Hence the reason for enabling CBO by default. BTW, CBO won't be on by default until all the Hive Unit tests pass. > CBO: enable by default > -- > > Key: HIVE-8395 > URL: https://issues.apache.org/jira/browse/HIVE-8395 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.14.0 > > Attachments: HIVE-8395.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8222) CBO Trunk Merge: Fix Check Style issues
[ https://issues.apache.org/jira/browse/HIVE-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165695#comment-14165695 ] Laljo John Pullokkaran commented on HIVE-8222: -- Many of the issues are already addressed in other patches. I will look at the patch by this weekend. > CBO Trunk Merge: Fix Check Style issues > --- > > Key: HIVE-8222 > URL: https://issues.apache.org/jira/browse/HIVE-8222 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8222.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8222) CBO Trunk Merge: Fix Check Style issues
[ https://issues.apache.org/jira/browse/HIVE-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165709#comment-14165709 ] Laljo John Pullokkaran commented on HIVE-8222: -- Lars, Could you add RB link in the bug? > CBO Trunk Merge: Fix Check Style issues > --- > > Key: HIVE-8222 > URL: https://issues.apache.org/jira/browse/HIVE-8222 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8222.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8222) CBO Trunk Merge: Fix Check Style issues
[ https://issues.apache.org/jira/browse/HIVE-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8222: - Assignee: Lars Francke (was: Laljo John Pullokkaran) > CBO Trunk Merge: Fix Check Style issues > --- > > Key: HIVE-8222 > URL: https://issues.apache.org/jira/browse/HIVE-8222 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Lars Francke > Attachments: HIVE-8222.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8222) CBO Trunk Merge: Fix Check Style issues
[ https://issues.apache.org/jira/browse/HIVE-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14167888#comment-14167888 ] Laljo John Pullokkaran commented on HIVE-8222: -- Lars, thanks for rebasing the patch. I have two generic comments: 1. This patch involves non CBO files 2. Removal of type casts in may places in the code. Since this is specific to CBO, please restrict style changes to CBO. Removal of type cast, at the least may require assertions for expected type. > CBO Trunk Merge: Fix Check Style issues > --- > > Key: HIVE-8222 > URL: https://issues.apache.org/jira/browse/HIVE-8222 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Lars Francke > Attachments: HIVE-8222.1.patch, HIVE-8222.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7914) Simplify join predicates for CBO to avoid cross products
[
https://issues.apache.org/jira/browse/HIVE-7914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-7914:
-
Attachment: HIVE-7914.patch
> Simplify join predicates for CBO to avoid cross products
>
>
> Key: HIVE-7914
> URL: https://issues.apache.org/jira/browse/HIVE-7914
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.13.1
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
> Fix For: 0.14.0
>
> Attachments: HIVE-7914.patch
>
>
> Simplify join predicates for disjunctive predicates to avoid cross products.
> For TPC-DS query 13 we generate a cross products.
> The join predicate on (store_sales x customer_demographics) , (store_sales x
> household_demographics) and (store_sales x customer_address) can be pull up
> to avoid the cross products
> {code}
> select avg(ss_quantity)
>,avg(ss_ext_sales_price)
>,avg(ss_ext_wholesale_cost)
>,sum(ss_ext_wholesale_cost)
> from store_sales
> ,store
> ,customer_demographics
> ,household_demographics
> ,customer_address
> ,date_dim
> where store.s_store_sk = store_sales.ss_store_sk
> and store_sales.ss_sold_date_sk = date_dim.d_date_sk and date_dim.d_year =
> 2001
> and((store_sales.ss_hdemo_sk=household_demographics.hd_demo_sk
> and customer_demographics.cd_demo_sk = store_sales.ss_cdemo_sk
> and customer_demographics.cd_marital_status = 'M'
> and customer_demographics.cd_education_status = '4 yr Degree'
> and store_sales.ss_sales_price between 100.00 and 150.00
> and household_demographics.hd_dep_count = 3
> )or
> (store_sales.ss_hdemo_sk=household_demographics.hd_demo_sk
> and customer_demographics.cd_demo_sk = store_sales.ss_cdemo_sk
> and customer_demographics.cd_marital_status = 'D'
> and customer_demographics.cd_education_status = 'Primary'
> and store_sales.ss_sales_price between 50.00 and 100.00
> and household_demographics.hd_dep_count = 1
> ) or
> (store_sales.ss_hdemo_sk=household_demographics.hd_demo_sk
> and customer_demographics.cd_demo_sk = ss_cdemo_sk
> and customer_demographics.cd_marital_status = 'U'
> and customer_demographics.cd_education_status = 'Advanced Degree'
> and store_sales.ss_sales_price between 150.00 and 200.00
> and household_demographics.hd_dep_count = 1
> ))
> and((store_sales.ss_addr_sk = customer_address.ca_address_sk
> and customer_address.ca_country = 'United States'
> and customer_address.ca_state in ('KY', 'GA', 'NM')
> and store_sales.ss_net_profit between 100 and 200
> ) or
> (store_sales.ss_addr_sk = customer_address.ca_address_sk
> and customer_address.ca_country = 'United States'
> and customer_address.ca_state in ('MT', 'OR', 'IN')
> and store_sales.ss_net_profit between 150 and 300
> ) or
> (store_sales.ss_addr_sk = customer_address.ca_address_sk
> and customer_address.ca_country = 'United States'
> and customer_address.ca_state in ('WI', 'MO', 'WV')
> and store_sales.ss_net_profit between 50 and 250
> ))
> ;
> {code}
> This is the plan currently generated without any predicate simplification
> {code}
> Warning: Map Join MAPJOIN[59][bigTable=?] in task 'Map 8' is a cross product
> Warning: Map Join MAPJOIN[58][bigTable=?] in task 'Map 8' is a cross product
> Warning: Shuffle Join JOIN[29][tables = [$hdt$_5, $hdt$_6]] in Stage 'Reducer
> 2' is a cross product
> OK
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 7 <- Map 8 (BROADCAST_EDGE)
> Map 8 <- Map 5 (BROADCAST_EDGE), Map 6 (BROADCAST_EDGE)
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (BROADCAST_EDGE), Map 7
> (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> DagName: mmokhtar_20140828155050_7059c24b-501b-4683-86c0-4f3c023f0b0e:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: customer_address
> Statistics: Num rows: 4000 Data size: 40595195284 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: ca_address_sk (type: int), ca_state (type:
> string), ca_country (type: string)
> outputColumnNames: _col0, _col1, _col2
> Statistics: Num rows: 4000 Data size: 40595195284
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 4000 Data size: 40595195284
> Basic stats: COMPLETE Column stats: NONE
> value expressions: _
[jira] [Updated] (HIVE-7914) Simplify join predicates for CBO to avoid cross products
[
https://issues.apache.org/jira/browse/HIVE-7914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-7914:
-
Status: Patch Available (was: Open)
> Simplify join predicates for CBO to avoid cross products
>
>
> Key: HIVE-7914
> URL: https://issues.apache.org/jira/browse/HIVE-7914
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.13.1
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
> Fix For: 0.14.0
>
> Attachments: HIVE-7914.patch
>
>
> Simplify join predicates for disjunctive predicates to avoid cross products.
> For TPC-DS query 13 we generate a cross products.
> The join predicate on (store_sales x customer_demographics) , (store_sales x
> household_demographics) and (store_sales x customer_address) can be pull up
> to avoid the cross products
> {code}
> select avg(ss_quantity)
>,avg(ss_ext_sales_price)
>,avg(ss_ext_wholesale_cost)
>,sum(ss_ext_wholesale_cost)
> from store_sales
> ,store
> ,customer_demographics
> ,household_demographics
> ,customer_address
> ,date_dim
> where store.s_store_sk = store_sales.ss_store_sk
> and store_sales.ss_sold_date_sk = date_dim.d_date_sk and date_dim.d_year =
> 2001
> and((store_sales.ss_hdemo_sk=household_demographics.hd_demo_sk
> and customer_demographics.cd_demo_sk = store_sales.ss_cdemo_sk
> and customer_demographics.cd_marital_status = 'M'
> and customer_demographics.cd_education_status = '4 yr Degree'
> and store_sales.ss_sales_price between 100.00 and 150.00
> and household_demographics.hd_dep_count = 3
> )or
> (store_sales.ss_hdemo_sk=household_demographics.hd_demo_sk
> and customer_demographics.cd_demo_sk = store_sales.ss_cdemo_sk
> and customer_demographics.cd_marital_status = 'D'
> and customer_demographics.cd_education_status = 'Primary'
> and store_sales.ss_sales_price between 50.00 and 100.00
> and household_demographics.hd_dep_count = 1
> ) or
> (store_sales.ss_hdemo_sk=household_demographics.hd_demo_sk
> and customer_demographics.cd_demo_sk = ss_cdemo_sk
> and customer_demographics.cd_marital_status = 'U'
> and customer_demographics.cd_education_status = 'Advanced Degree'
> and store_sales.ss_sales_price between 150.00 and 200.00
> and household_demographics.hd_dep_count = 1
> ))
> and((store_sales.ss_addr_sk = customer_address.ca_address_sk
> and customer_address.ca_country = 'United States'
> and customer_address.ca_state in ('KY', 'GA', 'NM')
> and store_sales.ss_net_profit between 100 and 200
> ) or
> (store_sales.ss_addr_sk = customer_address.ca_address_sk
> and customer_address.ca_country = 'United States'
> and customer_address.ca_state in ('MT', 'OR', 'IN')
> and store_sales.ss_net_profit between 150 and 300
> ) or
> (store_sales.ss_addr_sk = customer_address.ca_address_sk
> and customer_address.ca_country = 'United States'
> and customer_address.ca_state in ('WI', 'MO', 'WV')
> and store_sales.ss_net_profit between 50 and 250
> ))
> ;
> {code}
> This is the plan currently generated without any predicate simplification
> {code}
> Warning: Map Join MAPJOIN[59][bigTable=?] in task 'Map 8' is a cross product
> Warning: Map Join MAPJOIN[58][bigTable=?] in task 'Map 8' is a cross product
> Warning: Shuffle Join JOIN[29][tables = [$hdt$_5, $hdt$_6]] in Stage 'Reducer
> 2' is a cross product
> OK
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 7 <- Map 8 (BROADCAST_EDGE)
> Map 8 <- Map 5 (BROADCAST_EDGE), Map 6 (BROADCAST_EDGE)
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (BROADCAST_EDGE), Map 7
> (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> DagName: mmokhtar_20140828155050_7059c24b-501b-4683-86c0-4f3c023f0b0e:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: customer_address
> Statistics: Num rows: 4000 Data size: 40595195284 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: ca_address_sk (type: int), ca_state (type:
> string), ca_country (type: string)
> outputColumnNames: _col0, _col1, _col2
> Statistics: Num rows: 4000 Data size: 40595195284
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 4000 Data size: 40595195284
> Basic stats: COMPLETE Column stats: NONE
> value expre
[jira] [Commented] (HIVE-7913) Simplify filter predicates for CBO
[
https://issues.apache.org/jira/browse/HIVE-7913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14167934#comment-14167934
]
Laljo John Pullokkaran commented on HIVE-7913:
--
Duplicate of HIVE-7914.
> Simplify filter predicates for CBO
> --
>
> Key: HIVE-7913
> URL: https://issues.apache.org/jira/browse/HIVE-7913
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.13.1
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
> Fix For: 0.14.0
>
>
> Simplify predicates for disjunctive predicates so that can get pushed down to
> the scan.
> For TPC-DS query 13 we push down predicates in the following form
> where c_martial_status in ('M','D','U') etc..
> {code}
> select avg(ss_quantity)
>,avg(ss_ext_sales_price)
>,avg(ss_ext_wholesale_cost)
>,sum(ss_ext_wholesale_cost)
> from store_sales
> ,store
> ,customer_demographics
> ,household_demographics
> ,customer_address
> ,date_dim
> where store.s_store_sk = store_sales.ss_store_sk
> and store_sales.ss_sold_date_sk = date_dim.d_date_sk and date_dim.d_year =
> 2001
> and((store_sales.ss_hdemo_sk=household_demographics.hd_demo_sk
> and customer_demographics.cd_demo_sk = store_sales.ss_cdemo_sk
> and customer_demographics.cd_marital_status = 'M'
> and customer_demographics.cd_education_status = '4 yr Degree'
> and store_sales.ss_sales_price between 100.00 and 150.00
> and household_demographics.hd_dep_count = 3
> )or
> (store_sales.ss_hdemo_sk=household_demographics.hd_demo_sk
> and customer_demographics.cd_demo_sk = store_sales.ss_cdemo_sk
> and customer_demographics.cd_marital_status = 'D'
> and customer_demographics.cd_education_status = 'Primary'
> and store_sales.ss_sales_price between 50.00 and 100.00
> and household_demographics.hd_dep_count = 1
> ) or
> (store_sales.ss_hdemo_sk=household_demographics.hd_demo_sk
> and customer_demographics.cd_demo_sk = ss_cdemo_sk
> and customer_demographics.cd_marital_status = 'U'
> and customer_demographics.cd_education_status = 'Advanced Degree'
> and store_sales.ss_sales_price between 150.00 and 200.00
> and household_demographics.hd_dep_count = 1
> ))
> and((store_sales.ss_addr_sk = customer_address.ca_address_sk
> and customer_address.ca_country = 'United States'
> and customer_address.ca_state in ('KY', 'GA', 'NM')
> and store_sales.ss_net_profit between 100 and 200
> ) or
> (store_sales.ss_addr_sk = customer_address.ca_address_sk
> and customer_address.ca_country = 'United States'
> and customer_address.ca_state in ('MT', 'OR', 'IN')
> and store_sales.ss_net_profit between 150 and 300
> ) or
> (store_sales.ss_addr_sk = customer_address.ca_address_sk
> and customer_address.ca_country = 'United States'
> and customer_address.ca_state in ('WI', 'MO', 'WV')
> and store_sales.ss_net_profit between 50 and 250
> ))
> ;
> {code}
> This is the plan currently generated without any predicate simplification
> {code}
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 7 <- Map 8 (BROADCAST_EDGE)
> Map 8 <- Map 5 (BROADCAST_EDGE), Map 6 (BROADCAST_EDGE)
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (BROADCAST_EDGE), Map 7
> (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> DagName: mmokhtar_20140828155050_7059c24b-501b-4683-86c0-4f3c023f0b0e:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: customer_address
> Statistics: Num rows: 4000 Data size: 40595195284 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: ca_address_sk (type: int), ca_state (type:
> string), ca_country (type: string)
> outputColumnNames: _col0, _col1, _col2
> Statistics: Num rows: 4000 Data size: 40595195284
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 4000 Data size: 40595195284
> Basic stats: COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type:
> string), _col2 (type: string)
> Execution mode: vectorized
> Map 4
> Map Operator Tree:
> TableScan
> alias: date_dim
> filterExpr: ((d_year = 2001) and d_date_sk is not null)
> (type: boolean)
> Statistics: Num rows: 73049 Data size: 81741831 Basic
> stats: COMPLETE Column
[jira] [Updated] (HIVE-7985) With CBO enabled cross product is generated when a subquery is present
[
https://issues.apache.org/jira/browse/HIVE-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-7985:
-
Attachment: HIVE-7985.patch
> With CBO enabled cross product is generated when a subquery is present
> --
>
> Key: HIVE-7985
> URL: https://issues.apache.org/jira/browse/HIVE-7985
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.14.0
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
>Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-7985.patch
>
>
> This is a regression introduced in the latest build of the CBO branch.
> Removing the subquery for item will remove the cross products
> Query
> {code}
> select i_item_id,sum(ss_ext_sales_price) total_sales from store_sales,
> date_dim, item where item.i_item_id in (select i.i_item_id from item i where
> i_color in ('purple','burlywood','indian')) and ss_item_sk = i_item_sk and
> ss_sold_date_sk = d_date_sk and d_year = 2001 and d_moy = 1 group by
> i_item_id;
> {code}
> {code}
> Warning: Map Join MAPJOIN[38][bigTable=?] in task 'Map 1' is a cross product
> Warning: Map Join MAPJOIN[39][bigTable=store_sales] in task 'Map 4' is a
> cross product
> OK
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 1 <- Map 3 (BROADCAST_EDGE)
> Map 4 <- Map 1 (BROADCAST_EDGE), Map 2 (BROADCAST_EDGE)
> Reducer 5 <- Map 4 (SIMPLE_EDGE)
> DagName: mmokhtar_20140904141313_9c253f7e-aad1-4ca4-9be1-ea45e3d34496:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: item
> filterExpr: (true and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: i_item_id is not null (type: boolean)
> Statistics: Num rows: 231000 Data size: 331931080 Basic
> stats: COMPLETE Column stats: NONE
> Map Join Operator
> condition map:
>Inner Join 0 to 1
> condition expressions:
> 0 {i_item_sk} {i_item_id}
> 1 {d_date_sk}
> keys:
> 0
> 1
> outputColumnNames: _col0, _col1, _col25
> input vertices:
> 1 Map 3
> Statistics: Num rows: 254100 Data size: 365124192 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: _col0 (type: int), _col1 (type: string),
> _col25 (type: int)
> outputColumnNames: _col0, _col1, _col25
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type:
> string), _col25 (type: int)
> Execution mode: vectorized
> Map 2
> Map Operator Tree:
> TableScan
> alias: i
> filterExpr: ((i_color) IN ('purple', 'burlywood', 'indian')
> and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: ((i_color) IN ('purple', 'burlywood',
> 'indian') and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: i_item_id (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Group By Operator
> keys: _col0 (type: string)
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> key expressions: _col0 (type: string)
>
[jira] [Commented] (HIVE-7985) With CBO enabled cross product is generated when a subquery is present
[
https://issues.apache.org/jira/browse/HIVE-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14167938#comment-14167938
]
Laljo John Pullokkaran commented on HIVE-7985:
--
Need to wait for OPTIQ-438 to get checked in.
> With CBO enabled cross product is generated when a subquery is present
> --
>
> Key: HIVE-7985
> URL: https://issues.apache.org/jira/browse/HIVE-7985
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.14.0
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
>Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-7985.patch
>
>
> This is a regression introduced in the latest build of the CBO branch.
> Removing the subquery for item will remove the cross products
> Query
> {code}
> select i_item_id,sum(ss_ext_sales_price) total_sales from store_sales,
> date_dim, item where item.i_item_id in (select i.i_item_id from item i where
> i_color in ('purple','burlywood','indian')) and ss_item_sk = i_item_sk and
> ss_sold_date_sk = d_date_sk and d_year = 2001 and d_moy = 1 group by
> i_item_id;
> {code}
> {code}
> Warning: Map Join MAPJOIN[38][bigTable=?] in task 'Map 1' is a cross product
> Warning: Map Join MAPJOIN[39][bigTable=store_sales] in task 'Map 4' is a
> cross product
> OK
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 1 <- Map 3 (BROADCAST_EDGE)
> Map 4 <- Map 1 (BROADCAST_EDGE), Map 2 (BROADCAST_EDGE)
> Reducer 5 <- Map 4 (SIMPLE_EDGE)
> DagName: mmokhtar_20140904141313_9c253f7e-aad1-4ca4-9be1-ea45e3d34496:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: item
> filterExpr: (true and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: i_item_id is not null (type: boolean)
> Statistics: Num rows: 231000 Data size: 331931080 Basic
> stats: COMPLETE Column stats: NONE
> Map Join Operator
> condition map:
>Inner Join 0 to 1
> condition expressions:
> 0 {i_item_sk} {i_item_id}
> 1 {d_date_sk}
> keys:
> 0
> 1
> outputColumnNames: _col0, _col1, _col25
> input vertices:
> 1 Map 3
> Statistics: Num rows: 254100 Data size: 365124192 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: _col0 (type: int), _col1 (type: string),
> _col25 (type: int)
> outputColumnNames: _col0, _col1, _col25
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type:
> string), _col25 (type: int)
> Execution mode: vectorized
> Map 2
> Map Operator Tree:
> TableScan
> alias: i
> filterExpr: ((i_color) IN ('purple', 'burlywood', 'indian')
> and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: ((i_color) IN ('purple', 'burlywood',
> 'indian') and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: i_item_id (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Group By Operator
> keys: _col0 (type: string)
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operat
[jira] [Commented] (HIVE-8222) CBO Trunk Merge: Fix Check Style issues
[ https://issues.apache.org/jira/browse/HIVE-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171848#comment-14171848 ] Laljo John Pullokkaran commented on HIVE-8222: -- In PlanModifier the obj equality check is correct. I am not sure about other places; i will check these as i review your latest patch > CBO Trunk Merge: Fix Check Style issues > --- > > Key: HIVE-8222 > URL: https://issues.apache.org/jira/browse/HIVE-8222 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Lars Francke > Attachments: HIVE-8222.1.patch, HIVE-8222.2.patch, HIVE-8222.3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8222) CBO Trunk Merge: Fix Check Style issues
[ https://issues.apache.org/jira/browse/HIVE-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172886#comment-14172886 ] Laljo John Pullokkaran commented on HIVE-8222: -- Lars, 1. I am assuming you are just enforcing Hive's existing coding conventions 2. Please rebase the patch. Thanks > CBO Trunk Merge: Fix Check Style issues > --- > > Key: HIVE-8222 > URL: https://issues.apache.org/jira/browse/HIVE-8222 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Lars Francke > Attachments: HIVE-8222.1.patch, HIVE-8222.2.patch, HIVE-8222.3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8462) CBO duplicates columns
[
https://issues.apache.org/jira/browse/HIVE-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173132#comment-14173132
]
Laljo John Pullokkaran commented on HIVE-8462:
--
+1
> CBO duplicates columns
> --
>
> Key: HIVE-8462
> URL: https://issues.apache.org/jira/browse/HIVE-8462
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
> Attachments: HIVE-8462.01.patch, HIVE-8462.patch
>
>
> {noformat}
> select *, rank() over(partition by key order by value) as rr from src1
> {noformat}
> Original plan appears to be incorrect:
> {noformat}
> HiveProjectRel(key=[$0], value=[$1], (tok_function rank (tok_windowspec
> (tok_partitioningspec (tok_distributeby (tok_table_or_col key)) (tok_orderby
> (tok_tabsortcolnameasc (tok_table_or_col value))=[$5], rr=[$5])
> HiveProjectRel(key=[$0], value=[$1], block__offset__inside__file=[$2],
> input__file__name=[$3], row__id=[$4], (tok_function rank (tok_windowspec
> (tok_partitioningspec (tok_distributeby (tok_table_or_col key)) (tok_orderby
> (tok_tabsortcolnameasc (tok_table_or_col value))=[rank() OVER (PARTITION
> BY $0 ORDER BY $1 ROWS BETWEEN 2147483647 FOLLOWING AND 2147483647
> PRECEDING)])
> HiveTableScanRel(table=[[default.src1]])
> {noformat}
> and final AST has {noformat}
> TOK_SELEXPR
> .
>TOK_TABLE_OR_COL
> $hdt$_0
>(tok_function rank (tok_windowspec (tok_partitioningspec
> (tok_distributeby (tok_table_or_col key)) (tok_orderby (tok_tabsortcolnameasc
> (tok_table_or_col value))
> (tok_function rank (tok_windowspec (tok_partitioningspec
> (tok_distributeby (tok_table_or_col key)) (tok_orderby (tok_tabsortcolnameasc
> (tok_table_or_col value))
> TOK_SELEXPR
> .
>TOK_TABLE_OR_COL
> $hdt$_0
>(tok_function rank (tok_windowspec (tok_partitioningspec
> (tok_distributeby (tok_table_or_col key)) (tok_orderby (tok_tabsortcolnameasc
> (tok_table_or_col value))
> rr
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8522) Update Calcite Version to >0.9.2-incubating-SNAPSHOT
Laljo John Pullokkaran created HIVE-8522: Summary: Update Calcite Version to >0.9.2-incubating-SNAPSHOT Key: HIVE-8522 URL: https://issues.apache.org/jira/browse/HIVE-8522 Project: Hive Issue Type: Sub-task Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8522) Update Calcite Version to 0.9.2-incubating-SNAPSHOT
[ https://issues.apache.org/jira/browse/HIVE-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8522: - Summary: Update Calcite Version to 0.9.2-incubating-SNAPSHOT (was: Update Calcite Version to >0.9.2-incubating-SNAPSHOT) > Update Calcite Version to 0.9.2-incubating-SNAPSHOT > --- > > Key: HIVE-8522 > URL: https://issues.apache.org/jira/browse/HIVE-8522 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8522) Update Calcite Version to 0.9.2-incubating-SNAPSHOT
[ https://issues.apache.org/jira/browse/HIVE-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8522: - Status: Patch Available (was: Open) > Update Calcite Version to 0.9.2-incubating-SNAPSHOT > --- > > Key: HIVE-8522 > URL: https://issues.apache.org/jira/browse/HIVE-8522 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8522.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8522) Update Calcite Version to 0.9.2-incubating-SNAPSHOT
[ https://issues.apache.org/jira/browse/HIVE-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8522: - Attachment: HIVE-8522.patch > Update Calcite Version to 0.9.2-incubating-SNAPSHOT > --- > > Key: HIVE-8522 > URL: https://issues.apache.org/jira/browse/HIVE-8522 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8522.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8222) CBO Trunk Merge: Fix Check Style issues
[ https://issues.apache.org/jira/browse/HIVE-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177641#comment-14177641 ] Laljo John Pullokkaran commented on HIVE-8222: -- Lars, We may want to get this in sooner than later; otherwise we risk missing Hive 14. > CBO Trunk Merge: Fix Check Style issues > --- > > Key: HIVE-8222 > URL: https://issues.apache.org/jira/browse/HIVE-8222 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Lars Francke > Attachments: HIVE-8222.1.patch, HIVE-8222.2.patch, HIVE-8222.3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8522) CBO: Update Calcite Version to 0.9.2-incubating-SNAPSHOT
[ https://issues.apache.org/jira/browse/HIVE-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8522: - Summary: CBO: Update Calcite Version to 0.9.2-incubating-SNAPSHOT (was: Update Calcite Version to 0.9.2-incubating-SNAPSHOT) > CBO: Update Calcite Version to 0.9.2-incubating-SNAPSHOT > > > Key: HIVE-8522 > URL: https://issues.apache.org/jira/browse/HIVE-8522 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8522.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-8530) CBO: Preserve types of literals
Laljo John Pullokkaran created HIVE-8530: Summary: CBO: Preserve types of literals Key: HIVE-8530 URL: https://issues.apache.org/jira/browse/HIVE-8530 Project: Hive Issue Type: Sub-task Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8530) CBO: Preserve types of literals
[ https://issues.apache.org/jira/browse/HIVE-8530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8530: - Attachment: HIVE-8530.patch > CBO: Preserve types of literals > --- > > Key: HIVE-8530 > URL: https://issues.apache.org/jira/browse/HIVE-8530 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8530.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8530) CBO: Preserve types of literals
[ https://issues.apache.org/jira/browse/HIVE-8530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8530: - Status: Patch Available (was: Open) > CBO: Preserve types of literals > --- > > Key: HIVE-8530 > URL: https://issues.apache.org/jira/browse/HIVE-8530 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8530.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7985) With CBO enabled cross product is generated when a subquery is present
[
https://issues.apache.org/jira/browse/HIVE-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177727#comment-14177727
]
Laljo John Pullokkaran commented on HIVE-7985:
--
Hive Side patch for PPD through SemiJoinRel.
> With CBO enabled cross product is generated when a subquery is present
> --
>
> Key: HIVE-7985
> URL: https://issues.apache.org/jira/browse/HIVE-7985
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.14.0
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
>Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-7985.patch
>
>
> This is a regression introduced in the latest build of the CBO branch.
> Removing the subquery for item will remove the cross products
> Query
> {code}
> select i_item_id,sum(ss_ext_sales_price) total_sales from store_sales,
> date_dim, item where item.i_item_id in (select i.i_item_id from item i where
> i_color in ('purple','burlywood','indian')) and ss_item_sk = i_item_sk and
> ss_sold_date_sk = d_date_sk and d_year = 2001 and d_moy = 1 group by
> i_item_id;
> {code}
> {code}
> Warning: Map Join MAPJOIN[38][bigTable=?] in task 'Map 1' is a cross product
> Warning: Map Join MAPJOIN[39][bigTable=store_sales] in task 'Map 4' is a
> cross product
> OK
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 1 <- Map 3 (BROADCAST_EDGE)
> Map 4 <- Map 1 (BROADCAST_EDGE), Map 2 (BROADCAST_EDGE)
> Reducer 5 <- Map 4 (SIMPLE_EDGE)
> DagName: mmokhtar_20140904141313_9c253f7e-aad1-4ca4-9be1-ea45e3d34496:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: item
> filterExpr: (true and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: i_item_id is not null (type: boolean)
> Statistics: Num rows: 231000 Data size: 331931080 Basic
> stats: COMPLETE Column stats: NONE
> Map Join Operator
> condition map:
>Inner Join 0 to 1
> condition expressions:
> 0 {i_item_sk} {i_item_id}
> 1 {d_date_sk}
> keys:
> 0
> 1
> outputColumnNames: _col0, _col1, _col25
> input vertices:
> 1 Map 3
> Statistics: Num rows: 254100 Data size: 365124192 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: _col0 (type: int), _col1 (type: string),
> _col25 (type: int)
> outputColumnNames: _col0, _col1, _col25
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type:
> string), _col25 (type: int)
> Execution mode: vectorized
> Map 2
> Map Operator Tree:
> TableScan
> alias: i
> filterExpr: ((i_color) IN ('purple', 'burlywood', 'indian')
> and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: ((i_color) IN ('purple', 'burlywood',
> 'indian') and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: i_item_id (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Group By Operator
> keys: _col0 (type: string)
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operato
[jira] [Updated] (HIVE-7985) With CBO enabled cross product is generated when a subquery is present
[
https://issues.apache.org/jira/browse/HIVE-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-7985:
-
Attachment: HIVE-7985.patch
> With CBO enabled cross product is generated when a subquery is present
> --
>
> Key: HIVE-7985
> URL: https://issues.apache.org/jira/browse/HIVE-7985
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.14.0
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
>Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-7985.patch, HIVE-7985.patch
>
>
> This is a regression introduced in the latest build of the CBO branch.
> Removing the subquery for item will remove the cross products
> Query
> {code}
> select i_item_id,sum(ss_ext_sales_price) total_sales from store_sales,
> date_dim, item where item.i_item_id in (select i.i_item_id from item i where
> i_color in ('purple','burlywood','indian')) and ss_item_sk = i_item_sk and
> ss_sold_date_sk = d_date_sk and d_year = 2001 and d_moy = 1 group by
> i_item_id;
> {code}
> {code}
> Warning: Map Join MAPJOIN[38][bigTable=?] in task 'Map 1' is a cross product
> Warning: Map Join MAPJOIN[39][bigTable=store_sales] in task 'Map 4' is a
> cross product
> OK
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 1 <- Map 3 (BROADCAST_EDGE)
> Map 4 <- Map 1 (BROADCAST_EDGE), Map 2 (BROADCAST_EDGE)
> Reducer 5 <- Map 4 (SIMPLE_EDGE)
> DagName: mmokhtar_20140904141313_9c253f7e-aad1-4ca4-9be1-ea45e3d34496:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: item
> filterExpr: (true and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: i_item_id is not null (type: boolean)
> Statistics: Num rows: 231000 Data size: 331931080 Basic
> stats: COMPLETE Column stats: NONE
> Map Join Operator
> condition map:
>Inner Join 0 to 1
> condition expressions:
> 0 {i_item_sk} {i_item_id}
> 1 {d_date_sk}
> keys:
> 0
> 1
> outputColumnNames: _col0, _col1, _col25
> input vertices:
> 1 Map 3
> Statistics: Num rows: 254100 Data size: 365124192 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: _col0 (type: int), _col1 (type: string),
> _col25 (type: int)
> outputColumnNames: _col0, _col1, _col25
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type:
> string), _col25 (type: int)
> Execution mode: vectorized
> Map 2
> Map Operator Tree:
> TableScan
> alias: i
> filterExpr: ((i_color) IN ('purple', 'burlywood', 'indian')
> and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: ((i_color) IN ('purple', 'burlywood',
> 'indian') and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: i_item_id (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Group By Operator
> keys: _col0 (type: string)
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> key expressions: _col0 (
[jira] [Updated] (HIVE-7985) With CBO enabled cross product is generated when a subquery is present
[
https://issues.apache.org/jira/browse/HIVE-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-7985:
-
Attachment: (was: HIVE-7985.patch)
> With CBO enabled cross product is generated when a subquery is present
> --
>
> Key: HIVE-7985
> URL: https://issues.apache.org/jira/browse/HIVE-7985
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.14.0
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
>Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-7985.1.patch, HIVE-7985.patch
>
>
> This is a regression introduced in the latest build of the CBO branch.
> Removing the subquery for item will remove the cross products
> Query
> {code}
> select i_item_id,sum(ss_ext_sales_price) total_sales from store_sales,
> date_dim, item where item.i_item_id in (select i.i_item_id from item i where
> i_color in ('purple','burlywood','indian')) and ss_item_sk = i_item_sk and
> ss_sold_date_sk = d_date_sk and d_year = 2001 and d_moy = 1 group by
> i_item_id;
> {code}
> {code}
> Warning: Map Join MAPJOIN[38][bigTable=?] in task 'Map 1' is a cross product
> Warning: Map Join MAPJOIN[39][bigTable=store_sales] in task 'Map 4' is a
> cross product
> OK
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 1 <- Map 3 (BROADCAST_EDGE)
> Map 4 <- Map 1 (BROADCAST_EDGE), Map 2 (BROADCAST_EDGE)
> Reducer 5 <- Map 4 (SIMPLE_EDGE)
> DagName: mmokhtar_20140904141313_9c253f7e-aad1-4ca4-9be1-ea45e3d34496:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: item
> filterExpr: (true and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: i_item_id is not null (type: boolean)
> Statistics: Num rows: 231000 Data size: 331931080 Basic
> stats: COMPLETE Column stats: NONE
> Map Join Operator
> condition map:
>Inner Join 0 to 1
> condition expressions:
> 0 {i_item_sk} {i_item_id}
> 1 {d_date_sk}
> keys:
> 0
> 1
> outputColumnNames: _col0, _col1, _col25
> input vertices:
> 1 Map 3
> Statistics: Num rows: 254100 Data size: 365124192 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: _col0 (type: int), _col1 (type: string),
> _col25 (type: int)
> outputColumnNames: _col0, _col1, _col25
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type:
> string), _col25 (type: int)
> Execution mode: vectorized
> Map 2
> Map Operator Tree:
> TableScan
> alias: i
> filterExpr: ((i_color) IN ('purple', 'burlywood', 'indian')
> and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: ((i_color) IN ('purple', 'burlywood',
> 'indian') and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: i_item_id (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Group By Operator
> keys: _col0 (type: string)
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> key express
[jira] [Updated] (HIVE-7985) With CBO enabled cross product is generated when a subquery is present
[
https://issues.apache.org/jira/browse/HIVE-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-7985:
-
Attachment: HIVE-7985.1.patch
> With CBO enabled cross product is generated when a subquery is present
> --
>
> Key: HIVE-7985
> URL: https://issues.apache.org/jira/browse/HIVE-7985
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.14.0
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
>Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-7985.1.patch, HIVE-7985.patch
>
>
> This is a regression introduced in the latest build of the CBO branch.
> Removing the subquery for item will remove the cross products
> Query
> {code}
> select i_item_id,sum(ss_ext_sales_price) total_sales from store_sales,
> date_dim, item where item.i_item_id in (select i.i_item_id from item i where
> i_color in ('purple','burlywood','indian')) and ss_item_sk = i_item_sk and
> ss_sold_date_sk = d_date_sk and d_year = 2001 and d_moy = 1 group by
> i_item_id;
> {code}
> {code}
> Warning: Map Join MAPJOIN[38][bigTable=?] in task 'Map 1' is a cross product
> Warning: Map Join MAPJOIN[39][bigTable=store_sales] in task 'Map 4' is a
> cross product
> OK
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 1 <- Map 3 (BROADCAST_EDGE)
> Map 4 <- Map 1 (BROADCAST_EDGE), Map 2 (BROADCAST_EDGE)
> Reducer 5 <- Map 4 (SIMPLE_EDGE)
> DagName: mmokhtar_20140904141313_9c253f7e-aad1-4ca4-9be1-ea45e3d34496:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: item
> filterExpr: (true and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: i_item_id is not null (type: boolean)
> Statistics: Num rows: 231000 Data size: 331931080 Basic
> stats: COMPLETE Column stats: NONE
> Map Join Operator
> condition map:
>Inner Join 0 to 1
> condition expressions:
> 0 {i_item_sk} {i_item_id}
> 1 {d_date_sk}
> keys:
> 0
> 1
> outputColumnNames: _col0, _col1, _col25
> input vertices:
> 1 Map 3
> Statistics: Num rows: 254100 Data size: 365124192 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: _col0 (type: int), _col1 (type: string),
> _col25 (type: int)
> outputColumnNames: _col0, _col1, _col25
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type:
> string), _col25 (type: int)
> Execution mode: vectorized
> Map 2
> Map Operator Tree:
> TableScan
> alias: i
> filterExpr: ((i_color) IN ('purple', 'burlywood', 'indian')
> and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: ((i_color) IN ('purple', 'burlywood',
> 'indian') and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: i_item_id (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Group By Operator
> keys: _col0 (type: string)
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> key expressions: _co
[jira] [Commented] (HIVE-7985) With CBO enabled cross product is generated when a subquery is present
[
https://issues.apache.org/jira/browse/HIVE-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177740#comment-14177740
]
Laljo John Pullokkaran commented on HIVE-7985:
--
Need to wait for HIVE-8522
> With CBO enabled cross product is generated when a subquery is present
> --
>
> Key: HIVE-7985
> URL: https://issues.apache.org/jira/browse/HIVE-7985
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.14.0
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
>Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-7985.1.patch, HIVE-7985.patch
>
>
> This is a regression introduced in the latest build of the CBO branch.
> Removing the subquery for item will remove the cross products
> Query
> {code}
> select i_item_id,sum(ss_ext_sales_price) total_sales from store_sales,
> date_dim, item where item.i_item_id in (select i.i_item_id from item i where
> i_color in ('purple','burlywood','indian')) and ss_item_sk = i_item_sk and
> ss_sold_date_sk = d_date_sk and d_year = 2001 and d_moy = 1 group by
> i_item_id;
> {code}
> {code}
> Warning: Map Join MAPJOIN[38][bigTable=?] in task 'Map 1' is a cross product
> Warning: Map Join MAPJOIN[39][bigTable=store_sales] in task 'Map 4' is a
> cross product
> OK
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 1 <- Map 3 (BROADCAST_EDGE)
> Map 4 <- Map 1 (BROADCAST_EDGE), Map 2 (BROADCAST_EDGE)
> Reducer 5 <- Map 4 (SIMPLE_EDGE)
> DagName: mmokhtar_20140904141313_9c253f7e-aad1-4ca4-9be1-ea45e3d34496:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: item
> filterExpr: (true and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: i_item_id is not null (type: boolean)
> Statistics: Num rows: 231000 Data size: 331931080 Basic
> stats: COMPLETE Column stats: NONE
> Map Join Operator
> condition map:
>Inner Join 0 to 1
> condition expressions:
> 0 {i_item_sk} {i_item_id}
> 1 {d_date_sk}
> keys:
> 0
> 1
> outputColumnNames: _col0, _col1, _col25
> input vertices:
> 1 Map 3
> Statistics: Num rows: 254100 Data size: 365124192 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: _col0 (type: int), _col1 (type: string),
> _col25 (type: int)
> outputColumnNames: _col0, _col1, _col25
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type:
> string), _col25 (type: int)
> Execution mode: vectorized
> Map 2
> Map Operator Tree:
> TableScan
> alias: i
> filterExpr: ((i_color) IN ('purple', 'burlywood', 'indian')
> and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: ((i_color) IN ('purple', 'burlywood',
> 'indian') and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: i_item_id (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Group By Operator
> keys: _col0 (type: string)
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operat
[jira] [Resolved] (HIVE-8260) CBO : Query query has date_dim d1,date_dim d2 and date_dim d3 but the explain has d1, d1 and d1
[
https://issues.apache.org/jira/browse/HIVE-8260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran resolved HIVE-8260.
--
Resolution: Cannot Reproduce
> CBO : Query query has date_dim d1,date_dim d2 and date_dim d3 but the explain
> has d1, d1 and d1
>
>
> Key: HIVE-8260
> URL: https://issues.apache.org/jira/browse/HIVE-8260
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 0.14.0
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
> Fix For: 0.14.0
>
>
> For TPC-DS Q64 there is date_dim d1,date_dim d2 and date_dim d3 but the
> explain has d1, d1 and d1.
> This is a simplified version of query 64 that demonstrates the same issue :
> {code}
> select count(*)
> FROM store_sales
> JOIN store_returns ON store_sales.ss_item_sk =
> store_returns.sr_item_sk and store_sales.ss_ticket_number =
> store_returns.sr_ticket_number
> JOIN customer ON store_sales.ss_customer_sk = customer.c_customer_sk
> JOIN date_dim d1 ON store_sales.ss_sold_date_sk = d1.d_date_sk
> JOIN date_dim d2 ON customer.c_first_sales_date_sk = d2.d_date_sk
> JOIN date_dim d3 ON customer.c_first_shipto_date_sk = d3.d_date_sk
> JOIN store ON store_sales.ss_store_sk = store.s_store_sk
> JOIN customer_demographics cd1 ON store_sales.ss_cdemo_sk=
> cd1.cd_demo_sk
> JOIN customer_demographics cd2 ON customer.c_current_cdemo_sk =
> cd2.cd_demo_sk
> JOIN promotion ON store_sales.ss_promo_sk = promotion.p_promo_sk
> JOIN household_demographics hd1 ON store_sales.ss_hdemo_sk =
> hd1.hd_demo_sk
> JOIN household_demographics hd2 ON customer.c_current_hdemo_sk =
> hd2.hd_demo_sk
> JOIN customer_address ad1 ON store_sales.ss_addr_sk =
> ad1.ca_address_sk
> JOIN customer_address ad2 ON customer.c_current_addr_sk =
> ad2.ca_address_sk
> JOIN income_band ib1 ON hd1.hd_income_band_sk = ib1.ib_income_band_sk
> JOIN income_band ib2 ON hd2.hd_income_band_sk = ib2.ib_income_band_sk
> JOIN item ON store_sales.ss_item_sk = item.i_item_sk
> {code}
> The plan generated
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 13 <- Map 10 (BROADCAST_EDGE), Map 11 (BROADCAST_EDGE), Map 12
> (BROADCAST_EDGE), Map 15 (BROADCAST_EDGE), Map 16 (BROADCAST_EDGE), Map 18
> (BROADCAST_EDGE), Map 2 (BROADCAST_EDGE), Map 3 (BROADCAST_EDGE), Map 8
> (BROADCAST_EDGE)
> Map 16 <- Map 7 (BROADCAST_EDGE)
> Map 18 <- Map 1 (BROADCAST_EDGE), Map 17 (BROADCAST_EDGE), Map 4
> (BROADCAST_EDGE), Map 5 (BROADCAST_EDGE), Map 9 (BROADCAST_EDGE)
> Map 5 <- Map 6 (BROADCAST_EDGE)
> Reducer 14 <- Map 13 (SIMPLE_EDGE)
> DagName: mmokhtar_20140925180101_9c3b1d6b-61d3-44bc-a881-2beaf2ab143f:2
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: cd1
> filterExpr: cd_demo_sk is not null (type: boolean)
> Statistics: Num rows: 1920800 Data size: 718379200 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: cd_demo_sk is not null (type: boolean)
> Statistics: Num rows: 1920800 Data size: 7683200 Basic
> stats: COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: cd_demo_sk (type: int)
> outputColumnNames: _col0
> Statistics: Num rows: 1920800 Data size: 7683200 Basic
> stats: COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: int)
> sort order: +
> Map-reduce partition columns: _col0 (type: int)
> Statistics: Num rows: 1920800 Data size: 7683200
> Basic stats: COMPLETE Column stats: COMPLETE
> Execution mode: vectorized
> Map 10
> Map Operator Tree:
> TableScan
> alias: item
> filterExpr: i_item_sk is not null (type: boolean)
> Statistics: Num rows: 48000 Data size: 68732712 Basic
> stats: COMPLETE Column stats: COMPLETE
> Filter Operator
> predicate: i_item_sk is not null (type: boolean)
> Statistics: Num rows: 48000 Data size: 192000 Basic
> stats: COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: i_item_sk (type: int)
> outputColumnNames: _col0
> Statistics: Num r
[jira] [Updated] (HIVE-8232) CBO Trunk Merge: Address Review Comments
[ https://issues.apache.org/jira/browse/HIVE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8232: - Resolution: Fixed Status: Resolved (was: Patch Available) > CBO Trunk Merge: Address Review Comments > > > Key: HIVE-8232 > URL: https://issues.apache.org/jira/browse/HIVE-8232 > Project: Hive > Issue Type: Bug >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8232.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-7963) CBO Trunk Merge:Handle UDFS : Hash, round, if, datediff, date_add, date_sub, ascii, elt, coalesce, format_number, instr
[ https://issues.apache.org/jira/browse/HIVE-7963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran resolved HIVE-7963. -- Resolution: Cannot Reproduce > CBO Trunk Merge:Handle UDFS : Hash, round, if, datediff, date_add, date_sub, > ascii, elt, coalesce, format_number, instr > > > Key: HIVE-7963 > URL: https://issues.apache.org/jira/browse/HIVE-7963 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8125) CBO Trunk Merge: On Failure Fall Back to Non CBO
[ https://issues.apache.org/jira/browse/HIVE-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8125: - Resolution: Fixed Status: Resolved (was: Patch Available) > CBO Trunk Merge: On Failure Fall Back to Non CBO > > > Key: HIVE-8125 > URL: https://issues.apache.org/jira/browse/HIVE-8125 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8125.9.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-7666) Join selectivity calculation should use exponential back-off for conjunction predicates
[ https://issues.apache.org/jira/browse/HIVE-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran resolved HIVE-7666. -- Resolution: Fixed > Join selectivity calculation should use exponential back-off for conjunction > predicates > > > Key: HIVE-7666 > URL: https://issues.apache.org/jira/browse/HIVE-7666 > Project: Hive > Issue Type: Bug > Components: CBO >Affects Versions: 0.13.1 >Reporter: Mostafa Mokhtar >Assignee: Laljo John Pullokkaran > Labels: hive > Fix For: 0.14.0 > > > Assuming dependency for predicate (number of column joins & filters) will > almost always hurt us as implied correlations do actually exist. > Currently HiveRelMdSelectivity.computeInnerJoinSelectivity uses to log to > smoothen selectivity of conjunction predicates which results in un-optimal > plans. > The problem with log is that it still assumes dependency, For instance in > TPC-DS Q17 store_sales has 6 join predicates which explains why stor_sales is > in the wrong place in the plan. > Change the algorithm to use exponential back-off : > ndv(pe0) * ndv(pe1) ^(1/2) * ndv(pe2) ^(1/4) * ndv(pe3) ^(1/8) > Opposed to : > ndv(pex)*log(ndv(pe1))*log(ndv(pe2)) > For TPC-DS Q17 store_sales has 6 inner join predicates if we assume > selectivity of 0.7 for each join then join selectivity can end up being > 6.24285E-05 which is too low and eventually results in an un-optimal plan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8125) CBO Trunk Merge: On Failure Fall Back to Non CBO
[ https://issues.apache.org/jira/browse/HIVE-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178484#comment-14178484 ] Laljo John Pullokkaran commented on HIVE-8125: -- Hi Lefty, the fix was originally done in CBO branch which then got ported to Trunk. CBO branch is not relevant any more as all the CBO development now happens on trunk. hive.cbo.enable is set to false by default on trunk and branch 14. We may change this once all of the unit test issues are cleared. Optiq got renamed to Calcite this month (due to copyright issues); hence old CBO branch uses Optiq and trunk uses "Calcite". > CBO Trunk Merge: On Failure Fall Back to Non CBO > > > Key: HIVE-8125 > URL: https://issues.apache.org/jira/browse/HIVE-8125 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8125.9.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8522) CBO: Update Calcite Version to 0.9.2-incubating-SNAPSHOT
[ https://issues.apache.org/jira/browse/HIVE-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8522: - Status: Open (was: Patch Available) > CBO: Update Calcite Version to 0.9.2-incubating-SNAPSHOT > > > Key: HIVE-8522 > URL: https://issues.apache.org/jira/browse/HIVE-8522 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8522.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8522) CBO: Update Calcite Version to 0.9.2-incubating-SNAPSHOT
[ https://issues.apache.org/jira/browse/HIVE-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8522: - Status: Patch Available (was: Open) > CBO: Update Calcite Version to 0.9.2-incubating-SNAPSHOT > > > Key: HIVE-8522 > URL: https://issues.apache.org/jira/browse/HIVE-8522 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8522.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8522) CBO: Update Calcite Version to 0.9.2-incubating-SNAPSHOT
[ https://issues.apache.org/jira/browse/HIVE-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178687#comment-14178687 ] Laljo John Pullokkaran commented on HIVE-8522: -- 0.9.2-incubating-SNAPSHOT became available on Oct 20 8.40 PM. Retrying the patch. > CBO: Update Calcite Version to 0.9.2-incubating-SNAPSHOT > > > Key: HIVE-8522 > URL: https://issues.apache.org/jira/browse/HIVE-8522 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8522.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8433) CBO loses a column during AST conversion
[
https://issues.apache.org/jira/browse/HIVE-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178918#comment-14178918
]
Laljo John Pullokkaran commented on HIVE-8433:
--
Sergey, Could you add RB link?
> CBO loses a column during AST conversion
>
>
> Key: HIVE-8433
> URL: https://issues.apache.org/jira/browse/HIVE-8433
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
> Attachments: HIVE-8433.01.patch, HIVE-8433.02.patch, HIVE-8433.patch
>
>
> {noformat}
> SELECT
> CAST(value AS BINARY),
> value
> FROM src
> ORDER BY value
> LIMIT 100
> {noformat}
> returns only one column.
> Final CBO plan is
> {noformat}
> HiveSortRel(sort0=[$1], dir0=[ASC]): rowcount = 500.0, cumulative cost =
> {24858.432393688767 rows, 500.0 cpu, 0.0 io}, id = 44
> HiveProjectRel(value=[CAST($0):BINARY(2147483647) NOT NULL],
> value1=[$0]): rowcount = 500.0, cumulative cost = {0.0 rows, 0.0 cpu, 0.0
> io}, id = 42
> HiveProjectRel(value=[$1]): rowcount = 500.0, cumulative cost = {0.0
> rows, 0.0 cpu, 0.0 io}, id = 40
> HiveTableScanRel(table=[[default.src]]): rowcount = 500.0, cumulative
> cost = {0}, id = 0
> {noformat}
> but the resulting AST has only one column. Must be some bug in conversion,
> probably related to the name collision in the schema, judging by the alias of
> the column for the binary-cast value in the AST
> {noformat}
> TOK_QUERY
>TOK_FROM
> TOK_SUBQUERY
> TOK_QUERY
> TOK_FROM
>TOK_TABREF
> TOK_TABNAME
> default
> src
> src
> TOK_INSERT
>TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
>TOK_SELECT
> TOK_SELEXPR
> .
> TOK_TABLE_OR_COL
>src
> value
> value
> $hdt$_0
>TOK_INSERT
> TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
> TOK_SELECT
> TOK_SELEXPR
> TOK_FUNCTION
>TOK_BINARY
>.
> TOK_TABLE_OR_COL
> $hdt$_0
> value
> value
> TOK_ORDERBY
> TOK_TABSORTCOLNAMEASC
> TOK_TABLE_OR_COL
>value
> TOK_LIMIT
> 100
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8125) CBO Trunk Merge: On Failure Fall Back to Non CBO
[ https://issues.apache.org/jira/browse/HIVE-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8125: - Fix Version/s: cbo-branch Hi Lefty, it was fixed in CBO branch which then got ported to trunk, 14. I will update the fix version. Thanks > CBO Trunk Merge: On Failure Fall Back to Non CBO > > > Key: HIVE-8125 > URL: https://issues.apache.org/jira/browse/HIVE-8125 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Fix For: 0.14.0, cbo-branch > > Attachments: HIVE-8125.9.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8125) CBO Trunk Merge: On Failure Fall Back to Non CBO
[ https://issues.apache.org/jira/browse/HIVE-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8125: - Fix Version/s: 0.14.0 > CBO Trunk Merge: On Failure Fall Back to Non CBO > > > Key: HIVE-8125 > URL: https://issues.apache.org/jira/browse/HIVE-8125 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Fix For: 0.14.0, cbo-branch > > Attachments: HIVE-8125.9.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8125) CBO Trunk Merge: On Failure Fall Back to Non CBO
[ https://issues.apache.org/jira/browse/HIVE-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178927#comment-14178927 ] Laljo John Pullokkaran commented on HIVE-8125: -- BTW, Fix doesn't introduce any configuration flags. > CBO Trunk Merge: On Failure Fall Back to Non CBO > > > Key: HIVE-8125 > URL: https://issues.apache.org/jira/browse/HIVE-8125 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Fix For: 0.14.0, cbo-branch > > Attachments: HIVE-8125.9.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8530) CBO: Preserve types of literals
[ https://issues.apache.org/jira/browse/HIVE-8530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8530: - Attachment: HIVE-8530.1.patch > CBO: Preserve types of literals > --- > > Key: HIVE-8530 > URL: https://issues.apache.org/jira/browse/HIVE-8530 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8530.1.patch, HIVE-8530.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8530) CBO: Preserve types of literals
[ https://issues.apache.org/jira/browse/HIVE-8530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179166#comment-14179166 ] Laljo John Pullokkaran commented on HIVE-8530: -- Windowing, UDF can not handle Type qualifiers in literals. Updated patch. > CBO: Preserve types of literals > --- > > Key: HIVE-8530 > URL: https://issues.apache.org/jira/browse/HIVE-8530 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8530.1.patch, HIVE-8530.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8530) CBO: Preserve types of literals
[ https://issues.apache.org/jira/browse/HIVE-8530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8530: - Status: Patch Available (was: Open) > CBO: Preserve types of literals > --- > > Key: HIVE-8530 > URL: https://issues.apache.org/jira/browse/HIVE-8530 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8530.1.patch, HIVE-8530.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8433) CBO loses a column during AST conversion
[
https://issues.apache.org/jira/browse/HIVE-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179486#comment-14179486
]
Laljo John Pullokkaran commented on HIVE-8433:
--
Sergey,
Could you address the test failures?
I will look at the patch after that.
Thanks
> CBO loses a column during AST conversion
>
>
> Key: HIVE-8433
> URL: https://issues.apache.org/jira/browse/HIVE-8433
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
> Attachments: HIVE-8433.01.patch, HIVE-8433.02.patch,
> HIVE-8433.03.patch, HIVE-8433.patch
>
>
> {noformat}
> SELECT
> CAST(value AS BINARY),
> value
> FROM src
> ORDER BY value
> LIMIT 100
> {noformat}
> returns only one column.
> Final CBO plan is
> {noformat}
> HiveSortRel(sort0=[$1], dir0=[ASC]): rowcount = 500.0, cumulative cost =
> {24858.432393688767 rows, 500.0 cpu, 0.0 io}, id = 44
> HiveProjectRel(value=[CAST($0):BINARY(2147483647) NOT NULL],
> value1=[$0]): rowcount = 500.0, cumulative cost = {0.0 rows, 0.0 cpu, 0.0
> io}, id = 42
> HiveProjectRel(value=[$1]): rowcount = 500.0, cumulative cost = {0.0
> rows, 0.0 cpu, 0.0 io}, id = 40
> HiveTableScanRel(table=[[default.src]]): rowcount = 500.0, cumulative
> cost = {0}, id = 0
> {noformat}
> but the resulting AST has only one column. Must be some bug in conversion,
> probably related to the name collision in the schema, judging by the alias of
> the column for the binary-cast value in the AST
> {noformat}
> TOK_QUERY
>TOK_FROM
> TOK_SUBQUERY
> TOK_QUERY
> TOK_FROM
>TOK_TABREF
> TOK_TABNAME
> default
> src
> src
> TOK_INSERT
>TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
>TOK_SELECT
> TOK_SELEXPR
> .
> TOK_TABLE_OR_COL
>src
> value
> value
> $hdt$_0
>TOK_INSERT
> TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
> TOK_SELECT
> TOK_SELEXPR
> TOK_FUNCTION
>TOK_BINARY
>.
> TOK_TABLE_OR_COL
> $hdt$_0
> value
> value
> TOK_ORDERBY
> TOK_TABSORTCOLNAMEASC
> TOK_TABLE_OR_COL
>value
> TOK_LIMIT
> 100
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8530) CBO: Preserve types of literals
[ https://issues.apache.org/jira/browse/HIVE-8530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179555#comment-14179555 ] Laljo John Pullokkaran commented on HIVE-8530: -- Failed tests are not related to this change; this change would only be excessed by cbo_correctness. > CBO: Preserve types of literals > --- > > Key: HIVE-8530 > URL: https://issues.apache.org/jira/browse/HIVE-8530 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8530.1.patch, HIVE-8530.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7985) With CBO enabled cross product is generated when a subquery is present
[
https://issues.apache.org/jira/browse/HIVE-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-7985:
-
Status: Patch Available (was: Open)
> With CBO enabled cross product is generated when a subquery is present
> --
>
> Key: HIVE-7985
> URL: https://issues.apache.org/jira/browse/HIVE-7985
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.14.0
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
>Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-7985.1.patch, HIVE-7985.patch
>
>
> This is a regression introduced in the latest build of the CBO branch.
> Removing the subquery for item will remove the cross products
> Query
> {code}
> select i_item_id,sum(ss_ext_sales_price) total_sales from store_sales,
> date_dim, item where item.i_item_id in (select i.i_item_id from item i where
> i_color in ('purple','burlywood','indian')) and ss_item_sk = i_item_sk and
> ss_sold_date_sk = d_date_sk and d_year = 2001 and d_moy = 1 group by
> i_item_id;
> {code}
> {code}
> Warning: Map Join MAPJOIN[38][bigTable=?] in task 'Map 1' is a cross product
> Warning: Map Join MAPJOIN[39][bigTable=store_sales] in task 'Map 4' is a
> cross product
> OK
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 1 <- Map 3 (BROADCAST_EDGE)
> Map 4 <- Map 1 (BROADCAST_EDGE), Map 2 (BROADCAST_EDGE)
> Reducer 5 <- Map 4 (SIMPLE_EDGE)
> DagName: mmokhtar_20140904141313_9c253f7e-aad1-4ca4-9be1-ea45e3d34496:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: item
> filterExpr: (true and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: i_item_id is not null (type: boolean)
> Statistics: Num rows: 231000 Data size: 331931080 Basic
> stats: COMPLETE Column stats: NONE
> Map Join Operator
> condition map:
>Inner Join 0 to 1
> condition expressions:
> 0 {i_item_sk} {i_item_id}
> 1 {d_date_sk}
> keys:
> 0
> 1
> outputColumnNames: _col0, _col1, _col25
> input vertices:
> 1 Map 3
> Statistics: Num rows: 254100 Data size: 365124192 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: _col0 (type: int), _col1 (type: string),
> _col25 (type: int)
> outputColumnNames: _col0, _col1, _col25
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type:
> string), _col25 (type: int)
> Execution mode: vectorized
> Map 2
> Map Operator Tree:
> TableScan
> alias: i
> filterExpr: ((i_color) IN ('purple', 'burlywood', 'indian')
> and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: ((i_color) IN ('purple', 'burlywood',
> 'indian') and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: i_item_id (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Group By Operator
> keys: _col0 (type: string)
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> key expressio
[jira] [Commented] (HIVE-7985) With CBO enabled cross product is generated when a subquery is present
[
https://issues.apache.org/jira/browse/HIVE-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14180829#comment-14180829
]
Laljo John Pullokkaran commented on HIVE-7985:
--
These failures are not related to CBO.
> With CBO enabled cross product is generated when a subquery is present
> --
>
> Key: HIVE-7985
> URL: https://issues.apache.org/jira/browse/HIVE-7985
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 0.14.0
>Reporter: Mostafa Mokhtar
>Assignee: Laljo John Pullokkaran
>Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-7985.1.patch, HIVE-7985.2.patch, HIVE-7985.patch
>
>
> This is a regression introduced in the latest build of the CBO branch.
> Removing the subquery for item will remove the cross products
> Query
> {code}
> select i_item_id,sum(ss_ext_sales_price) total_sales from store_sales,
> date_dim, item where item.i_item_id in (select i.i_item_id from item i where
> i_color in ('purple','burlywood','indian')) and ss_item_sk = i_item_sk and
> ss_sold_date_sk = d_date_sk and d_year = 2001 and d_moy = 1 group by
> i_item_id;
> {code}
> {code}
> Warning: Map Join MAPJOIN[38][bigTable=?] in task 'Map 1' is a cross product
> Warning: Map Join MAPJOIN[39][bigTable=store_sales] in task 'Map 4' is a
> cross product
> OK
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> Edges:
> Map 1 <- Map 3 (BROADCAST_EDGE)
> Map 4 <- Map 1 (BROADCAST_EDGE), Map 2 (BROADCAST_EDGE)
> Reducer 5 <- Map 4 (SIMPLE_EDGE)
> DagName: mmokhtar_20140904141313_9c253f7e-aad1-4ca4-9be1-ea45e3d34496:1
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: item
> filterExpr: (true and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: i_item_id is not null (type: boolean)
> Statistics: Num rows: 231000 Data size: 331931080 Basic
> stats: COMPLETE Column stats: NONE
> Map Join Operator
> condition map:
>Inner Join 0 to 1
> condition expressions:
> 0 {i_item_sk} {i_item_id}
> 1 {d_date_sk}
> keys:
> 0
> 1
> outputColumnNames: _col0, _col1, _col25
> input vertices:
> 1 Map 3
> Statistics: Num rows: 254100 Data size: 365124192 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: _col0 (type: int), _col1 (type: string),
> _col25 (type: int)
> outputColumnNames: _col0, _col1, _col25
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 254100 Data size: 365124192
> Basic stats: COMPLETE Column stats: NONE
> value expressions: _col0 (type: int), _col1 (type:
> string), _col25 (type: int)
> Execution mode: vectorized
> Map 2
> Map Operator Tree:
> TableScan
> alias: i
> filterExpr: ((i_color) IN ('purple', 'burlywood', 'indian')
> and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 462000 Data size: 663862160 Basic
> stats: COMPLETE Column stats: NONE
> Filter Operator
> predicate: ((i_color) IN ('purple', 'burlywood',
> 'indian') and i_item_id is not null) (type: boolean)
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Select Operator
> expressions: i_item_id (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540 Basic
> stats: COMPLETE Column stats: NONE
> Group By Operator
> keys: _col0 (type: string)
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 115500 Data size: 165965540
> Basic stats: COMPLETE Column stats: NONE
>
[jira] [Commented] (HIVE-8575) CBO: decimal_udf is broken by recent changes (and other tests have type changes)
[
https://issues.apache.org/jira/browse/HIVE-8575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14181708#comment-14181708
]
Laljo John Pullokkaran commented on HIVE-8575:
--
I am guessing following additions in ASTBuilder below line 169 (case INTEGER)
should fix the problem.
val = literal.getValue3();
type = HiveParser.BigintLiteral;
break;
> CBO: decimal_udf is broken by recent changes (and other tests have type
> changes)
>
>
> Key: HIVE-8575
> URL: https://issues.apache.org/jira/browse/HIVE-8575
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
>
> Seems like unqualified integral values went from ints to bigints
> {noformat}
> EXPLAIN SELECT histogram_numeric(key, 3) FROM DECIMAL_UDF
> org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: Only an integer
> argument is accepted as parameter 2, but bigint was passed instead.
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFHistogramNumeric.getEvaluator(GenericUDAFHistogramNumeric.java:103)
> at
> org.apache.hadoop.hive.ql.udf.generic.AbstractGenericUDAFResolver.getEvaluator(AbstractGenericUDAFResolver.java:47)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:1143)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:3717)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapGroupByOperator(SemanticAnalyzer.java:4392)
> ... snip
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-8582) Outer Join Simplification is broken
Laljo John Pullokkaran created HIVE-8582: Summary: Outer Join Simplification is broken Key: HIVE-8582 URL: https://issues.apache.org/jira/browse/HIVE-8582 Project: Hive Issue Type: Sub-task Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8582) Outer Join Simplification is broken
[ https://issues.apache.org/jira/browse/HIVE-8582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8582: - Attachment: HIVE-8582.patch > Outer Join Simplification is broken > --- > > Key: HIVE-8582 > URL: https://issues.apache.org/jira/browse/HIVE-8582 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8582.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8582) Outer Join Simplification is broken
[ https://issues.apache.org/jira/browse/HIVE-8582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182034#comment-14182034 ] Laljo John Pullokkaran commented on HIVE-8582: -- CALCITE-444 has the corresponding calcite patch. > Outer Join Simplification is broken > --- > > Key: HIVE-8582 > URL: https://issues.apache.org/jira/browse/HIVE-8582 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8582.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8582) Outer Join Simplification is broken
[ https://issues.apache.org/jira/browse/HIVE-8582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8582: - Status: Patch Available (was: Open) > Outer Join Simplification is broken > --- > > Key: HIVE-8582 > URL: https://issues.apache.org/jira/browse/HIVE-8582 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8582.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8561) Expose Hive optiq operator tree to be able to support other sql on hadoop query engines
[ https://issues.apache.org/jira/browse/HIVE-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183151#comment-14183151 ] Laljo John Pullokkaran commented on HIVE-8561: -- Na Yang, If i understand correctly, goal of this patch is to use Hive for query parsing, resolving, cost based optimization and use Drill as the execution engine. If my guess is right this patch makes Hive's Optiq Op tree a public interface. The Hive's Optiq Op tree is not meant to be a public interface and it would go through many changes as we add more to CBO & support for more operators. Why can't Drill be plugged in as another execution engine just like MR, TEZ, Spark? > Expose Hive optiq operator tree to be able to support other sql on hadoop > query engines > --- > > Key: HIVE-8561 > URL: https://issues.apache.org/jira/browse/HIVE-8561 > Project: Hive > Issue Type: Task > Components: CBO >Affects Versions: 0.14.0 >Reporter: Na Yang >Assignee: Na Yang > Attachments: HIVE-8561.patch > > > Hive-0.14 added cost based optimization and optiq operator tree is created > for select queries. However, the optiq operator tree is not visible from > outside and hard to be used by other Sql on Hadoop query engine such as > apache Drill. To be able to allow drill to access the hive optiq operator > tree, we need to add a public api to return the hive optiq operator tree. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8433) CBO loses a column during AST conversion
[
https://issues.apache.org/jira/browse/HIVE-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183388#comment-14183388
]
Laljo John Pullokkaran commented on HIVE-8433:
--
Patch is being reviewed.
> CBO loses a column during AST conversion
>
>
> Key: HIVE-8433
> URL: https://issues.apache.org/jira/browse/HIVE-8433
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-8433.01.patch, HIVE-8433.02.patch,
> HIVE-8433.03.patch, HIVE-8433.04.patch, HIVE-8433.patch
>
>
> {noformat}
> SELECT
> CAST(value AS BINARY),
> value
> FROM src
> ORDER BY value
> LIMIT 100
> {noformat}
> returns only one column.
> Final CBO plan is
> {noformat}
> HiveSortRel(sort0=[$1], dir0=[ASC]): rowcount = 500.0, cumulative cost =
> {24858.432393688767 rows, 500.0 cpu, 0.0 io}, id = 44
> HiveProjectRel(value=[CAST($0):BINARY(2147483647) NOT NULL],
> value1=[$0]): rowcount = 500.0, cumulative cost = {0.0 rows, 0.0 cpu, 0.0
> io}, id = 42
> HiveProjectRel(value=[$1]): rowcount = 500.0, cumulative cost = {0.0
> rows, 0.0 cpu, 0.0 io}, id = 40
> HiveTableScanRel(table=[[default.src]]): rowcount = 500.0, cumulative
> cost = {0}, id = 0
> {noformat}
> but the resulting AST has only one column. Must be some bug in conversion,
> probably related to the name collision in the schema, judging by the alias of
> the column for the binary-cast value in the AST
> {noformat}
> TOK_QUERY
>TOK_FROM
> TOK_SUBQUERY
> TOK_QUERY
> TOK_FROM
>TOK_TABREF
> TOK_TABNAME
> default
> src
> src
> TOK_INSERT
>TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
>TOK_SELECT
> TOK_SELEXPR
> .
> TOK_TABLE_OR_COL
>src
> value
> value
> $hdt$_0
>TOK_INSERT
> TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
> TOK_SELECT
> TOK_SELEXPR
> TOK_FUNCTION
>TOK_BINARY
>.
> TOK_TABLE_OR_COL
> $hdt$_0
> value
> value
> TOK_ORDERBY
> TOK_TABSORTCOLNAMEASC
> TOK_TABLE_OR_COL
>value
> TOK_LIMIT
> 100
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8575) CBO: decimal_udf is broken by recent changes (and other tests have type changes)
[
https://issues.apache.org/jira/browse/HIVE-8575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183465#comment-14183465
]
Laljo John Pullokkaran commented on HIVE-8575:
--
+1
> CBO: decimal_udf is broken by recent changes (and other tests have type
> changes)
>
>
> Key: HIVE-8575
> URL: https://issues.apache.org/jira/browse/HIVE-8575
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
> Attachments: HIVE-8575.patch, HIVE-8575.patch
>
>
> Seems like unqualified integral values went from ints to bigints
> {noformat}
> EXPLAIN SELECT histogram_numeric(key, 3) FROM DECIMAL_UDF
> org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: Only an integer
> argument is accepted as parameter 2, but bigint was passed instead.
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFHistogramNumeric.getEvaluator(GenericUDAFHistogramNumeric.java:103)
> at
> org.apache.hadoop.hive.ql.udf.generic.AbstractGenericUDAFResolver.getEvaluator(AbstractGenericUDAFResolver.java:47)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:1143)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:3717)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapGroupByOperator(SemanticAnalyzer.java:4392)
> ... snip
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8021) CBO: support CTAS and insert ... select
[ https://issues.apache.org/jira/browse/HIVE-8021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183464#comment-14183464 ] Laljo John Pullokkaran commented on HIVE-8021: -- +1 > CBO: support CTAS and insert ... select > --- > > Key: HIVE-8021 > URL: https://issues.apache.org/jira/browse/HIVE-8021 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-8021.01.patch, HIVE-8021.01.patch, > HIVE-8021.02.patch, HIVE-8021.03.patch, HIVE-8021.04.patch, > HIVE-8021.05.patch, HIVE-8021.06.patch, HIVE-8021.06.patch, > HIVE-8021.07.patch, HIVE-8021.08.patch, HIVE-8021.patch, > HIVE-8021.preliminary.patch > > > Need to send only the select part to CBO for now -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8433) CBO loses a column during AST conversion
[
https://issues.apache.org/jira/browse/HIVE-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183472#comment-14183472
]
Laljo John Pullokkaran commented on HIVE-8433:
--
+1
> CBO loses a column during AST conversion
>
>
> Key: HIVE-8433
> URL: https://issues.apache.org/jira/browse/HIVE-8433
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-8433.01.patch, HIVE-8433.02.patch,
> HIVE-8433.03.patch, HIVE-8433.04.patch, HIVE-8433.patch
>
>
> {noformat}
> SELECT
> CAST(value AS BINARY),
> value
> FROM src
> ORDER BY value
> LIMIT 100
> {noformat}
> returns only one column.
> Final CBO plan is
> {noformat}
> HiveSortRel(sort0=[$1], dir0=[ASC]): rowcount = 500.0, cumulative cost =
> {24858.432393688767 rows, 500.0 cpu, 0.0 io}, id = 44
> HiveProjectRel(value=[CAST($0):BINARY(2147483647) NOT NULL],
> value1=[$0]): rowcount = 500.0, cumulative cost = {0.0 rows, 0.0 cpu, 0.0
> io}, id = 42
> HiveProjectRel(value=[$1]): rowcount = 500.0, cumulative cost = {0.0
> rows, 0.0 cpu, 0.0 io}, id = 40
> HiveTableScanRel(table=[[default.src]]): rowcount = 500.0, cumulative
> cost = {0}, id = 0
> {noformat}
> but the resulting AST has only one column. Must be some bug in conversion,
> probably related to the name collision in the schema, judging by the alias of
> the column for the binary-cast value in the AST
> {noformat}
> TOK_QUERY
>TOK_FROM
> TOK_SUBQUERY
> TOK_QUERY
> TOK_FROM
>TOK_TABREF
> TOK_TABNAME
> default
> src
> src
> TOK_INSERT
>TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
>TOK_SELECT
> TOK_SELEXPR
> .
> TOK_TABLE_OR_COL
>src
> value
> value
> $hdt$_0
>TOK_INSERT
> TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
> TOK_SELECT
> TOK_SELEXPR
> TOK_FUNCTION
>TOK_BINARY
>.
> TOK_TABLE_OR_COL
> $hdt$_0
> value
> value
> TOK_ORDERBY
> TOK_TABSORTCOLNAMEASC
> TOK_TABLE_OR_COL
>value
> TOK_LIMIT
> 100
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-8619) CBO causes some more type problems
[ https://issues.apache.org/jira/browse/HIVE-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran reassigned HIVE-8619: Assignee: Laljo John Pullokkaran > CBO causes some more type problems > -- > > Key: HIVE-8619 > URL: https://issues.apache.org/jira/browse/HIVE-8619 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Sergey Shelukhin >Assignee: Laljo John Pullokkaran >Priority: Critical > > On recent rerun, tests such as char_cast, decimal_2, literal_double, > vector_elt (incl. in Tez), udf4, vector_decimal_math_funcs have changes in > results it looks like something is broken with types again; ".0" suffix > disappeared in some cases (in char_cast it's probably the simplest to see), > number formats changed when casts are involved. > Also, windowing_expressions and couple more tests fail due to attempts to > cast Double to Int writables, might be the same issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8619) CBO causes some more type problems
[ https://issues.apache.org/jira/browse/HIVE-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8619: - Assignee: (was: Sergey Shelukhin) > CBO causes some more type problems > -- > > Key: HIVE-8619 > URL: https://issues.apache.org/jira/browse/HIVE-8619 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Sergey Shelukhin >Priority: Critical > > On recent rerun, tests such as char_cast, decimal_2, literal_double, > vector_elt (incl. in Tez), udf4, vector_decimal_math_funcs have changes in > results it looks like something is broken with types again; ".0" suffix > disappeared in some cases (in char_cast it's probably the simplest to see), > number formats changed when casts are involved. > Also, windowing_expressions and couple more tests fail due to attempts to > cast Double to Int writables, might be the same issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8619) CBO causes some more type problems
[ https://issues.apache.org/jira/browse/HIVE-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8619: - Attachment: HIVE-8619.patch > CBO causes some more type problems > -- > > Key: HIVE-8619 > URL: https://issues.apache.org/jira/browse/HIVE-8619 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Sergey Shelukhin >Assignee: Laljo John Pullokkaran >Priority: Critical > Attachments: HIVE-8619.patch > > > On recent rerun, tests such as char_cast, decimal_2, literal_double, > vector_elt (incl. in Tez), udf4, vector_decimal_math_funcs have changes in > results it looks like something is broken with types again; ".0" suffix > disappeared in some cases (in char_cast it's probably the simplest to see), > number formats changed when casts are involved. > Also, windowing_expressions and couple more tests fail due to attempts to > cast Double to Int writables, might be the same issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8619) CBO causes some more type problems
[ https://issues.apache.org/jira/browse/HIVE-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14187545#comment-14187545 ] Laljo John Pullokkaran commented on HIVE-8619: -- Sergey, could you get a QA run with this patch? All of the tests passed (including windowing_expressions). > CBO causes some more type problems > -- > > Key: HIVE-8619 > URL: https://issues.apache.org/jira/browse/HIVE-8619 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Sergey Shelukhin >Assignee: Laljo John Pullokkaran >Priority: Critical > Attachments: HIVE-8619.patch > > > On recent rerun, tests such as char_cast, decimal_2, literal_double, > vector_elt (incl. in Tez), udf4, vector_decimal_math_funcs have changes in > results it looks like something is broken with types again; ".0" suffix > disappeared in some cases (in char_cast it's probably the simplest to see), > number formats changed when casts are involved. > Also, windowing_expressions and couple more tests fail due to attempts to > cast Double to Int writables, might be the same issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8619) CBO causes some more type problems
[ https://issues.apache.org/jira/browse/HIVE-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14187604#comment-14187604 ] Laljo John Pullokkaran commented on HIVE-8619: -- Hive UDAFs seem to treat double's differently than other types. > CBO causes some more type problems > -- > > Key: HIVE-8619 > URL: https://issues.apache.org/jira/browse/HIVE-8619 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Sergey Shelukhin >Assignee: Laljo John Pullokkaran >Priority: Critical > Attachments: HIVE-8619.patch > > > On recent rerun, tests such as char_cast, decimal_2, literal_double, > vector_elt (incl. in Tez), udf4, vector_decimal_math_funcs have changes in > results it looks like something is broken with types again; ".0" suffix > disappeared in some cases (in char_cast it's probably the simplest to see), > number formats changed when casts are involved. > Also, windowing_expressions and couple more tests fail due to attempts to > cast Double to Int writables, might be the same issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-8652) Update committer list - Add Laljo John Pullokkaran
Laljo John Pullokkaran created HIVE-8652: Summary: Update committer list - Add Laljo John Pullokkaran Key: HIVE-8652 URL: https://issues.apache.org/jira/browse/HIVE-8652 Project: Hive Issue Type: Improvement Components: Documentation Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8582) CBO: Outer Join Simplification is broken
[ https://issues.apache.org/jira/browse/HIVE-8582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8582: - Summary: CBO: Outer Join Simplification is broken (was: Outer Join Simplification is broken) > CBO: Outer Join Simplification is broken > > > Key: HIVE-8582 > URL: https://issues.apache.org/jira/browse/HIVE-8582 > Project: Hive > Issue Type: Sub-task > Components: CBO >Affects Versions: 0.14.0 >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran >Priority: Critical > Fix For: 0.14.0 > > Attachments: HIVE-8582.patch, HIVE-8582.patch > > > CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-8653) CBO: Push Semi Join through, Project/Filter/Join
Laljo John Pullokkaran created HIVE-8653: Summary: CBO: Push Semi Join through, Project/Filter/Join Key: HIVE-8653 URL: https://issues.apache.org/jira/browse/HIVE-8653 Project: Hive Issue Type: Sub-task Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran Fix For: 0.14.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8653) CBO: Push Semi Join through, Project/Filter/Join
[ https://issues.apache.org/jira/browse/HIVE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8653: - Attachment: HIVE-8653.patch > CBO: Push Semi Join through, Project/Filter/Join > > > Key: HIVE-8653 > URL: https://issues.apache.org/jira/browse/HIVE-8653 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Fix For: 0.14.0 > > Attachments: HIVE-8653.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8653) CBO: Push Semi Join through, Project/Filter/Join
[ https://issues.apache.org/jira/browse/HIVE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8653: - Status: Patch Available (was: Open) > CBO: Push Semi Join through, Project/Filter/Join > > > Key: HIVE-8653 > URL: https://issues.apache.org/jira/browse/HIVE-8653 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Fix For: 0.14.0 > > Attachments: HIVE-8653.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8653) CBO: Push Semi Join through, Project/Filter/Join
[ https://issues.apache.org/jira/browse/HIVE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14188932#comment-14188932 ] Laljo John Pullokkaran commented on HIVE-8653: -- CALCITE-447 has corresponding CALCITE patch. > CBO: Push Semi Join through, Project/Filter/Join > > > Key: HIVE-8653 > URL: https://issues.apache.org/jira/browse/HIVE-8653 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Fix For: 0.14.0 > > Attachments: HIVE-8653.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8652) Update committer list - Add Laljo John Pullokkaran
[ https://issues.apache.org/jira/browse/HIVE-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8652: - Attachment: HIVE-8652.patch > Update committer list - Add Laljo John Pullokkaran > -- > > Key: HIVE-8652 > URL: https://issues.apache.org/jira/browse/HIVE-8652 > Project: Hive > Issue Type: Improvement > Components: Documentation >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8652.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8652) Update committer list - Add Laljo John Pullokkaran
[ https://issues.apache.org/jira/browse/HIVE-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8652: - Status: Patch Available (was: Open) > Update committer list - Add Laljo John Pullokkaran > -- > > Key: HIVE-8652 > URL: https://issues.apache.org/jira/browse/HIVE-8652 > Project: Hive > Issue Type: Improvement > Components: Documentation >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-8652.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8653) CBO: Push Semi Join through, Project/Filter/Join
[ https://issues.apache.org/jira/browse/HIVE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8653: - Status: Open (was: Patch Available) > CBO: Push Semi Join through, Project/Filter/Join > > > Key: HIVE-8653 > URL: https://issues.apache.org/jira/browse/HIVE-8653 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Fix For: 0.14.0 > > Attachments: HIVE-8653.patch > > > CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8653) CBO: Push Semi Join through, Project/Filter/Join
[ https://issues.apache.org/jira/browse/HIVE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8653: - Description: CLEAR LIBRARY CACHE > CBO: Push Semi Join through, Project/Filter/Join > > > Key: HIVE-8653 > URL: https://issues.apache.org/jira/browse/HIVE-8653 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Fix For: 0.14.0 > > Attachments: HIVE-8653.patch > > > CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8653) CBO: Push Semi Join through, Project/Filter/Join
[ https://issues.apache.org/jira/browse/HIVE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189025#comment-14189025 ] Laljo John Pullokkaran commented on HIVE-8653: -- Resubmitting patch with clear cache description (so that mvn cache gets cleared). > CBO: Push Semi Join through, Project/Filter/Join > > > Key: HIVE-8653 > URL: https://issues.apache.org/jira/browse/HIVE-8653 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Fix For: 0.14.0 > > Attachments: HIVE-8653.patch > > > CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8653) CBO: Push Semi Join through, Project/Filter/Join
[ https://issues.apache.org/jira/browse/HIVE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8653: - Attachment: HIVE-8653.1.patch > CBO: Push Semi Join through, Project/Filter/Join > > > Key: HIVE-8653 > URL: https://issues.apache.org/jira/browse/HIVE-8653 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Fix For: 0.14.0 > > Attachments: HIVE-8653.1.patch, HIVE-8653.patch > > > CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8653) CBO: Push Semi Join through, Project/Filter/Join
[ https://issues.apache.org/jira/browse/HIVE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-8653: - Status: Patch Available (was: Open) > CBO: Push Semi Join through, Project/Filter/Join > > > Key: HIVE-8653 > URL: https://issues.apache.org/jira/browse/HIVE-8653 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Fix For: 0.14.0 > > Attachments: HIVE-8653.1.patch, HIVE-8653.patch > > > CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7852) [CBO] Handle unary operators
[ https://issues.apache.org/jira/browse/HIVE-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107449#comment-14107449 ] Laljo John Pullokkaran commented on HIVE-7852: -- +1 > [CBO] Handle unary operators > > > Key: HIVE-7852 > URL: https://issues.apache.org/jira/browse/HIVE-7852 > Project: Hive > Issue Type: Sub-task > Components: CBO >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: h-7852.patch > > > Currently, query like select c1 from t1 where c2 = -6; throws exception > because cbo path confuses unary -ve with binary -ve -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HIVE-7721) CBO: support case statement translation to optiq
[
https://issues.apache.org/jira/browse/HIVE-7721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran reassigned HIVE-7721:
Assignee: Laljo John Pullokkaran
> CBO: support case statement translation to optiq
>
>
> Key: HIVE-7721
> URL: https://issues.apache.org/jira/browse/HIVE-7721
> Project: Hive
> Issue Type: Sub-task
>Reporter: Harish Butani
>Assignee: Laljo John Pullokkaran
>
> Following query:
> {code}
> explain select case when key > '104' then null else key end as key from src
> {code}
> fails with:
> {quote}
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.RuntimeException: Unsupported Expression
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11808)
>
> aused by: java.lang.RuntimeException: Unsupported Expression
> at
> org.apache.hadoop.hive.ql.optimizer.optiq.translator.RexNodeConverter.convert(RexNodeConverter.java:91)
> at
> org.apache.hadoop.hive.ql.optimizer.optiq.translator.RexNodeConverter.convert(RexNodeConverter.java:124)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7720) CBO: rank translation to Optiq RelNode tree failing
[
https://issues.apache.org/jira/browse/HIVE-7720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-7720:
-
Assignee: Laljo John Pullokkaran
> CBO: rank translation to Optiq RelNode tree failing
> ---
>
> Key: HIVE-7720
> URL: https://issues.apache.org/jira/browse/HIVE-7720
> Project: Hive
> Issue Type: Sub-task
> Components: Query Processor
>Reporter: Harish Butani
>Assignee: Laljo John Pullokkaran
>
> Following query:
> {code}
> explain select p_name
> from (select p_mfgr, p_name, p_size, rank() over(partition by p_mfgr order by
> p_size) as r from part) a
> where r <= 2;
> {code}
> fails with
> {quote}
> org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: One or more
> arguments are expected.
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRank.getEvaluator(GenericUDAFRank.java:61)
> at
> org.apache.hadoop.hive.ql.udf.generic.AbstractGenericUDAFResolver.getEvaluator(AbstractGenericUDAFResolver.java:47)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:1110)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:3506)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getHiveAggInfo(SemanticAnalyzer.java:12496)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genWindowingProj(SemanticAnalyzer.java:12858)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7721) CBO: support case statement translation to optiq
[
https://issues.apache.org/jira/browse/HIVE-7721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109405#comment-14109405
]
Laljo John Pullokkaran commented on HIVE-7721:
--
Fixed by HIVE-7841
> CBO: support case statement translation to optiq
>
>
> Key: HIVE-7721
> URL: https://issues.apache.org/jira/browse/HIVE-7721
> Project: Hive
> Issue Type: Sub-task
>Reporter: Harish Butani
>Assignee: Laljo John Pullokkaran
>
> Following query:
> {code}
> explain select case when key > '104' then null else key end as key from src
> {code}
> fails with:
> {quote}
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.RuntimeException: Unsupported Expression
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11808)
>
> aused by: java.lang.RuntimeException: Unsupported Expression
> at
> org.apache.hadoop.hive.ql.optimizer.optiq.translator.RexNodeConverter.convert(RexNodeConverter.java:91)
> at
> org.apache.hadoop.hive.ql.optimizer.optiq.translator.RexNodeConverter.convert(RexNodeConverter.java:124)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HIVE-7721) CBO: support case statement translation to optiq
[
https://issues.apache.org/jira/browse/HIVE-7721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran resolved HIVE-7721.
--
Resolution: Fixed
> CBO: support case statement translation to optiq
>
>
> Key: HIVE-7721
> URL: https://issues.apache.org/jira/browse/HIVE-7721
> Project: Hive
> Issue Type: Sub-task
>Reporter: Harish Butani
>Assignee: Laljo John Pullokkaran
>
> Following query:
> {code}
> explain select case when key > '104' then null else key end as key from src
> {code}
> fails with:
> {quote}
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.RuntimeException: Unsupported Expression
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11808)
>
> aused by: java.lang.RuntimeException: Unsupported Expression
> at
> org.apache.hadoop.hive.ql.optimizer.optiq.translator.RexNodeConverter.convert(RexNodeConverter.java:91)
> at
> org.apache.hadoop.hive.ql.optimizer.optiq.translator.RexNodeConverter.convert(RexNodeConverter.java:124)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7720) CBO: rank translation to Optiq RelNode tree failing
[
https://issues.apache.org/jira/browse/HIVE-7720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109411#comment-14109411
]
Laljo John Pullokkaran commented on HIVE-7720:
--
Support all Windowing UDAF.
row_number, rank, dense_rank, percent_rank, cume_dist, first_value, last_value,
lead, lag.
> CBO: rank translation to Optiq RelNode tree failing
> ---
>
> Key: HIVE-7720
> URL: https://issues.apache.org/jira/browse/HIVE-7720
> Project: Hive
> Issue Type: Sub-task
> Components: Query Processor
>Reporter: Harish Butani
>Assignee: Laljo John Pullokkaran
>
> Following query:
> {code}
> explain select p_name
> from (select p_mfgr, p_name, p_size, rank() over(partition by p_mfgr order by
> p_size) as r from part) a
> where r <= 2;
> {code}
> fails with
> {quote}
> org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: One or more
> arguments are expected.
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRank.getEvaluator(GenericUDAFRank.java:61)
> at
> org.apache.hadoop.hive.ql.udf.generic.AbstractGenericUDAFResolver.getEvaluator(AbstractGenericUDAFResolver.java:47)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:1110)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:3506)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getHiveAggInfo(SemanticAnalyzer.java:12496)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genWindowingProj(SemanticAnalyzer.java:12858)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7720) CBO: rank translation to Optiq RelNode tree failing
[
https://issues.apache.org/jira/browse/HIVE-7720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-7720:
-
Attachment: HIVE-7720.patch
> CBO: rank translation to Optiq RelNode tree failing
> ---
>
> Key: HIVE-7720
> URL: https://issues.apache.org/jira/browse/HIVE-7720
> Project: Hive
> Issue Type: Sub-task
> Components: Query Processor
>Reporter: Harish Butani
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-7720.patch
>
>
> Following query:
> {code}
> explain select p_name
> from (select p_mfgr, p_name, p_size, rank() over(partition by p_mfgr order by
> p_size) as r from part) a
> where r <= 2;
> {code}
> fails with
> {quote}
> org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: One or more
> arguments are expected.
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRank.getEvaluator(GenericUDAFRank.java:61)
> at
> org.apache.hadoop.hive.ql.udf.generic.AbstractGenericUDAFResolver.getEvaluator(AbstractGenericUDAFResolver.java:47)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:1110)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:3506)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getHiveAggInfo(SemanticAnalyzer.java:12496)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genWindowingProj(SemanticAnalyzer.java:12858)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7720) CBO: rank translation to Optiq RelNode tree failing
[
https://issues.apache.org/jira/browse/HIVE-7720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-7720:
-
Status: Patch Available (was: Open)
> CBO: rank translation to Optiq RelNode tree failing
> ---
>
> Key: HIVE-7720
> URL: https://issues.apache.org/jira/browse/HIVE-7720
> Project: Hive
> Issue Type: Sub-task
> Components: Query Processor
>Reporter: Harish Butani
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-7720.patch
>
>
> Following query:
> {code}
> explain select p_name
> from (select p_mfgr, p_name, p_size, rank() over(partition by p_mfgr order by
> p_size) as r from part) a
> where r <= 2;
> {code}
> fails with
> {quote}
> org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: One or more
> arguments are expected.
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRank.getEvaluator(GenericUDAFRank.java:61)
> at
> org.apache.hadoop.hive.ql.udf.generic.AbstractGenericUDAFResolver.getEvaluator(AbstractGenericUDAFResolver.java:47)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:1110)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:3506)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getHiveAggInfo(SemanticAnalyzer.java:12496)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genWindowingProj(SemanticAnalyzer.java:12858)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7830) CBO: Some UDF(case, lead, lag..) doesn't get translated correctly
[ https://issues.apache.org/jira/browse/HIVE-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14110965#comment-14110965 ] Laljo John Pullokkaran commented on HIVE-7830: -- Fixed by HIVE-7720 > CBO: Some UDF(case, lead, lag..) doesn't get translated correctly > - > > Key: HIVE-7830 > URL: https://issues.apache.org/jira/browse/HIVE-7830 > Project: Hive > Issue Type: Sub-task >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HIVE-7830) CBO: Some UDF(case, lead, lag..) doesn't get translated correctly
[ https://issues.apache.org/jira/browse/HIVE-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran resolved HIVE-7830. -- Resolution: Fixed > CBO: Some UDF(case, lead, lag..) doesn't get translated correctly > - > > Key: HIVE-7830 > URL: https://issues.apache.org/jira/browse/HIVE-7830 > Project: Hive > Issue Type: Sub-task >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7392) Support Columns Stats for Partition Columns
[ https://issues.apache.org/jira/browse/HIVE-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14111040#comment-14111040 ] Laljo John Pullokkaran commented on HIVE-7392: -- +1 > Support Columns Stats for Partition Columns > --- > > Key: HIVE-7392 > URL: https://issues.apache.org/jira/browse/HIVE-7392 > Project: Hive > Issue Type: Sub-task >Reporter: Laljo John Pullokkaran >Assignee: Ashutosh Chauhan > Attachments: h-7392.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7720) CBO: rank translation to Optiq RelNode tree failing
[
https://issues.apache.org/jira/browse/HIVE-7720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laljo John Pullokkaran updated HIVE-7720:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> CBO: rank translation to Optiq RelNode tree failing
> ---
>
> Key: HIVE-7720
> URL: https://issues.apache.org/jira/browse/HIVE-7720
> Project: Hive
> Issue Type: Sub-task
> Components: Query Processor
>Reporter: Harish Butani
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-7720.patch
>
>
> Following query:
> {code}
> explain select p_name
> from (select p_mfgr, p_name, p_size, rank() over(partition by p_mfgr order by
> p_size) as r from part) a
> where r <= 2;
> {code}
> fails with
> {quote}
> org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: One or more
> arguments are expected.
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRank.getEvaluator(GenericUDAFRank.java:61)
> at
> org.apache.hadoop.hive.ql.udf.generic.AbstractGenericUDAFResolver.getEvaluator(AbstractGenericUDAFResolver.java:47)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:1110)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:3506)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getHiveAggInfo(SemanticAnalyzer.java:12496)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genWindowingProj(SemanticAnalyzer.java:12858)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7814) CBO: Predicate Push Down Enhancements
[ https://issues.apache.org/jira/browse/HIVE-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14112607#comment-14112607 ] Laljo John Pullokkaran commented on HIVE-7814: -- Depends on OPTIQ-389 > CBO: Predicate Push Down Enhancements > - > > Key: HIVE-7814 > URL: https://issues.apache.org/jira/browse/HIVE-7814 > Project: Hive > Issue Type: Sub-task >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7814) CBO: Predicate Push Down Enhancements
[ https://issues.apache.org/jira/browse/HIVE-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14112611#comment-14112611 ] Laljo John Pullokkaran commented on HIVE-7814: -- Enhancements: 1. PPD to keep pushing predicates as long as possible 2. Push Predicate through Project/Select, Union All 3. Merge Filters 4. Remove trivial Project/Select operator 5. Simplify Join (outer to inner) > CBO: Predicate Push Down Enhancements > - > > Key: HIVE-7814 > URL: https://issues.apache.org/jira/browse/HIVE-7814 > Project: Hive > Issue Type: Sub-task >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7814) CBO: Predicate Push Down Enhancements
[ https://issues.apache.org/jira/browse/HIVE-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-7814: - Status: Patch Available (was: Open) > CBO: Predicate Push Down Enhancements > - > > Key: HIVE-7814 > URL: https://issues.apache.org/jira/browse/HIVE-7814 > Project: Hive > Issue Type: Sub-task >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-7814.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7814) CBO: Predicate Push Down Enhancements
[ https://issues.apache.org/jira/browse/HIVE-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-7814: - Attachment: HIVE-7814.patch > CBO: Predicate Push Down Enhancements > - > > Key: HIVE-7814 > URL: https://issues.apache.org/jira/browse/HIVE-7814 > Project: Hive > Issue Type: Sub-task >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-7814.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7908) CBO: Handle Windowing functions part of expressions
Laljo John Pullokkaran created HIVE-7908: Summary: CBO: Handle Windowing functions part of expressions Key: HIVE-7908 URL: https://issues.apache.org/jira/browse/HIVE-7908 Project: Hive Issue Type: Bug Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7908) CBO: Handle Windowing functions part of expressions
[ https://issues.apache.org/jira/browse/HIVE-7908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-7908: - Attachment: HIVE-7908.patch > CBO: Handle Windowing functions part of expressions > --- > > Key: HIVE-7908 > URL: https://issues.apache.org/jira/browse/HIVE-7908 > Project: Hive > Issue Type: Bug >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-7908.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7908) CBO: Handle Windowing functions part of expressions
[ https://issues.apache.org/jira/browse/HIVE-7908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laljo John Pullokkaran updated HIVE-7908: - Status: Patch Available (was: Open) > CBO: Handle Windowing functions part of expressions > --- > > Key: HIVE-7908 > URL: https://issues.apache.org/jira/browse/HIVE-7908 > Project: Hive > Issue Type: Bug >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > Attachments: HIVE-7908.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7460) CBO:Add rule to push Filter Through Project
[ https://issues.apache.org/jira/browse/HIVE-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14116093#comment-14116093 ] Laljo John Pullokkaran commented on HIVE-7460: -- Fixed by HIVE-7814 > CBO:Add rule to push Filter Through Project > --- > > Key: HIVE-7460 > URL: https://issues.apache.org/jira/browse/HIVE-7460 > Project: Hive > Issue Type: Sub-task >Reporter: Laljo John Pullokkaran >Assignee: Laljo John Pullokkaran > -- This message was sent by Atlassian JIRA (v6.2#6252)
