[jira] [Updated] (HIVE-9695) Redundant filter operator in reducer Vertex when CBO is disabled
[ https://issues.apache.org/jira/browse/HIVE-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9695: -- Affects Version/s: (was: 1.1.0) (was: 1.0.0) (was: 0.14.0) 2.0.0 > Redundant filter operator in reducer Vertex when CBO is disabled > > > Key: HIVE-9695 > URL: https://issues.apache.org/jira/browse/HIVE-9695 > Project: Hive > Issue Type: Improvement > Components: Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Mostafa Mokhtar >Assignee: Jesus Camacho Rodriguez > Fix For: 2.0.0 > > Attachments: HIVE-9695.01.patch, HIVE-9695.01.patch, HIVE-9695.patch > > > There is a redundant filter operator in reducer Vertex when CBO is disabled. > Query > {code} > select > ss_item_sk, ss_ticket_number, ss_store_sk > from > store_sales a, store_returns b, store > where > a.ss_item_sk = b.sr_item_sk > and a.ss_ticket_number = b.sr_ticket_number > and ss_sold_date_sk between 2450816 and 2451500 > and sr_returned_date_sk between 2450816 and 2451500 > and s_store_sk = ss_store_sk; > {code} > Plan snippet > {code} > Statistics: Num rows: 57439344 Data size: 1838059008 Basic stats: COMPLETE > Column stats: COMPLETE > Filter Operator > predicate: (_col1 = _col27) and (_col8 = _col34)) and > _col22 BETWEEN 2450816 AND 2451500) and _col45 BETWEEN 2450816 AND 2451500) > and (_col49 = _col6)) (type: boolean) > {code} > Full plan with CBO disabled > {code} > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 depends on stages: Stage-1 > STAGE PLANS: > Stage: Stage-1 > Tez > Edges: > Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (BROADCAST_EDGE), Map 4 > (SIMPLE_EDGE) > DagName: mmokhtar_20150214182626_ad6820c7-b667-4652-ab25-cb60deed1a6d:13 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: b > filterExpr: ((sr_item_sk is not null and sr_ticket_number > is not null) and sr_returned_date_sk BETWEEN 2450816 AND 2451500) (type: > boolean) > Statistics: Num rows: 2370038095 Data size: 170506118656 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: (sr_item_sk is not null and sr_ticket_number > is not null) (type: boolean) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: sr_item_sk (type: int), > sr_ticket_number (type: int) > sort order: ++ > Map-reduce partition columns: sr_item_sk (type: int), > sr_ticket_number (type: int) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > value expressions: sr_returned_date_sk (type: int) > Execution mode: vectorized > Map 3 > Map Operator Tree: > TableScan > alias: store > filterExpr: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 3256276 Basic stats: > COMPLETE Column stats: COMPLETE > Filter Operator > predicate: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: s_store_sk (type: int) > sort order: + > Map-reduce partition columns: s_store_sk (type: int) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Execution mode: vectorized > Map 4 > Map Operator Tree: > TableScan > alias: a > filterExpr: (((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) and ss_sold_date_sk BETWEEN 2450816 > AND 2451500) (type: boolean) > Statistics: Num rows: 28878719387 Data size: 2405805439460 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: ((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) (type: boolean) > Statistics: Num rows: 8405840828 Data size: 110101408700 > Bas
[jira] [Updated] (HIVE-9695) Redundant filter operator in reducer Vertex when CBO is disabled
[ https://issues.apache.org/jira/browse/HIVE-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-9695: --- Affects Version/s: (was: 2.0.0) 0.14.0 1.0.0 1.1.0 > Redundant filter operator in reducer Vertex when CBO is disabled > > > Key: HIVE-9695 > URL: https://issues.apache.org/jira/browse/HIVE-9695 > Project: Hive > Issue Type: Improvement > Components: Logical Optimizer >Affects Versions: 0.14.0, 1.0.0, 1.1.0 >Reporter: Mostafa Mokhtar >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-9695.01.patch, HIVE-9695.01.patch, HIVE-9695.patch > > > There is a redundant filter operator in reducer Vertex when CBO is disabled. > Query > {code} > select > ss_item_sk, ss_ticket_number, ss_store_sk > from > store_sales a, store_returns b, store > where > a.ss_item_sk = b.sr_item_sk > and a.ss_ticket_number = b.sr_ticket_number > and ss_sold_date_sk between 2450816 and 2451500 > and sr_returned_date_sk between 2450816 and 2451500 > and s_store_sk = ss_store_sk; > {code} > Plan snippet > {code} > Statistics: Num rows: 57439344 Data size: 1838059008 Basic stats: COMPLETE > Column stats: COMPLETE > Filter Operator > predicate: (_col1 = _col27) and (_col8 = _col34)) and > _col22 BETWEEN 2450816 AND 2451500) and _col45 BETWEEN 2450816 AND 2451500) > and (_col49 = _col6)) (type: boolean) > {code} > Full plan with CBO disabled > {code} > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 depends on stages: Stage-1 > STAGE PLANS: > Stage: Stage-1 > Tez > Edges: > Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (BROADCAST_EDGE), Map 4 > (SIMPLE_EDGE) > DagName: mmokhtar_20150214182626_ad6820c7-b667-4652-ab25-cb60deed1a6d:13 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: b > filterExpr: ((sr_item_sk is not null and sr_ticket_number > is not null) and sr_returned_date_sk BETWEEN 2450816 AND 2451500) (type: > boolean) > Statistics: Num rows: 2370038095 Data size: 170506118656 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: (sr_item_sk is not null and sr_ticket_number > is not null) (type: boolean) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: sr_item_sk (type: int), > sr_ticket_number (type: int) > sort order: ++ > Map-reduce partition columns: sr_item_sk (type: int), > sr_ticket_number (type: int) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > value expressions: sr_returned_date_sk (type: int) > Execution mode: vectorized > Map 3 > Map Operator Tree: > TableScan > alias: store > filterExpr: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 3256276 Basic stats: > COMPLETE Column stats: COMPLETE > Filter Operator > predicate: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: s_store_sk (type: int) > sort order: + > Map-reduce partition columns: s_store_sk (type: int) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Execution mode: vectorized > Map 4 > Map Operator Tree: > TableScan > alias: a > filterExpr: (((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) and ss_sold_date_sk BETWEEN 2450816 > AND 2451500) (type: boolean) > Statistics: Num rows: 28878719387 Data size: 2405805439460 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: ((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) (type: boolean) > Statistics: Num rows: 8405840828 Data size: 110101408700 > Basic stats: COMPLETE Column stats: COMPLETE >
[jira] [Updated] (HIVE-9695) Redundant filter operator in reducer Vertex when CBO is disabled
[ https://issues.apache.org/jira/browse/HIVE-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-9695: --- Component/s: (was: Physical Optimizer) Logical Optimizer > Redundant filter operator in reducer Vertex when CBO is disabled > > > Key: HIVE-9695 > URL: https://issues.apache.org/jira/browse/HIVE-9695 > Project: Hive > Issue Type: Improvement > Components: Logical Optimizer >Affects Versions: 2.0.0 >Reporter: Mostafa Mokhtar >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-9695.01.patch, HIVE-9695.01.patch, HIVE-9695.patch > > > There is a redundant filter operator in reducer Vertex when CBO is disabled. > Query > {code} > select > ss_item_sk, ss_ticket_number, ss_store_sk > from > store_sales a, store_returns b, store > where > a.ss_item_sk = b.sr_item_sk > and a.ss_ticket_number = b.sr_ticket_number > and ss_sold_date_sk between 2450816 and 2451500 > and sr_returned_date_sk between 2450816 and 2451500 > and s_store_sk = ss_store_sk; > {code} > Plan snippet > {code} > Statistics: Num rows: 57439344 Data size: 1838059008 Basic stats: COMPLETE > Column stats: COMPLETE > Filter Operator > predicate: (_col1 = _col27) and (_col8 = _col34)) and > _col22 BETWEEN 2450816 AND 2451500) and _col45 BETWEEN 2450816 AND 2451500) > and (_col49 = _col6)) (type: boolean) > {code} > Full plan with CBO disabled > {code} > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 depends on stages: Stage-1 > STAGE PLANS: > Stage: Stage-1 > Tez > Edges: > Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (BROADCAST_EDGE), Map 4 > (SIMPLE_EDGE) > DagName: mmokhtar_20150214182626_ad6820c7-b667-4652-ab25-cb60deed1a6d:13 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: b > filterExpr: ((sr_item_sk is not null and sr_ticket_number > is not null) and sr_returned_date_sk BETWEEN 2450816 AND 2451500) (type: > boolean) > Statistics: Num rows: 2370038095 Data size: 170506118656 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: (sr_item_sk is not null and sr_ticket_number > is not null) (type: boolean) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: sr_item_sk (type: int), > sr_ticket_number (type: int) > sort order: ++ > Map-reduce partition columns: sr_item_sk (type: int), > sr_ticket_number (type: int) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > value expressions: sr_returned_date_sk (type: int) > Execution mode: vectorized > Map 3 > Map Operator Tree: > TableScan > alias: store > filterExpr: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 3256276 Basic stats: > COMPLETE Column stats: COMPLETE > Filter Operator > predicate: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: s_store_sk (type: int) > sort order: + > Map-reduce partition columns: s_store_sk (type: int) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Execution mode: vectorized > Map 4 > Map Operator Tree: > TableScan > alias: a > filterExpr: (((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) and ss_sold_date_sk BETWEEN 2450816 > AND 2451500) (type: boolean) > Statistics: Num rows: 28878719387 Data size: 2405805439460 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: ((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) (type: boolean) > Statistics: Num rows: 8405840828 Data size: 110101408700 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > ke
[jira] [Updated] (HIVE-9695) Redundant filter operator in reducer Vertex when CBO is disabled
[ https://issues.apache.org/jira/browse/HIVE-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9695: -- Attachment: HIVE-9695.01.patch > Redundant filter operator in reducer Vertex when CBO is disabled > > > Key: HIVE-9695 > URL: https://issues.apache.org/jira/browse/HIVE-9695 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.0.0 >Reporter: Mostafa Mokhtar >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-9695.01.patch, HIVE-9695.01.patch, HIVE-9695.patch > > > There is a redundant filter operator in reducer Vertex when CBO is disabled. > Query > {code} > select > ss_item_sk, ss_ticket_number, ss_store_sk > from > store_sales a, store_returns b, store > where > a.ss_item_sk = b.sr_item_sk > and a.ss_ticket_number = b.sr_ticket_number > and ss_sold_date_sk between 2450816 and 2451500 > and sr_returned_date_sk between 2450816 and 2451500 > and s_store_sk = ss_store_sk; > {code} > Plan snippet > {code} > Statistics: Num rows: 57439344 Data size: 1838059008 Basic stats: COMPLETE > Column stats: COMPLETE > Filter Operator > predicate: (_col1 = _col27) and (_col8 = _col34)) and > _col22 BETWEEN 2450816 AND 2451500) and _col45 BETWEEN 2450816 AND 2451500) > and (_col49 = _col6)) (type: boolean) > {code} > Full plan with CBO disabled > {code} > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 depends on stages: Stage-1 > STAGE PLANS: > Stage: Stage-1 > Tez > Edges: > Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (BROADCAST_EDGE), Map 4 > (SIMPLE_EDGE) > DagName: mmokhtar_20150214182626_ad6820c7-b667-4652-ab25-cb60deed1a6d:13 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: b > filterExpr: ((sr_item_sk is not null and sr_ticket_number > is not null) and sr_returned_date_sk BETWEEN 2450816 AND 2451500) (type: > boolean) > Statistics: Num rows: 2370038095 Data size: 170506118656 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: (sr_item_sk is not null and sr_ticket_number > is not null) (type: boolean) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: sr_item_sk (type: int), > sr_ticket_number (type: int) > sort order: ++ > Map-reduce partition columns: sr_item_sk (type: int), > sr_ticket_number (type: int) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > value expressions: sr_returned_date_sk (type: int) > Execution mode: vectorized > Map 3 > Map Operator Tree: > TableScan > alias: store > filterExpr: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 3256276 Basic stats: > COMPLETE Column stats: COMPLETE > Filter Operator > predicate: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: s_store_sk (type: int) > sort order: + > Map-reduce partition columns: s_store_sk (type: int) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Execution mode: vectorized > Map 4 > Map Operator Tree: > TableScan > alias: a > filterExpr: (((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) and ss_sold_date_sk BETWEEN 2450816 > AND 2451500) (type: boolean) > Statistics: Num rows: 28878719387 Data size: 2405805439460 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: ((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) (type: boolean) > Statistics: Num rows: 8405840828 Data size: 110101408700 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: ss_item_sk (type:
[jira] [Updated] (HIVE-9695) Redundant filter operator in reducer Vertex when CBO is disabled
[ https://issues.apache.org/jira/browse/HIVE-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9695: -- Attachment: HIVE-9695.01.patch > Redundant filter operator in reducer Vertex when CBO is disabled > > > Key: HIVE-9695 > URL: https://issues.apache.org/jira/browse/HIVE-9695 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.0.0 >Reporter: Mostafa Mokhtar >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-9695.01.patch, HIVE-9695.patch > > > There is a redundant filter operator in reducer Vertex when CBO is disabled. > Query > {code} > select > ss_item_sk, ss_ticket_number, ss_store_sk > from > store_sales a, store_returns b, store > where > a.ss_item_sk = b.sr_item_sk > and a.ss_ticket_number = b.sr_ticket_number > and ss_sold_date_sk between 2450816 and 2451500 > and sr_returned_date_sk between 2450816 and 2451500 > and s_store_sk = ss_store_sk; > {code} > Plan snippet > {code} > Statistics: Num rows: 57439344 Data size: 1838059008 Basic stats: COMPLETE > Column stats: COMPLETE > Filter Operator > predicate: (_col1 = _col27) and (_col8 = _col34)) and > _col22 BETWEEN 2450816 AND 2451500) and _col45 BETWEEN 2450816 AND 2451500) > and (_col49 = _col6)) (type: boolean) > {code} > Full plan with CBO disabled > {code} > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 depends on stages: Stage-1 > STAGE PLANS: > Stage: Stage-1 > Tez > Edges: > Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (BROADCAST_EDGE), Map 4 > (SIMPLE_EDGE) > DagName: mmokhtar_20150214182626_ad6820c7-b667-4652-ab25-cb60deed1a6d:13 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: b > filterExpr: ((sr_item_sk is not null and sr_ticket_number > is not null) and sr_returned_date_sk BETWEEN 2450816 AND 2451500) (type: > boolean) > Statistics: Num rows: 2370038095 Data size: 170506118656 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: (sr_item_sk is not null and sr_ticket_number > is not null) (type: boolean) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: sr_item_sk (type: int), > sr_ticket_number (type: int) > sort order: ++ > Map-reduce partition columns: sr_item_sk (type: int), > sr_ticket_number (type: int) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > value expressions: sr_returned_date_sk (type: int) > Execution mode: vectorized > Map 3 > Map Operator Tree: > TableScan > alias: store > filterExpr: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 3256276 Basic stats: > COMPLETE Column stats: COMPLETE > Filter Operator > predicate: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: s_store_sk (type: int) > sort order: + > Map-reduce partition columns: s_store_sk (type: int) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Execution mode: vectorized > Map 4 > Map Operator Tree: > TableScan > alias: a > filterExpr: (((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) and ss_sold_date_sk BETWEEN 2450816 > AND 2451500) (type: boolean) > Statistics: Num rows: 28878719387 Data size: 2405805439460 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: ((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) (type: boolean) > Statistics: Num rows: 8405840828 Data size: 110101408700 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: ss_item_sk (type: int), > ss_ticket_
[jira] [Updated] (HIVE-9695) Redundant filter operator in reducer Vertex when CBO is disabled
[ https://issues.apache.org/jira/browse/HIVE-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9695: -- Issue Type: Improvement (was: Bug) > Redundant filter operator in reducer Vertex when CBO is disabled > > > Key: HIVE-9695 > URL: https://issues.apache.org/jira/browse/HIVE-9695 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.0.0 >Reporter: Mostafa Mokhtar >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-9695.patch > > > There is a redundant filter operator in reducer Vertex when CBO is disabled. > Query > {code} > select > ss_item_sk, ss_ticket_number, ss_store_sk > from > store_sales a, store_returns b, store > where > a.ss_item_sk = b.sr_item_sk > and a.ss_ticket_number = b.sr_ticket_number > and ss_sold_date_sk between 2450816 and 2451500 > and sr_returned_date_sk between 2450816 and 2451500 > and s_store_sk = ss_store_sk; > {code} > Plan snippet > {code} > Statistics: Num rows: 57439344 Data size: 1838059008 Basic stats: COMPLETE > Column stats: COMPLETE > Filter Operator > predicate: (_col1 = _col27) and (_col8 = _col34)) and > _col22 BETWEEN 2450816 AND 2451500) and _col45 BETWEEN 2450816 AND 2451500) > and (_col49 = _col6)) (type: boolean) > {code} > Full plan with CBO disabled > {code} > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 depends on stages: Stage-1 > STAGE PLANS: > Stage: Stage-1 > Tez > Edges: > Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (BROADCAST_EDGE), Map 4 > (SIMPLE_EDGE) > DagName: mmokhtar_20150214182626_ad6820c7-b667-4652-ab25-cb60deed1a6d:13 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: b > filterExpr: ((sr_item_sk is not null and sr_ticket_number > is not null) and sr_returned_date_sk BETWEEN 2450816 AND 2451500) (type: > boolean) > Statistics: Num rows: 2370038095 Data size: 170506118656 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: (sr_item_sk is not null and sr_ticket_number > is not null) (type: boolean) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: sr_item_sk (type: int), > sr_ticket_number (type: int) > sort order: ++ > Map-reduce partition columns: sr_item_sk (type: int), > sr_ticket_number (type: int) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > value expressions: sr_returned_date_sk (type: int) > Execution mode: vectorized > Map 3 > Map Operator Tree: > TableScan > alias: store > filterExpr: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 3256276 Basic stats: > COMPLETE Column stats: COMPLETE > Filter Operator > predicate: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: s_store_sk (type: int) > sort order: + > Map-reduce partition columns: s_store_sk (type: int) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Execution mode: vectorized > Map 4 > Map Operator Tree: > TableScan > alias: a > filterExpr: (((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) and ss_sold_date_sk BETWEEN 2450816 > AND 2451500) (type: boolean) > Statistics: Num rows: 28878719387 Data size: 2405805439460 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: ((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) (type: boolean) > Statistics: Num rows: 8405840828 Data size: 110101408700 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: ss_item_sk (type: int), > ss_ticket_number (type: i
[jira] [Updated] (HIVE-9695) Redundant filter operator in reducer Vertex when CBO is disabled
[ https://issues.apache.org/jira/browse/HIVE-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9695: -- Affects Version/s: (was: 0.14.0) 2.0.0 > Redundant filter operator in reducer Vertex when CBO is disabled > > > Key: HIVE-9695 > URL: https://issues.apache.org/jira/browse/HIVE-9695 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Affects Versions: 2.0.0 >Reporter: Mostafa Mokhtar >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-9695.patch > > > There is a redundant filter operator in reducer Vertex when CBO is disabled. > Query > {code} > select > ss_item_sk, ss_ticket_number, ss_store_sk > from > store_sales a, store_returns b, store > where > a.ss_item_sk = b.sr_item_sk > and a.ss_ticket_number = b.sr_ticket_number > and ss_sold_date_sk between 2450816 and 2451500 > and sr_returned_date_sk between 2450816 and 2451500 > and s_store_sk = ss_store_sk; > {code} > Plan snippet > {code} > Statistics: Num rows: 57439344 Data size: 1838059008 Basic stats: COMPLETE > Column stats: COMPLETE > Filter Operator > predicate: (_col1 = _col27) and (_col8 = _col34)) and > _col22 BETWEEN 2450816 AND 2451500) and _col45 BETWEEN 2450816 AND 2451500) > and (_col49 = _col6)) (type: boolean) > {code} > Full plan with CBO disabled > {code} > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 depends on stages: Stage-1 > STAGE PLANS: > Stage: Stage-1 > Tez > Edges: > Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (BROADCAST_EDGE), Map 4 > (SIMPLE_EDGE) > DagName: mmokhtar_20150214182626_ad6820c7-b667-4652-ab25-cb60deed1a6d:13 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: b > filterExpr: ((sr_item_sk is not null and sr_ticket_number > is not null) and sr_returned_date_sk BETWEEN 2450816 AND 2451500) (type: > boolean) > Statistics: Num rows: 2370038095 Data size: 170506118656 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: (sr_item_sk is not null and sr_ticket_number > is not null) (type: boolean) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: sr_item_sk (type: int), > sr_ticket_number (type: int) > sort order: ++ > Map-reduce partition columns: sr_item_sk (type: int), > sr_ticket_number (type: int) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > value expressions: sr_returned_date_sk (type: int) > Execution mode: vectorized > Map 3 > Map Operator Tree: > TableScan > alias: store > filterExpr: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 3256276 Basic stats: > COMPLETE Column stats: COMPLETE > Filter Operator > predicate: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: s_store_sk (type: int) > sort order: + > Map-reduce partition columns: s_store_sk (type: int) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Execution mode: vectorized > Map 4 > Map Operator Tree: > TableScan > alias: a > filterExpr: (((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) and ss_sold_date_sk BETWEEN 2450816 > AND 2451500) (type: boolean) > Statistics: Num rows: 28878719387 Data size: 2405805439460 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: ((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) (type: boolean) > Statistics: Num rows: 8405840828 Data size: 110101408700 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: ss_item_sk (type: int), > ss_
[jira] [Updated] (HIVE-9695) Redundant filter operator in reducer Vertex when CBO is disabled
[ https://issues.apache.org/jira/browse/HIVE-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-9695: -- Attachment: HIVE-9695.patch > Redundant filter operator in reducer Vertex when CBO is disabled > > > Key: HIVE-9695 > URL: https://issues.apache.org/jira/browse/HIVE-9695 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Affects Versions: 0.14.0 >Reporter: Mostafa Mokhtar >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-9695.patch > > > There is a redundant filter operator in reducer Vertex when CBO is disabled. > Query > {code} > select > ss_item_sk, ss_ticket_number, ss_store_sk > from > store_sales a, store_returns b, store > where > a.ss_item_sk = b.sr_item_sk > and a.ss_ticket_number = b.sr_ticket_number > and ss_sold_date_sk between 2450816 and 2451500 > and sr_returned_date_sk between 2450816 and 2451500 > and s_store_sk = ss_store_sk; > {code} > Plan snippet > {code} > Statistics: Num rows: 57439344 Data size: 1838059008 Basic stats: COMPLETE > Column stats: COMPLETE > Filter Operator > predicate: (_col1 = _col27) and (_col8 = _col34)) and > _col22 BETWEEN 2450816 AND 2451500) and _col45 BETWEEN 2450816 AND 2451500) > and (_col49 = _col6)) (type: boolean) > {code} > Full plan with CBO disabled > {code} > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 depends on stages: Stage-1 > STAGE PLANS: > Stage: Stage-1 > Tez > Edges: > Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (BROADCAST_EDGE), Map 4 > (SIMPLE_EDGE) > DagName: mmokhtar_20150214182626_ad6820c7-b667-4652-ab25-cb60deed1a6d:13 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: b > filterExpr: ((sr_item_sk is not null and sr_ticket_number > is not null) and sr_returned_date_sk BETWEEN 2450816 AND 2451500) (type: > boolean) > Statistics: Num rows: 2370038095 Data size: 170506118656 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: (sr_item_sk is not null and sr_ticket_number > is not null) (type: boolean) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: sr_item_sk (type: int), > sr_ticket_number (type: int) > sort order: ++ > Map-reduce partition columns: sr_item_sk (type: int), > sr_ticket_number (type: int) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > value expressions: sr_returned_date_sk (type: int) > Execution mode: vectorized > Map 3 > Map Operator Tree: > TableScan > alias: store > filterExpr: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 3256276 Basic stats: > COMPLETE Column stats: COMPLETE > Filter Operator > predicate: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: s_store_sk (type: int) > sort order: + > Map-reduce partition columns: s_store_sk (type: int) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Execution mode: vectorized > Map 4 > Map Operator Tree: > TableScan > alias: a > filterExpr: (((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) and ss_sold_date_sk BETWEEN 2450816 > AND 2451500) (type: boolean) > Statistics: Num rows: 28878719387 Data size: 2405805439460 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: ((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) (type: boolean) > Statistics: Num rows: 8405840828 Data size: 110101408700 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: ss_item_sk (type: int), > ss_ticket_number (type: int) >
[jira] [Updated] (HIVE-9695) Redundant filter operator in reducer Vertex when CBO is disabled
[ https://issues.apache.org/jira/browse/HIVE-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushanth Sowmyan updated HIVE-9695: --- Fix Version/s: (was: 1.2.0) > Redundant filter operator in reducer Vertex when CBO is disabled > > > Key: HIVE-9695 > URL: https://issues.apache.org/jira/browse/HIVE-9695 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Affects Versions: 0.14.0 >Reporter: Mostafa Mokhtar >Assignee: Laljo John Pullokkaran > > There is a redundant filter operator in reducer Vertex when CBO is disabled. > Query > {code} > select > ss_item_sk, ss_ticket_number, ss_store_sk > from > store_sales a, store_returns b, store > where > a.ss_item_sk = b.sr_item_sk > and a.ss_ticket_number = b.sr_ticket_number > and ss_sold_date_sk between 2450816 and 2451500 > and sr_returned_date_sk between 2450816 and 2451500 > and s_store_sk = ss_store_sk; > {code} > Plan snippet > {code} > Statistics: Num rows: 57439344 Data size: 1838059008 Basic stats: COMPLETE > Column stats: COMPLETE > Filter Operator > predicate: (_col1 = _col27) and (_col8 = _col34)) and > _col22 BETWEEN 2450816 AND 2451500) and _col45 BETWEEN 2450816 AND 2451500) > and (_col49 = _col6)) (type: boolean) > {code} > Full plan with CBO disabled > {code} > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 depends on stages: Stage-1 > STAGE PLANS: > Stage: Stage-1 > Tez > Edges: > Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (BROADCAST_EDGE), Map 4 > (SIMPLE_EDGE) > DagName: mmokhtar_20150214182626_ad6820c7-b667-4652-ab25-cb60deed1a6d:13 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: b > filterExpr: ((sr_item_sk is not null and sr_ticket_number > is not null) and sr_returned_date_sk BETWEEN 2450816 AND 2451500) (type: > boolean) > Statistics: Num rows: 2370038095 Data size: 170506118656 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: (sr_item_sk is not null and sr_ticket_number > is not null) (type: boolean) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: sr_item_sk (type: int), > sr_ticket_number (type: int) > sort order: ++ > Map-reduce partition columns: sr_item_sk (type: int), > sr_ticket_number (type: int) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > value expressions: sr_returned_date_sk (type: int) > Execution mode: vectorized > Map 3 > Map Operator Tree: > TableScan > alias: store > filterExpr: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 3256276 Basic stats: > COMPLETE Column stats: COMPLETE > Filter Operator > predicate: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: s_store_sk (type: int) > sort order: + > Map-reduce partition columns: s_store_sk (type: int) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Execution mode: vectorized > Map 4 > Map Operator Tree: > TableScan > alias: a > filterExpr: (((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) and ss_sold_date_sk BETWEEN 2450816 > AND 2451500) (type: boolean) > Statistics: Num rows: 28878719387 Data size: 2405805439460 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: ((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) (type: boolean) > Statistics: Num rows: 8405840828 Data size: 110101408700 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: ss_item_sk (type: int), > ss_ticket_number (type: int) > sort order: ++ >
[jira] [Updated] (HIVE-9695) Redundant filter operator in reducer Vertex when CBO is disabled
[ https://issues.apache.org/jira/browse/HIVE-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-9695: - Assignee: Laljo John Pullokkaran (was: Gunther Hagleitner) > Redundant filter operator in reducer Vertex when CBO is disabled > > > Key: HIVE-9695 > URL: https://issues.apache.org/jira/browse/HIVE-9695 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Affects Versions: 0.14.0 >Reporter: Mostafa Mokhtar >Assignee: Laljo John Pullokkaran > Fix For: 1.2.0 > > > There is a redundant filter operator in reducer Vertex when CBO is disabled. > Query > {code} > select > ss_item_sk, ss_ticket_number, ss_store_sk > from > store_sales a, store_returns b, store > where > a.ss_item_sk = b.sr_item_sk > and a.ss_ticket_number = b.sr_ticket_number > and ss_sold_date_sk between 2450816 and 2451500 > and sr_returned_date_sk between 2450816 and 2451500 > and s_store_sk = ss_store_sk; > {code} > Plan snippet > {code} > Statistics: Num rows: 57439344 Data size: 1838059008 Basic stats: COMPLETE > Column stats: COMPLETE > Filter Operator > predicate: (_col1 = _col27) and (_col8 = _col34)) and > _col22 BETWEEN 2450816 AND 2451500) and _col45 BETWEEN 2450816 AND 2451500) > and (_col49 = _col6)) (type: boolean) > {code} > Full plan with CBO disabled > {code} > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 depends on stages: Stage-1 > STAGE PLANS: > Stage: Stage-1 > Tez > Edges: > Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (BROADCAST_EDGE), Map 4 > (SIMPLE_EDGE) > DagName: mmokhtar_20150214182626_ad6820c7-b667-4652-ab25-cb60deed1a6d:13 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: b > filterExpr: ((sr_item_sk is not null and sr_ticket_number > is not null) and sr_returned_date_sk BETWEEN 2450816 AND 2451500) (type: > boolean) > Statistics: Num rows: 2370038095 Data size: 170506118656 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: (sr_item_sk is not null and sr_ticket_number > is not null) (type: boolean) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: sr_item_sk (type: int), > sr_ticket_number (type: int) > sort order: ++ > Map-reduce partition columns: sr_item_sk (type: int), > sr_ticket_number (type: int) > Statistics: Num rows: 706893063 Data size: 6498502768 > Basic stats: COMPLETE Column stats: COMPLETE > value expressions: sr_returned_date_sk (type: int) > Execution mode: vectorized > Map 3 > Map Operator Tree: > TableScan > alias: store > filterExpr: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 3256276 Basic stats: > COMPLETE Column stats: COMPLETE > Filter Operator > predicate: s_store_sk is not null (type: boolean) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: s_store_sk (type: int) > sort order: + > Map-reduce partition columns: s_store_sk (type: int) > Statistics: Num rows: 1704 Data size: 6816 Basic stats: > COMPLETE Column stats: COMPLETE > Execution mode: vectorized > Map 4 > Map Operator Tree: > TableScan > alias: a > filterExpr: (((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) and ss_sold_date_sk BETWEEN 2450816 > AND 2451500) (type: boolean) > Statistics: Num rows: 28878719387 Data size: 2405805439460 > Basic stats: COMPLETE Column stats: COMPLETE > Filter Operator > predicate: ((ss_item_sk is not null and ss_ticket_number > is not null) and ss_store_sk is not null) (type: boolean) > Statistics: Num rows: 8405840828 Data size: 110101408700 > Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: ss_item_sk (type: int), > ss_ticket_number (type: int)