Qifan Chen has posted comments on this change. (
http://gerrit.cloudera.org:8080/18023 )
Change subject: IMPALA-7942: Add query hints for cardinalities and selectivities
......................................................................
Patch Set 7:
Tried a slightly different sql parser and did not see any shift-reduce or
reduce-reduce conflicts.
With change, it is possible to handle selectivity hints for predicate in
general. Examples shown below.
Query: explain select * from functional.alltypestiny where int_col = 1 /*
+SELECTIVITY(0.5) */
+------------------------------------------------------------+
| Explain String |
+------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=4.01MB Threads=2 |
| Per-Host Resource Estimates: Memory=20MB |
| Codegen disabled by planner |
| |
| PLAN-ROOT SINK |
| | |
| 00:SCAN HDFS [functional.alltypestiny] |
| HDFS partitions=4/4 files=4 size=460B |
| predicates: int_col = 1 |
| row-size=89B cardinality=4 |
+------------------------------------------------------------+
Fetched 10 row(s) in 0.09s
[14:54:50 qchen@qifan-10229:
IMPALA-7942_Add_query_hints_for_cardinalities_and_selectivities_wangsheng] !vi
vi dml
[14:55:02 qchen@qifan-10229:
IMPALA-7942_Add_query_hints_for_cardinalities_and_selectivities_wangsheng] sql
dml
Starting Impala Shell with no authentication using Python 2.7.16
Warning: live_progress only applies to interactive shell sessions, and is being
skipped for now.
Opened TCP connection to localhost:21050
Connected to localhost:21050
Server version: impalad version 4.1.0-SNAPSHOT DEBUG (build
ad29ce70b3f4cfab4105be52ef664129060fa04e)
Query: explain select * from functional.alltypestiny where
(int_col = 1) /* +SELECTIVITY(0.5) */
+------------------------------------------------------------+
| Explain String |
+------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=4.01MB Threads=2 |
| Per-Host Resource Estimates: Memory=20MB |
| Codegen disabled by planner |
| |
| PLAN-ROOT SINK |
| | |
| 00:SCAN HDFS [functional.alltypestiny] |
| HDFS partitions=4/4 files=4 size=460B |
| predicates: (int_col = 1) |
| row-size=89B cardinality=4 |
+------------------------------------------------------------+
Fetched 10 row(s) in 0.01s
[14:55:04 qchen@qifan-10229:
IMPALA-7942_Add_query_hints_for_cardinalities_and_selectivities_wangsheng] !vi
vi dml
[14:55:18 qchen@qifan-10229:
IMPALA-7942_Add_query_hints_for_cardinalities_and_selectivities_wangsheng] sql
dml
Starting Impala Shell with no authentication using Python 2.7.16
Warning: live_progress only applies to interactive shell sessions, and is being
skipped for now.
Opened TCP connection to localhost:21050
Connected to localhost:21050
Server version: impalad version 4.1.0-SNAPSHOT DEBUG (build
ad29ce70b3f4cfab4105be52ef664129060fa04e)
Query: explain select * from functional.alltypestiny where
(int_col = 1 or bigint_col < 10000) /* +SELECTIVITY(0.5) */
+------------------------------------------------------------+
| Explain String |
+------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=4.01MB Threads=2 |
| Per-Host Resource Estimates: Memory=20MB |
| Codegen disabled by planner |
| |
| PLAN-ROOT SINK |
| | |
| 00:SCAN HDFS [functional.alltypestiny] |
| HDFS partitions=4/4 files=4 size=460B |
| predicates: (int_col = 1 OR bigint_col < 10000) |
| row-size=89B cardinality=1 |
+------------------------------------------------------------+
Fetched 10 row(s) in 0.01s
[14:55:19 qchen@qifan-10229:
IMPALA-7942_Add_query_hints_for_cardinalities_and_selectivities_wangsheng] !vi
vi dml
[14:55:33 qchen@qifan-10229:
IMPALA-7942_Add_query_hints_for_cardinalities_and_selectivities_wangsheng] cp
dml dml1
[14:55:35 qchen@qifan-10229:
IMPALA-7942_Add_query_hints_for_cardinalities_and_selectivities_wangsheng] vi
dml1
[14:56:19 qchen@qifan-10229:
IMPALA-7942_Add_query_hints_for_cardinalities_and_selectivities_wangsheng] sql
dml1
Starting Impala Shell with no authentication using Python 2.7.16
Warning: live_progress only applies to interactive shell sessions, and is being
skipped for now.
Opened TCP connection to localhost:21050
Connected to localhost:21050
Server version: impalad version 4.1.0-SNAPSHOT DEBUG (build
ad29ce70b3f4cfab4105be52ef664129060fa04e)
Query: explain select * from functional.alltypestiny where
(int_col = 1 or bigint_col < 10000) /* +SELECTIVITY(0.5) */
or
float_col < 2.0 /* +SELECTIVITY(0.5) */
+-----------------------------------------------------------------------+
| Explain String |
+-----------------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=4.01MB Threads=2 |
| Per-Host Resource Estimates: Memory=20MB |
| Codegen disabled by planner |
| |
| PLAN-ROOT SINK |
| | |
| 00:SCAN HDFS [functional.alltypestiny] |
| HDFS partitions=4/4 files=4 size=460B |
| predicates: (int_col = 1 OR bigint_col < 10000) OR float_col < 2.0 |
| row-size=89B cardinality=1 |
+-----------------------------------------------------------------------+
Fetched 10 row(s) in 0.01s
[14:56:21 qchen@qifan-10229:
IMPALA-7942_Add_query_hints_for_cardinalities_and_selectivities_wangsheng] !vi
vi dml1
[14:57:04 qchen@qifan-10229:
IMPALA-7942_Add_query_hints_for_cardinalities_and_selectivities_wangsheng] sql
dml1
Starting Impala Shell with no authentication using Python 2.7.16
Warning: live_progress only applies to interactive shell sessions, and is being
skipped for now.
Opened TCP connection to localhost:21050
Connected to localhost:21050
Server version: impalad version 4.1.0-SNAPSHOT DEBUG (build
ad29ce70b3f4cfab4105be52ef664129060fa04e)
Query: explain select * from functional.alltypestiny where
(int_col = 1 or bigint_col < 10000) /* +SELECTIVITY(0.5) */
or
float_col < 2.0 /* +SELECTIVITY(0.5) */
or
(float_col < 2.0) /* +SELECTIVITY(0.5) */ and double_col < 0.0
+---------------------------------------------------------------------------------------------------------------+
| Explain String
|
+---------------------------------------------------------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=4.01MB Threads=2
|
| Per-Host Resource Estimates: Memory=20MB
|
| Codegen disabled by planner
|
|
|
| PLAN-ROOT SINK
|
| |
|
| 00:SCAN HDFS [functional.alltypestiny]
|
| HDFS partitions=4/4 files=4 size=460B
|
| predicates: (int_col = 1 OR bigint_col < 10000) OR float_col < 2.0 OR
(float_col < 2.0) AND double_col < 0 |
| row-size=89B cardinality=1
|
+---------------------------------------------------------------------------------------------------------------+
Fetched 10 row(s) in 0.01s
explain
select * from functional.alltypestiny where
(int_col = 1 or bigint_col < 10000) /* +SELECTIVITY(0.5) */
or
float_col < 2.0 /* +SELECTIVITY(0.5) */
or
(float_col < 2.0) /* +SELECTIVITY(0.5) */ and double_col < 0.0
;
--
To view, visit http://gerrit.cloudera.org:8080/18023
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2776b9bbd878b8a21d9c866b400140a454f59e1b
Gerrit-Change-Number: 18023
Gerrit-PatchSet: 7
Gerrit-Owner: wangsheng <[email protected]>
Gerrit-Reviewer: Amogh Margoor <[email protected]>
Gerrit-Reviewer: Fucun Chu <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Qifan Chen <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
Gerrit-Reviewer: wangsheng <[email protected]>
Gerrit-Comment-Date: Fri, 17 Dec 2021 20:01:00 +0000
Gerrit-HasComments: No
