[
https://issues.apache.org/jira/browse/HIVE-25938?focusedWorklogId=730814&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730814
]
ASF GitHub Bot logged work on HIVE-25938:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Feb/22 11:33
Start Date: 22/Feb/22 11:33
Worklog Time Spent: 10m
Work Description: asolimando commented on a change in pull request #3011:
URL: https://github.com/apache/hive/pull/3011#discussion_r811850345
##########
File path: ql/src/test/results/clientpositive/llap/excluded_rule_explain.q.out
##########
@@ -0,0 +1,112 @@
+PREHOOK: query: EXPLAIN CBO SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+CBO PLAN:
+HiveProject(_o__c0=[1])
+ HiveTableScan(table=[[_dummy_database, _dummy_table]],
table:alias=[_dummy_table])
+
+PREHOOK: query: EXPLAIN FORMATTED CBO SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN FORMATTED CBO SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+{"CBOPlan":"{\n \"rels\": [\n {\n \"id\": \"0\",\n \"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan\",\n
\"table\": [\n \"_dummy_database\",\n \"_dummy_table\"\n
],\n \"table:alias\": \"_dummy_table\",\n \"inputs\": [],\n
\"rowCount\": 1.0,\n \"avgRowSize\": 129.0,\n \"rowType\": [\n
{\n \"type\": \"BIGINT\",\n \"nullable\": true,\n
\"name\": \"BLOCK__OFFSET__INSIDE__FILE\"\n },\n {\n
\"type\": \"VARCHAR\",\n \"nullable\": true,\n \"precision\":
2147483647,\n \"name\": \"INPUT__FILE__NAME\"\n },\n {\n
\"fields\": [\n {\n \"type\": \"BIGINT\",\n
\"nullable\": true,\n \"name\": \"writeid\"\n
},\n {\n \"type\": \"INTEGER\",\n
\"nullable\": true,\n \"name\": \"bucketid\"\n },\n
{\n \"type\": \"BIGINT\",\n \"nullable\":
true,\n \"name\": \"rowid\"\n }\n ],\n
\"name\": \"ROW__ID\"\n },\n {\n \"type\":
\"BOOLEAN\",\n \"nullable\": true,\n \"name\":
\"ROW__IS__DELETED\"\n }\n ]\n },\n {\n \"id\": \"1\",\n
\"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject\",\n
\"fields\": [\n \"_o__c0\"\n ],\n \"exprs\": [\n {\n
\"literal\": 1,\n \"type\": {\n \"type\":
\"INTEGER\",\n \"nullable\": false\n }\n }\n
],\n \"rowCount\": 1.0\n }\n ]\n}"}
+PREHOOK: query: EXPLAIN SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+STAGE DEPENDENCIES:
+ Stage-0 is a root stage
+
+STAGE PLANS:
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ TableScan
+ alias: _dummy_table
+ Row Limit Per Split: 1
+ Select Operator
+ expressions: 1 (type: int)
+ outputColumnNames: _col0
+ ListSink
+
+PREHOOK: query: EXPLAIN FORMATTED SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN FORMATTED SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+{"CBOPlan":"{\n \"rels\": [\n {\n \"id\": \"0\",\n \"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan\",\n
\"table\": [\n \"_dummy_database\",\n \"_dummy_table\"\n
],\n \"table:alias\": \"_dummy_table\",\n \"inputs\": [],\n
\"rowCount\": 1.0,\n \"avgRowSize\": 0.0,\n \"rowType\": [\n
{\n \"type\": \"BIGINT\",\n \"nullable\": true,\n
\"name\": \"BLOCK__OFFSET__INSIDE__FILE\"\n },\n {\n
\"type\": \"VARCHAR\",\n \"nullable\": true,\n \"precision\":
2147483647,\n \"name\": \"INPUT__FILE__NAME\"\n },\n {\n
\"fields\": [\n {\n \"type\": \"BIGINT\",\n
\"nullable\": true,\n \"name\": \"writeid\"\n
},\n {\n \"type\": \"INTEGER\",\n
\"nullable\": true,\n \"name\": \"bucketid\"\n },\n
{\n \"type\": \"BIGINT\",\n \"nullable\":
true,\n \"name\": \"rowid\"\n }\n ],\n
\"name\": \"ROW__ID\"\n },\n {\n \"type\":
\"BOOLEAN\",\n \"nullable\": true,\n \"name\":
\"ROW__IS__DELETED\"\n }\n ]\n },\n {\n \"id\": \"1\",\n
\"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject\",\n
\"fields\": [\n \"_o__c0\"\n ],\n \"exprs\": [\n {\n
\"literal\": 1,\n \"type\": {\n \"type\":
\"INTEGER\",\n \"nullable\": false\n }\n }\n
],\n \"rowCount\": 1.0\n }\n ]\n}","optimizedSQL":"SELECT 1 AS
`_o__c0`\nFROM `_dummy_database`.`_dummy_table`","cboInfo":"Plan optimized by
CBO.","STAGE DEPENDENCIES":{"Stage-0":{"ROOT STAGE":"TRUE"}},"STAGE
PLANS":{"Stage-0":{"Fetch Operator":{"limit:":"-1","Processor
Tree:":{"TableScan":{"alias:":"_dummy_table","database:":"_dummy_database","Row
Limit Per
Split:":"1","table:":"_dummy_table","isTempTable:":"false","OperatorId:":"TS_0","children":{"Select
Operator":{"expressions:":"1 (type:
int)","columnExprMap:":{"_col0":"1"},"outputColumnNames:":["_col0"],"OperatorId:":"SEL_1","children":{"ListSink":{"OperatorId:":"LIST_SINK_3"}}}}}}}}}}
+PREHOOK: query: EXPLAIN CBO SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+Excluded rules: HiveJoinPushTransitivePredicatesRule|HivePreFilteringRule
+
+CBO PLAN:
+HiveProject(_o__c0=[1])
+ HiveTableScan(table=[[_dummy_database, _dummy_table]],
table:alias=[_dummy_table])
+
+PREHOOK: query: EXPLAIN FORMATTED CBO SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN FORMATTED CBO SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+{"CBOPlan":"{\n \"rels\": [\n {\n \"id\": \"0\",\n \"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan\",\n
\"table\": [\n \"_dummy_database\",\n \"_dummy_table\"\n
],\n \"table:alias\": \"_dummy_table\",\n \"inputs\": [],\n
\"rowCount\": 1.0,\n \"avgRowSize\": 129.0,\n \"rowType\": [\n
{\n \"type\": \"BIGINT\",\n \"nullable\": true,\n
\"name\": \"BLOCK__OFFSET__INSIDE__FILE\"\n },\n {\n
\"type\": \"VARCHAR\",\n \"nullable\": true,\n \"precision\":
2147483647,\n \"name\": \"INPUT__FILE__NAME\"\n },\n {\n
\"fields\": [\n {\n \"type\": \"BIGINT\",\n
\"nullable\": true,\n \"name\": \"writeid\"\n
},\n {\n \"type\": \"INTEGER\",\n
\"nullable\": true,\n \"name\": \"bucketid\"\n },\n
{\n \"type\": \"BIGINT\",\n \"nullable\":
true,\n \"name\": \"rowid\"\n }\n ],\n
\"name\": \"ROW__ID\"\n },\n {\n \"type\":
\"BOOLEAN\",\n \"nullable\": true,\n \"name\":
\"ROW__IS__DELETED\"\n }\n ]\n },\n {\n \"id\": \"1\",\n
\"relOp\":
\"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject\",\n
\"fields\": [\n \"_o__c0\"\n ],\n \"exprs\": [\n {\n
\"literal\": 1,\n \"type\": {\n \"type\":
\"INTEGER\",\n \"nullable\": false\n }\n }\n
],\n \"rowCount\": 1.0\n }\n ]\n}","cboInfo":"Excluded rules:
HiveJoinPushTransitivePredicatesRule|HivePreFilteringRule"}
+PREHOOK: query: EXPLAIN SELECT 1
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN SELECT 1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+Excluded rules: HiveJoinPushTransitivePredicatesRule|HivePreFilteringRule
Review comment:
Sorry, I must have missed the line you were mentioning. In this specific
case it's the `hive.explain.user` property that is missing. The default value
is `true`, however, there must be an override somewhere because it's not
activated by default.
When setting `set hive.explain.user=true;`, `EXPLAIN SELECT 1;` prints `Plan
optimized by CBO. Excluded rules:
HiveJoinPushTransitivePredicatesRule|HivePreFilteringRule` instead of the
simple `Excluded rules:
HiveJoinPushTransitivePredicatesRule|HivePreFilteringRule`.
If you want I can add what follows at the end of the .q file:
```
set hive.explain.user=true;
EXPLAIN SELECT 1;
```
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 730814)
Time Spent: 1h 40m (was: 1.5h)
> Print excluded rules from CBO
> -----------------------------
>
> Key: HIVE-25938
> URL: https://issues.apache.org/jira/browse/HIVE-25938
> Project: Hive
> Issue Type: Improvement
> Components: CBO
> Affects Versions: 4.0.0
> Reporter: Alessandro Solimando
> Assignee: Alessandro Solimando
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> HIVE-25880 introduced a configuration parameter for excluding CBO rules based
> on a regex on their description.
> Calcite logs when a rule is excluded (see
> [AbstractRelOptPlanner.java#L316|https://github.com/apache/calcite/blob/e42b85a45bd16dd58db1546736e653deda5463fe/core/src/main/java/org/apache/calcite/plan/AbstractRelOptPlanner.java#L316]
> and
> [VolcanoRuleCall.java#L169|https://github.com/apache/calcite/blob/e42b85a45bd16dd58db1546736e653deda5463fe/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoRuleCall.java#L169]).
> To ease investigations, this should be complemented by printing the regex
> used (if not blank) in Hive DEBUG logs, and the same in CBO information
> (i.e., EXPLAIN CBO's output).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
