[jira] [Work logged] (HIVE-25485) Transform selects of literals under a UNION ALL to inline table scan
[
https://issues.apache.org/jira/browse/HIVE-25485?focusedWorklogId=653988&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653988
]
ASF GitHub Bot logged work on HIVE-25485:
-
Author: ASF GitHub Bot
Created on: 22/Sep/21 09:31
Start Date: 22/Sep/21 09:31
Worklog Time Spent: 10m
Work Description: kgyrtkirk merged pull request #2608:
URL: https://github.com/apache/hive/pull/2608
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653988)
Time Spent: 1.5h (was: 1h 20m)
> Transform selects of literals under a UNION ALL to inline table scan
>
>
> Key: HIVE-25485
> URL: https://issues.apache.org/jira/browse/HIVE-25485
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> {code}
> select 1
> union all
> select 1
> union all
> [...]
> union all
> select 1
> {code}
> results in a very big plan; which will have vertexes proportional to the
> number of union all branch - hence it could be slow to execute it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25485) Transform selects of literals under a UNION ALL to inline table scan
[
https://issues.apache.org/jira/browse/HIVE-25485?focusedWorklogId=653483&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653483
]
ASF GitHub Bot logged work on HIVE-25485:
-
Author: ASF GitHub Bot
Created on: 21/Sep/21 10:52
Start Date: 21/Sep/21 10:52
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #2608:
URL: https://github.com/apache/hive/pull/2608#issuecomment-923865132
@kasakrisz could you please take another look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653483)
Time Spent: 1h 20m (was: 1h 10m)
> Transform selects of literals under a UNION ALL to inline table scan
>
>
> Key: HIVE-25485
> URL: https://issues.apache.org/jira/browse/HIVE-25485
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> {code}
> select 1
> union all
> select 1
> union all
> [...]
> union all
> select 1
> {code}
> results in a very big plan; which will have vertexes proportional to the
> number of union all branch - hence it could be slow to execute it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25485) Transform selects of literals under a UNION ALL to inline table scan
[
https://issues.apache.org/jira/browse/HIVE-25485?focusedWorklogId=653481&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653481
]
ASF GitHub Bot logged work on HIVE-25485:
-
Author: ASF GitHub Bot
Created on: 21/Sep/21 10:52
Start Date: 21/Sep/21 10:52
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #2608:
URL: https://github.com/apache/hive/pull/2608#discussion_r712923238
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveTransformSimpleSelectsToInlineTableInUnion.java
##
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Project;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelRecordType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException;
+import org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable;
+import org.apache.hadoop.hive.ql.optimizer.calcite.TraitsUtil;
+import
org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableFunctionScan;
+import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan;
+import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveUnion;
+import
org.apache.hadoop.hive.ql.optimizer.calcite.translator.SqlFunctionConverter;
+import org.apache.hadoop.hive.ql.parse.SemanticAnalyzer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.collect.ImmutableList;
+
+/**
+ * Transforms SELECTS of literals under UNION ALL into inline table scans.
+ */
Review comment:
added some apidoc/etc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653481)
Time Spent: 1h 10m (was: 1h)
> Transform selects of literals under a UNION ALL to inline table scan
>
>
> Key: HIVE-25485
> URL: https://issues.apache.org/jira/browse/HIVE-25485
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {code}
> select 1
> union all
> select 1
> union all
> [...]
> union all
> select 1
> {code}
> results in a very big plan; which will have vertexes proportional to the
> number of union all branch - hence it could be slow to execute it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25485) Transform selects of literals under a UNION ALL to inline table scan
[ https://issues.apache.org/jira/browse/HIVE-25485?focusedWorklogId=651811&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651811 ] ASF GitHub Bot logged work on HIVE-25485: - Author: ASF GitHub Bot Created on: 16/Sep/21 15:26 Start Date: 16/Sep/21 15:26 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on a change in pull request #2608: URL: https://github.com/apache/hive/pull/2608#discussion_r710227073 ## File path: ql/src/test/results/clientpositive/llap/union_literals.q.out ## @@ -0,0 +1,397 @@ +PREHOOK: query: explain +SELECT * FROM ( + VALUES(1, '1'), + (2, 'orange'), + (5, 'yellow'), + (10, 'green'), + (11, 'blue'), + (12, 'indigo'), + (20, 'violet')) + AS Colors +PREHOOK: type: QUERY +PREHOOK: Input: _dummy_database@_dummy_table + A masked pattern was here +POSTHOOK: query: explain +SELECT * FROM ( + VALUES(1, '1'), + (2, 'orange'), + (5, 'yellow'), + (10, 'green'), + (11, 'blue'), + (12, 'indigo'), + (20, 'violet')) + AS Colors +POSTHOOK: type: QUERY +POSTHOOK: Input: _dummy_database@_dummy_table + A masked pattern was here +STAGE DEPENDENCIES: + Stage-0 is a root stage + +STAGE PLANS: + Stage: Stage-0 +Fetch Operator + limit: -1 + Processor Tree: +TableScan + alias: _dummy_table + Row Limit Per Split: 1 + Select Operator +expressions: array(const struct(1,'1'),const struct(2,'orange'),const struct(5,'yellow'),const struct(10,'green'),const struct(11,'blue'),const struct(12,'indigo'),const struct(20,'violet')) (type: array>) +outputColumnNames: _col0 +UDTF Operator + function name: inline + Select Operator +expressions: col1 (type: int), col2 (type: string) +outputColumnNames: _col0, _col1 +ListSink + +PREHOOK: query: explain +SELECT * FROM ( + VALUES(1, '1'), + (2, 'orange'), + (5, 'yellow'), + (10, 'green'), + (11, 'blue'), + (12, 'indigo'), + (20, 'violet')) + AS Colors +union all + select 2,'2' +union all + select 2,'2' +PREHOOK: type: QUERY +PREHOOK: Input: _dummy_database@_dummy_table + A masked pattern was here +POSTHOOK: query: explain +SELECT * FROM ( + VALUES(1, '1'), + (2, 'orange'), + (5, 'yellow'), + (10, 'green'), + (11, 'blue'), + (12, 'indigo'), + (20, 'violet')) + AS Colors +union all + select 2,'2' +union all + select 2,'2' +POSTHOOK: type: QUERY +POSTHOOK: Input: _dummy_database@_dummy_table + A masked pattern was here +STAGE DEPENDENCIES: + Stage-1 is a root stage + Stage-0 depends on stages: Stage-1 + +STAGE PLANS: + Stage: Stage-1 +Tez + A masked pattern was here + Edges: +Map 1 <- Union 2 (CONTAINS) +Map 3 <- Union 2 (CONTAINS) + A masked pattern was here + Vertices: +Map 1 +Map Operator Tree: +TableScan + alias: _dummy_table + Row Limit Per Split: 1 + Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: COMPLETE + Select Operator +expressions: array(const struct(2,'2'),const struct(2,'2')) (type: array>) +outputColumnNames: _col0 +Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE +UDTF Operator + Statistics: Num rows: 1 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE + function name: inline + Select Operator +expressions: col1 (type: int), col2 (type: string) +outputColumnNames: _col0, _col1 +Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE +File Output Operator + compressed: false + Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE + table: + input format: org.apache.hadoop.mapred.SequenceFileInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat + serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe +Execution mode: llap +LLAP IO: no inputs +Map 3 +Map Operator Tree: +TableScan + alias: _dummy_table + Row Limit Per Split: 1 + Statistics: Num rows: 1 Data size: 10 Basic stats: CO
[jira] [Work logged] (HIVE-25485) Transform selects of literals under a UNION ALL to inline table scan
[
https://issues.apache.org/jira/browse/HIVE-25485?focusedWorklogId=651787&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651787
]
ASF GitHub Bot logged work on HIVE-25485:
-
Author: ASF GitHub Bot
Created on: 16/Sep/21 15:07
Start Date: 16/Sep/21 15:07
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #2608:
URL: https://github.com/apache/hive/pull/2608#discussion_r710210766
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveTransformSimpleSelectsToInlineTableInUnion.java
##
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Project;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelRecordType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException;
+import org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable;
+import org.apache.hadoop.hive.ql.optimizer.calcite.TraitsUtil;
+import
org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableFunctionScan;
+import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan;
+import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveUnion;
+import
org.apache.hadoop.hive.ql.optimizer.calcite.translator.SqlFunctionConverter;
+import org.apache.hadoop.hive.ql.parse.SemanticAnalyzer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.collect.ImmutableList;
+
+/**
+ * Transforms SELECTS of literals under UNION ALL into inline table scans.
+ */
+public class HiveTransformSimpleSelectsToInlineTableInUnion extends RelOptRule
{
Review comment:
I was unable to give it a decent name
renamed the class :+1:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 651787)
Time Spent: 50m (was: 40m)
> Transform selects of literals under a UNION ALL to inline table scan
>
>
> Key: HIVE-25485
> URL: https://issues.apache.org/jira/browse/HIVE-25485
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {code}
> select 1
> union all
> select 1
> union all
> [...]
> union all
> select 1
> {code}
> results in a very big plan; which will have vertexes proportional to the
> number of union all branch - hence it could be slow to execute it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25485) Transform selects of literals under a UNION ALL to inline table scan
[
https://issues.apache.org/jira/browse/HIVE-25485?focusedWorklogId=651058&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651058
]
ASF GitHub Bot logged work on HIVE-25485:
-
Author: ASF GitHub Bot
Created on: 15/Sep/21 12:27
Start Date: 15/Sep/21 12:27
Worklog Time Spent: 10m
Work Description: kasakrisz commented on a change in pull request #2608:
URL: https://github.com/apache/hive/pull/2608#discussion_r709113570
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveTransformSimpleSelectsToInlineTableInUnion.java
##
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Project;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelRecordType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException;
+import org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable;
+import org.apache.hadoop.hive.ql.optimizer.calcite.TraitsUtil;
+import
org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableFunctionScan;
+import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan;
+import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveUnion;
+import
org.apache.hadoop.hive.ql.optimizer.calcite.translator.SqlFunctionConverter;
+import org.apache.hadoop.hive.ql.parse.SemanticAnalyzer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.collect.ImmutableList;
+
+/**
+ * Transforms SELECTS of literals under UNION ALL into inline table scans.
+ */
Review comment:
Could you please add a simple example?
Also a input and output plan something like
from
```
HiveUnion(all=true)
HiveProject(a=[1])
...
HiveProject(a=[42])
...
```
to
```
HiveProject(a=[$0])
HiveTableScan(table=...)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 651058)
Time Spent: 40m (was: 0.5h)
> Transform selects of literals under a UNION ALL to inline table scan
>
>
> Key: HIVE-25485
> URL: https://issues.apache.org/jira/browse/HIVE-25485
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {code}
> select 1
> union all
> select 1
> union all
> [...]
> union all
> select 1
> {code}
> results in a very big plan; which will have vertexes proportional to the
> number of union all branch - hence it could be slow to execute it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25485) Transform selects of literals under a UNION ALL to inline table scan
[
https://issues.apache.org/jira/browse/HIVE-25485?focusedWorklogId=651057&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651057
]
ASF GitHub Bot logged work on HIVE-25485:
-
Author: ASF GitHub Bot
Created on: 15/Sep/21 12:26
Start Date: 15/Sep/21 12:26
Worklog Time Spent: 10m
Work Description: kasakrisz commented on a change in pull request #2608:
URL: https://github.com/apache/hive/pull/2608#discussion_r709110069
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveTransformSimpleSelectsToInlineTableInUnion.java
##
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Project;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelRecordType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException;
+import org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable;
+import org.apache.hadoop.hive.ql.optimizer.calcite.TraitsUtil;
+import
org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableFunctionScan;
+import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan;
+import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveUnion;
+import
org.apache.hadoop.hive.ql.optimizer.calcite.translator.SqlFunctionConverter;
+import org.apache.hadoop.hive.ql.parse.SemanticAnalyzer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.collect.ImmutableList;
+
+/**
+ * Transforms SELECTS of literals under UNION ALL into inline table scans.
+ */
+public class HiveTransformSimpleSelectsToInlineTableInUnion extends RelOptRule
{
Review comment:
It is a long name but in case of other rules we add the word `Rule`.
How about `HiveUnionSimpleSelectsToInlineTableRule` ?
##
File path: ql/src/test/results/clientpositive/llap/union_literals.q.out
##
@@ -0,0 +1,397 @@
+PREHOOK: query: explain
+SELECT * FROM (
+ VALUES(1, '1'),
+ (2, 'orange'),
+ (5, 'yellow'),
+ (10, 'green'),
+ (11, 'blue'),
+ (12, 'indigo'),
+ (20, 'violet'))
+ AS Colors
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+ A masked pattern was here
+POSTHOOK: query: explain
+SELECT * FROM (
+ VALUES(1, '1'),
+ (2, 'orange'),
+ (5, 'yellow'),
+ (10, 'green'),
+ (11, 'blue'),
+ (12, 'indigo'),
+ (20, 'violet'))
+ AS Colors
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+ A masked pattern was here
+STAGE DEPENDENCIES:
+ Stage-0 is a root stage
+
+STAGE PLANS:
+ Stage: Stage-0
+Fetch Operator
+ limit: -1
+ Processor Tree:
+TableScan
+ alias: _dummy_table
+ Row Limit Per Split: 1
+ Select Operator
+expressions: array(const struct(1,'1'),const
struct(2,'orange'),const struct(5,'yellow'),const struct(10,'green'),const
struct(11,'blue'),const struct(12,'indigo'),const struct(20,'violet')) (type:
array>)
+outputColumnNames: _col0
+UDTF Operator
+ function name: inline
+ Select Operator
+expressions: col1 (type: int), col2 (type: string)
+outputColumnNames: _col0, _col1
+ListSink
+
+PREHOOK: query: explain
+SELECT * FROM (
+ VALUES(1, '1'),
+ (2, 'orange'),
+ (5, 'yellow'),
+
[jira] [Work logged] (HIVE-25485) Transform selects of literals under a UNION ALL to inline table scan
[
https://issues.apache.org/jira/browse/HIVE-25485?focusedWorklogId=651033&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651033
]
ASF GitHub Bot logged work on HIVE-25485:
-
Author: ASF GitHub Bot
Created on: 15/Sep/21 11:34
Start Date: 15/Sep/21 11:34
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #2608:
URL: https://github.com/apache/hive/pull/2608#issuecomment-919939445
@kasakrisz could you please take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 651033)
Time Spent: 20m (was: 10m)
> Transform selects of literals under a UNION ALL to inline table scan
>
>
> Key: HIVE-25485
> URL: https://issues.apache.org/jira/browse/HIVE-25485
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code}
> select 1
> union all
> select 1
> union all
> [...]
> union all
> select 1
> {code}
> results in a very big plan; which will have vertexes proportional to the
> number of union all branch - hence it could be slow to execute it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25485) Transform selects of literals under a UNION ALL to inline table scan
[
https://issues.apache.org/jira/browse/HIVE-25485?focusedWorklogId=642851&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-642851
]
ASF GitHub Bot logged work on HIVE-25485:
-
Author: ASF GitHub Bot
Created on: 27/Aug/21 14:41
Start Date: 27/Aug/21 14:41
Worklog Time Spent: 10m
Work Description: kgyrtkirk opened a new pull request #2608:
URL: https://github.com/apache/hive/pull/2608
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 642851)
Remaining Estimate: 0h
Time Spent: 10m
> Transform selects of literals under a UNION ALL to inline table scan
>
>
> Key: HIVE-25485
> URL: https://issues.apache.org/jira/browse/HIVE-25485
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> select 1
> union all
> select 1
> union all
> [...]
> union all
> select 1
> {code}
> results in a very big plan; which will have vertexes proportional to the
> number of union all branch - hence it could be slow to execute it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
