scarlin-cloudera commented on code in PR #4442:
URL: https://github.com/apache/hive/pull/4442#discussion_r1286159251
##########
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java:
##########
@@ -577,6 +588,77 @@ private QueryBlockInfo convertSource(RelNode r) throws
CalciteSemanticException
ast = ASTBuilder.subQuery(left, sqAlias);
s = new Schema((Union) r, sqAlias);
}
+ } else if (r instanceof HiveTableFunctionScan &&
+ !canOptimizeOutLateralView((HiveTableFunctionScan) r)) {
Review Comment:
I don't think this is possible to do with a CBO rule. Let me try to explain
why I think this will be a problem:
This is really a physical optimization, not a logical optimization. The
logical representation is the same so we can't make a rule.
You can see in some qtests that there are Lateral View Operators created.
This is needed when an explicit join is being made from the base table and the
"exploded" array fields. Each array element produces a separate row. If a
column is needed from the base table, that's when a join is needed.
The optimization that is being done here is to rid ourselves of this join
operation. If the only columns selected out are from the array and none from
the base table, no join is needed. We can therefore just use the UDTF Operator
rather than a Lateral Join.
This code could potentially be moved where we do physical operations, but I
felt this is the easiest place to implement this since the LATERAL VIEW syntax
is cleanly mapped with the Lateral view operator and the UDTF operator is used
when the AST tree is created without the LATERAL VIEW syntax.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
