Hongrong Cao created KYLIN-5723:
-----------------------------------
Summary: Optimize Calcite plan to convert spark logical plan
Key: KYLIN-5723
URL: https://issues.apache.org/jira/browse/KYLIN-5723
Project: Kylin
Issue Type: Improvement
Affects Versions: 5.0-beta
Reporter: Hongrong Cao
Assignee: Hongrong Cao
Fix For: 5.0.0
The problem with calcite plan to spark LogicalPlan is that it uses the
DataFrame interface, which means that it parses and builds the LogicalPlan at
the same time.
During the process of calcite plan to spark plan, the select agg and other
operators execute the following methods, where the execution of
qe.assertAnalyzed() is of interest
{quote}def ofRows(sparkSession: SparkSession, logicalPlan: LogicalPlan):
DataFrame =
sparkSession.withActive {
val qe = sparkSession.sessionState.executePlan(logicalPlan)
qe.assertAnalyzed()
new Dataset[Row](qe, RowEncoder(qe.analyzed.schema))
}
{quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
