[jira] [Commented] (KYLIN-5774) Optimize Calcite plan to convert spark logical plan

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/KYLIN-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835449#comment-17835449
 ] 

ASF subversion and git services commented on KYLIN-5774:


Commit 9db3502f80265753b38b4ecbb0ee57f7871b6466 in kylin's branch 
refs/heads/kylin5 from Mingming Ge
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=9db3502f80 ]

KYLIN-5774 Optimize Calcite plan to convert spark logical plan


> Optimize Calcite plan to convert spark logical plan
> ---
>
> Key: KYLIN-5774
> URL: https://issues.apache.org/jira/browse/KYLIN-5774
> Project: Kylin
>  Issue Type: Improvement
>  Components: Query Engine
>Affects Versions: 5.0-beta
>Reporter: pengfei.zhan
>Assignee: pengfei.zhan
>Priority: Major
> Fix For: 5.0-beta
>
>
> When converting the calcite plan to spark LogicalPlan ，using the DataFrame 
> interface means parsing while building the LogicalPlan.
> In this process，select, aggregation and other operators will execute the 
> method below, we need care the invoking of "qe.assertAnalyzed()".
> {code:java}
> def ofRows(sparkSession: SparkSession, logicalPlan: LogicalPlan): DataFrame =
>   sparkSession.withActive {
> val qe = sparkSession.sessionState.executePlan(logicalPlan)
> qe.assertAnalyzed()
> new Dataset[Row](qe, RowEncoder(qe.analyzed.schema))
> } {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (KYLIN-5774) Optimize Calcite plan to convert spark logical plan

[pengfei.zhan (Jira)]

[ 
https://issues.apache.org/jira/browse/KYLIN-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835376#comment-17835376
 ] 

pengfei.zhan commented on KYLIN-5774:
-

h1. Design

Do this work on LogicalPlan, rather than on DataFrame.

> Optimize Calcite plan to convert spark logical plan
> ---
>
> Key: KYLIN-5774
> URL: https://issues.apache.org/jira/browse/KYLIN-5774
> Project: Kylin
>  Issue Type: Improvement
>  Components: Query Engine
>Affects Versions: 5.0-beta
>Reporter: pengfei.zhan
>Assignee: pengfei.zhan
>Priority: Major
> Fix For: 5.0-beta
>
>
> When converting the calcite plan to spark LogicalPlan ，using the DataFrame 
> interface means parsing while building the LogicalPlan.
> In this process，select, aggregation and other operators will execute the 
> method below, we need care the invoking of "qe.assertAnalyzed()".
> {code:java}
> def ofRows(sparkSession: SparkSession, logicalPlan: LogicalPlan): DataFrame =
>   sparkSession.withActive {
> val qe = sparkSession.sessionState.executePlan(logicalPlan)
> qe.assertAnalyzed()
> new Dataset[Row](qe, RowEncoder(qe.analyzed.schema))
> } {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)