[
https://issues.apache.org/jira/browse/SPARK-43074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zzzzming95 updated SPARK-43074:
-------------------------------
Description:
{code:java}
```
df = spark.sql("select 1")
catalyst_plan = df._jdf.queryExecution().logical()
print('catalyst_plan: ', catalyst_plan)
df_size = spark._jsparkSession.sessionState().executePlan(catalyst_plan)
```
{code}
will get exception beacause py4j not support default value of scala.
[https://github.com/apache/spark/blob/d8b720a579a456aec9c7f843d7feaa1454ebf9d4/sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala#L124]
{code:java}
```
def executePlan(
plan: LogicalPlan,
mode: CommandExecutionMode.Value = CommandExecutionMode.ALL): QueryExecution =
createQueryExecution(plan, mode)
```
{code}
Will get Exception:
{code:java}
```
py4j.protocol.Py4JError: An error occurred while calling o87.executePlan. Trace:
py4j.Py4JException: Method executePlan([class
org.apache.spark.sql.catalyst.plans.logical.Project]) does not exist
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:321)
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:329)
at py4j.Gateway.invoke(Gateway.java:274)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at
py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
at java.lang.Thread.run(Thread.java:748)
```
{code}
was:
```
df = spark.sql("select 1")
catalyst_plan = df._jdf.queryExecution().logical()
print('catalyst_plan: ', catalyst_plan)
df_size = spark._jsparkSession.sessionState().executePlan(catalyst_plan)
```
will get exception beacause py4j not support default value of scala.
https://github.com/apache/spark/blob/d8b720a579a456aec9c7f843d7feaa1454ebf9d4/sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala#L124
```
def executePlan(
plan: LogicalPlan,
mode: CommandExecutionMode.Value = CommandExecutionMode.ALL): QueryExecution =
createQueryExecution(plan, mode)
```
Will get Exception:
```
py4j.protocol.Py4JError: An error occurred while calling o87.executePlan. Trace:
py4j.Py4JException: Method executePlan([class
org.apache.spark.sql.catalyst.plans.logical.Project]) does not exist
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:321)
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:329)
at py4j.Gateway.invoke(Gateway.java:274)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at
py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
at java.lang.Thread.run(Thread.java:748)
```
> Add the function without constant parameters of `SessionState#executePlan`
> --------------------------------------------------------------------------
>
> Key: SPARK-43074
> URL: https://issues.apache.org/jira/browse/SPARK-43074
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.2.3
> Reporter: zzzzming95
> Priority: Major
>
>
> {code:java}
> ```
> df = spark.sql("select 1")
> catalyst_plan = df._jdf.queryExecution().logical()
> print('catalyst_plan: ', catalyst_plan)
> df_size = spark._jsparkSession.sessionState().executePlan(catalyst_plan)
> ```
> {code}
>
>
> will get exception beacause py4j not support default value of scala.
>
> [https://github.com/apache/spark/blob/d8b720a579a456aec9c7f843d7feaa1454ebf9d4/sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala#L124]
>
> {code:java}
> ```
> def executePlan(
> plan: LogicalPlan,
> mode: CommandExecutionMode.Value = CommandExecutionMode.ALL): QueryExecution =
> createQueryExecution(plan, mode)
> ```
> {code}
>
>
> Will get Exception:
>
>
> {code:java}
> ```
> py4j.protocol.Py4JError: An error occurred while calling o87.executePlan.
> Trace:
> py4j.Py4JException: Method executePlan([class
> org.apache.spark.sql.catalyst.plans.logical.Project]) does not exist
> at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:321)
> at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:329)
> at py4j.Gateway.invoke(Gateway.java:274)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at
> py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
> at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
> at java.lang.Thread.run(Thread.java:748)
> ```
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
