[
https://issues.apache.org/jira/browse/SPARK-54562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuheng Chang updated SPARK-54562:
---------------------------------
Description: Previously, we block eager analysis / execution inside the
flow function by intercepting the AnalyzePlan / ExecutePlan request from Python
the Spark Connect client and perform the validation. Now, since we have
PipelineAnalysisContext, we should be able to perform the validation from the
server side. (was: Support adding definition path as part of the
PipelineAnalysisContext holding info about the defining files of the users code
that triggers analysis)
> Block eager analysis / execution inside flow function from server side rather
> than client side
> ----------------------------------------------------------------------------------------------
>
> Key: SPARK-54562
> URL: https://issues.apache.org/jira/browse/SPARK-54562
> Project: Spark
> Issue Type: Sub-task
> Components: Declarative Pipelines
> Affects Versions: 4.1.1
> Reporter: Yuheng Chang
> Priority: Major
>
> Previously, we block eager analysis / execution inside the flow function by
> intercepting the AnalyzePlan / ExecutePlan request from Python the Spark
> Connect client and perform the validation. Now, since we have
> PipelineAnalysisContext, we should be able to perform the validation from the
> server side.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
