Rui Wang created SPARK-40586:
--------------------------------
Summary: Decouple plan transformation and validation on server
side
Key: SPARK-40586
URL: https://issues.apache.org/jira/browse/SPARK-40586
Project: Spark
Issue Type: Sub-task
Components: Connect
Affects Versions: 3.4.0
Reporter: Rui Wang
Project connect, from some perspectives, can be thought as replacing the SQL
parser to generate a parsed (but the difference that is unresolved) plan, then
the plan is passed to the analyzer. This means that connect should also do
validation on the proto as there are many in-validate parser cases that
analyzer does not expect to see, which potentially could cause problems if
connect only pass through the proto (of course have it translated) to analyzer.
Meanwhile I think this is a good idea to decouple the validation and
transformation so that we have two stages:
stage 1: proto validation. For example validate if necessary fields are
populated or not.
stage 2: transformation, which convert the proto to a plan with assumption that
the plan is valid parsed version of the plan.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
