[
https://issues.apache.org/jira/browse/SPARK-48338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18086723#comment-18086723
]
Wenchen Fan edited comment on SPARK-48338 at 6/7/26 1:27 PM:
-------------------------------------------------------------
[~dongjoon] thanks for the cleanup! As I mentioned, we should have resolved
this ticket before releasing 4.1.0, and I'm not hiding this mistake.
My question is more about the policy we want to enforce (I don't think it's
clearly established today): shall we allow people to create more subtasks on a
resolved umbrella ticket? Or can we resolve an umbrella ticket when there are
still open subtasks but non-blocking?
For release notes: this does not matter. If an umbrella ticket is resolved at
4.1.0, we won't mention it again in later versions. So new subtasks in it will
be treated as independent tickets. The extra effort to move new subtasks out
may not be worthwhile.
was (Author: cloud_fan):
[~dongjoon] thanks for the cleanup! As I mentioned, we should have resolved
this ticket before releasing 4.1.0, and I'm not hiding this mistake.
My question is more about the policy we want to enforce (I don't think it's
clearly established today): shall we allow people to create more subtasks on a
resolved umbrella ticket? Or can we resolve an umbrella ticket when there are
still open subtasks but non-blocking?
For release notes: this does not matter. If an umbrella ticket is resolved at
4.1.0, we won't mention it again in later versions. So new subtasks in it will
be treated as independent tickets.
> Sql Scripting support for Spark SQL
> -----------------------------------
>
> Key: SPARK-48338
> URL: https://issues.apache.org/jira/browse/SPARK-48338
> Project: Spark
> Issue Type: Umbrella
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Aleksandar Tomic
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
> Attachments: Screenshot 2026-06-04 at 21.12.59.png, Sql Scripting -
> OSS.odt, [Design Doc] Sql Scripting - OSS.pdf
>
>
> Design doc for this feature is in attachment.
> *High level example of Sql Script:*
> {code:java}
> BEGIN
> DECLARE c INT = 10;
> WHILE c > 0 DO
> INSERT INTO tscript VALUES (c);
> SET c = c - 1;
> END WHILE;
> END{code}
> *High level motivation behind this feature:*
> SQL Scripting gives customers the ability to develop complex ETL and analysis
> entirely in SQL. Until now, customers have had to write verbose SQL
> statements or combine SQL + Python to efficiently write business logic.
> Coming from another system, customers have to choose whether or not they want
> to migrate to pyspark. Some customers end up not using Spark because of this
> gap. SQL Scripting is a key milestone towards enabling SQL practitioners to
> write sophisticated queries, without the need to use pyspark. Further, SQL
> Scripting is a necessary step towards support for SQL Stored Procedures, and
> along with SQL Variables (released) and Temp Tables (in progress), will allow
> for more seamless data warehouse migrations.
> *Work items classification:*
> * M0 - basic support
> * M1 - features and changes required to enable SQL Scripting by default
> * M2 - follow-up improvements and additional functionalities that are
> non-fundamental and should not block M1
> * M3 - potential improvements for the future, need investigation
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
