Re: Apache Spark 4.0.0 Dev Item Planning (SPARK-44111)

[Dongjoon Hyun]

Thank you, Yuming.

Please update SPARK-44111 by adding links to those JIRA for visibility.
Otherwise, we may miss them during the up-coming discussion.

Dongjoon.



On Tue, Jun 20, 2023 at 6:40 PM Yuming Wang  wrote:

> Thank you Dongjoon. I'd like to add these items.
>
> *Support for more SQL syntax*
> SPARK-31561  Add
> QUALIFY clause
> SPARK-24497  Support
> recursive SQL
> SPARK-32064  Support
> temporary table
>
> *Improve Query performance in specific scenarios*
> SPARK-8682  Range Join
> for Spark SQL. We have a blog in Chinese
>  about this
> optimization.
> SPARK-38506  Push
> partial aggregation through join
>
>
> On Wed, Jun 21, 2023 at 4:42 AM Dongjoon Hyun  wrote:
>
>> Hi, All.
>>
>> As a continuation of our previous discussion, the official Apache Spark
>> 4.0 Plan JIRA is created today in order to collect the community dev items.
>> Feel free to add your work items, ideas, suggestions, aspirations and
>> interests. We will moderate together.
>>
>> https://issues.apache.org/jira/browse/SPARK-44111
>> Prepare Apache Spark 4.0.0
>>
>> In addition, we are going to include all left-over items which Apache
>> Spark 3.5 cannot include on July 16th (Feature Freeze,
>> https://spark.apache.org/versioning-policy.html)
>>
>>
>>
>> === PREVIOUS THREADS ===
>>
>> 2023-05-28 Apache Spark 3.5.0 Expectations (?)
>> https://lists.apache.org/thread/3x6dh17bmy20n3frtt3crgxjydnxh2o0
>>
>> 2023-05-30 Apache Spark 4.0 Timeframe?
>> https://lists.apache.org/thread/xhkgj60j361gdpywoxxz7qspp2w80ry6
>>
>> 2023-06-05 ASF policy violation and Scala version issues
>> https://lists.apache.org/thread/k7gr65wt0fwtldc7hp7bd0vkg1k93rrb
>>
>> 2023-06-12 [VOTE] Release Plan for Apache Spark 4.0.0 (June 2024)
>> https://lists.apache.org/thread/r0zn6rd8y25yn2dg59ktw3ttrwxzqrfb
>>
>> 2023-06-16 [VOTE][RESULT] Release Plan for Apache Spark 4.0.0 (June 2024)
>> https://lists.apache.org/thread/5vfof0nm82gt5b2k2o0ws944hofz232g
>>
>

Re: Apache Spark 4.0.0 Dev Item Planning (SPARK-44111)

[Yuming Wang]

Thank you Dongjoon. I'd like to add these items.

*Support for more SQL syntax*
SPARK-31561  Add QUALIFY
clause
SPARK-24497  Support
recursive SQL
SPARK-32064  Support
temporary table

*Improve Query performance in specific scenarios*
SPARK-8682  Range Join
for Spark SQL. We have a blog in Chinese
 about this optimization.
SPARK-38506  Push
partial aggregation through join


On Wed, Jun 21, 2023 at 4:42 AM Dongjoon Hyun  wrote:

> Hi, All.
>
> As a continuation of our previous discussion, the official Apache Spark
> 4.0 Plan JIRA is created today in order to collect the community dev items.
> Feel free to add your work items, ideas, suggestions, aspirations and
> interests. We will moderate together.
>
> https://issues.apache.org/jira/browse/SPARK-44111
> Prepare Apache Spark 4.0.0
>
> In addition, we are going to include all left-over items which Apache
> Spark 3.5 cannot include on July 16th (Feature Freeze,
> https://spark.apache.org/versioning-policy.html)
>
>
>
> === PREVIOUS THREADS ===
>
> 2023-05-28 Apache Spark 3.5.0 Expectations (?)
> https://lists.apache.org/thread/3x6dh17bmy20n3frtt3crgxjydnxh2o0
>
> 2023-05-30 Apache Spark 4.0 Timeframe?
> https://lists.apache.org/thread/xhkgj60j361gdpywoxxz7qspp2w80ry6
>
> 2023-06-05 ASF policy violation and Scala version issues
> https://lists.apache.org/thread/k7gr65wt0fwtldc7hp7bd0vkg1k93rrb
>
> 2023-06-12 [VOTE] Release Plan for Apache Spark 4.0.0 (June 2024)
> https://lists.apache.org/thread/r0zn6rd8y25yn2dg59ktw3ttrwxzqrfb
>
> 2023-06-16 [VOTE][RESULT] Release Plan for Apache Spark 4.0.0 (June 2024)
> https://lists.apache.org/thread/5vfof0nm82gt5b2k2o0ws944hofz232g
>

Apache Spark 4.0.0 Dev Item Planning (SPARK-44111)

[Dongjoon Hyun]

Hi, All.

As a continuation of our previous discussion, the official Apache Spark 4.0
Plan JIRA is created today in order to collect the community dev items.
Feel free to add your work items, ideas, suggestions, aspirations and
interests. We will moderate together.

https://issues.apache.org/jira/browse/SPARK-44111
Prepare Apache Spark 4.0.0

In addition, we are going to include all left-over items which Apache Spark
3.5 cannot include on July 16th (Feature Freeze,
https://spark.apache.org/versioning-policy.html)



=== PREVIOUS THREADS ===

2023-05-28 Apache Spark 3.5.0 Expectations (?)
https://lists.apache.org/thread/3x6dh17bmy20n3frtt3crgxjydnxh2o0

2023-05-30 Apache Spark 4.0 Timeframe?
https://lists.apache.org/thread/xhkgj60j361gdpywoxxz7qspp2w80ry6

2023-06-05 ASF policy violation and Scala version issues
https://lists.apache.org/thread/k7gr65wt0fwtldc7hp7bd0vkg1k93rrb

2023-06-12 [VOTE] Release Plan for Apache Spark 4.0.0 (June 2024)
https://lists.apache.org/thread/r0zn6rd8y25yn2dg59ktw3ttrwxzqrfb

2023-06-16 [VOTE][RESULT] Release Plan for Apache Spark 4.0.0 (June 2024)
https://lists.apache.org/thread/5vfof0nm82gt5b2k2o0ws944hofz232g