Re: The draft of the Spark 3.3.0 release notes

[Bjørn Jørgensen]

   -

   Support lambda `column` parameter of `DataFrame.rename`(SPARK-38763
   )


This did work before 18.jan.2022, see JIRA post for more information.
I think we can remove this one from the list.

fre. 3. jun. 2022 kl. 20:42 skrev Dongjoon Hyun :

> You are right.
>
> After SPARK-36837, we tried to ship Apache Spark 3.3.0 with Apache Kafka
> 3.1.1 via the following PR.
>
> https://github.com/apache/spark/pull/36135
> [WIP][SPARK-38850][BUILD] Upgrade Kafka to 3.1.1
>
> However, the final decision was to revert it from `branch-3.3` and move
> directly to Apache Kafka 3.2.0 at `master` branch. We need to remove it
> from the 3.3.0 release note.
>
>
> On Fri, Jun 3, 2022 at 9:54 AM Koert Kuipers  wrote:
>
>> i thought SPARK-36837 didnt make it in? i see it in notes
>>
>> On Fri, Jun 3, 2022 at 4:31 AM Maxim Gekk
>>  wrote:
>>
>>> Hi All,
>>>
>>> I am preparing the release notes of Spark 3.3.0. Here is a draft
>>> document:
>>>
>>> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>>>
>>> Please take a look and let me know if I missed any major changes or
>>> something.
>>>
>>> Maxim Gekk
>>>
>>> Software Engineer
>>>
>>> Databricks, Inc.
>>>
>>
>> CONFIDENTIALITY NOTICE: This electronic communication and any files
>> transmitted with it are confidential, privileged and intended solely for
>> the use of the individual or entity to whom they are addressed. If you are
>> not the intended recipient, you are hereby notified that any disclosure,
>> copying, distribution (electronic or otherwise) or forwarding of, or the
>> taking of any action in reliance on the contents of this transmission is
>> strictly prohibited. Please notify the sender immediately by e-mail if you
>> have received this email by mistake and delete this email from your system.
>>
>> Is it necessary to print this email? If you care about the environment
>> like we do, please refrain from printing emails. It helps to keep the
>> environment forested and litter-free.
>
>

-- 
Bjørn Jørgensen
Vestre Aspehaug 4, 6010 Ålesund
Norge

+47 480 94 297

Re: The draft of the Spark 3.3.0 release notes

[Dongjoon Hyun]

You are right.

After SPARK-36837, we tried to ship Apache Spark 3.3.0 with Apache Kafka
3.1.1 via the following PR.

https://github.com/apache/spark/pull/36135
[WIP][SPARK-38850][BUILD] Upgrade Kafka to 3.1.1

However, the final decision was to revert it from `branch-3.3` and move
directly to Apache Kafka 3.2.0 at `master` branch. We need to remove it
from the 3.3.0 release note.


On Fri, Jun 3, 2022 at 9:54 AM Koert Kuipers  wrote:

> i thought SPARK-36837 didnt make it in? i see it in notes
>
> On Fri, Jun 3, 2022 at 4:31 AM Maxim Gekk
>  wrote:
>
>> Hi All,
>>
>> I am preparing the release notes of Spark 3.3.0. Here is a draft document:
>>
>> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>>
>> Please take a look and let me know if I missed any major changes or
>> something.
>>
>> Maxim Gekk
>>
>> Software Engineer
>>
>> Databricks, Inc.
>>
>
> CONFIDENTIALITY NOTICE: This electronic communication and any files
> transmitted with it are confidential, privileged and intended solely for
> the use of the individual or entity to whom they are addressed. If you are
> not the intended recipient, you are hereby notified that any disclosure,
> copying, distribution (electronic or otherwise) or forwarding of, or the
> taking of any action in reliance on the contents of this transmission is
> strictly prohibited. Please notify the sender immediately by e-mail if you
> have received this email by mistake and delete this email from your system.
>
> Is it necessary to print this email? If you care about the environment
> like we do, please refrain from printing emails. It helps to keep the
> environment forested and litter-free.

Re: The draft of the Spark 3.3.0 release notes

[Koert Kuipers]

i thought SPARK-36837 didnt make it in? i see it in notes

On Fri, Jun 3, 2022 at 4:31 AM Maxim Gekk 
wrote:

> Hi All,
>
> I am preparing the release notes of Spark 3.3.0. Here is a draft document:
>
> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>
> Please take a look and let me know if I missed any major changes or
> something.
>
> Maxim Gekk
>
> Software Engineer
>
> Databricks, Inc.
>

-- 
CONFIDENTIALITY NOTICE: This electronic communication and any files 
transmitted with it are confidential, privileged and intended solely for 
the use of the individual or entity to whom they are addressed. If you are 
not the intended recipient, you are hereby notified that any disclosure, 
copying, distribution (electronic or otherwise) or forwarding of, or the 
taking of any action in reliance on the contents of this transmission is 
strictly prohibited. Please notify the sender immediately by e-mail if you 
have received this email by mistake and delete this email from your system.


Is it necessary to print this email? If you care about the environment 
like we do, please refrain from printing emails. It helps to keep the 
environment forested and litter-free.

Re: The draft of the Spark 3.3.0 release notes

[Adam Binford]

It's more of a missing feature than a bug, but I guess it's all subjective.

Adam

On Fri, Jun 3, 2022 at 10:24 AM Maxim Gekk 
wrote:

> Hi Adam,
>
> The release notes focus on new features and user-facing improvements
> mostly. SPARK-37618  seems
> like a bug fix, that's why I didn't put it in the doc.
>
> Maxim Gekk
>
> Software Engineer
>
> Databricks, Inc.
>
>
> On Fri, Jun 3, 2022 at 2:20 PM Adam Binford  wrote:
>
>> I don't think I see https://issues.apache.org/jira/browse/SPARK-37618
>> which got merged post branch cut.
>>
>> Adam
>>
>> On Fri, Jun 3, 2022 at 4:25 AM Maxim Gekk
>>  wrote:
>>
>>> Hi All,
>>>
>>> I am preparing the release notes of Spark 3.3.0. Here is a draft
>>> document:
>>>
>>> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>>>
>>> Please take a look and let me know if I missed any major changes or
>>> something.
>>>
>>> Maxim Gekk
>>>
>>> Software Engineer
>>>
>>> Databricks, Inc.
>>>
>>
>>
>> --
>> Adam Binford
>>
>

-- 
Adam Binford

Re: The draft of the Spark 3.3.0 release notes

[Maxim Gekk]

Hi Adam,

The release notes focus on new features and user-facing improvements
mostly. SPARK-37618  seems
like a bug fix, that's why I didn't put it in the doc.

Maxim Gekk

Software Engineer

Databricks, Inc.


On Fri, Jun 3, 2022 at 2:20 PM Adam Binford  wrote:

> I don't think I see https://issues.apache.org/jira/browse/SPARK-37618
> which got merged post branch cut.
>
> Adam
>
> On Fri, Jun 3, 2022 at 4:25 AM Maxim Gekk
>  wrote:
>
>> Hi All,
>>
>> I am preparing the release notes of Spark 3.3.0. Here is a draft document:
>>
>> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>>
>> Please take a look and let me know if I missed any major changes or
>> something.
>>
>> Maxim Gekk
>>
>> Software Engineer
>>
>> Databricks, Inc.
>>
>
>
> --
> Adam Binford
>

Re: The draft of the Spark 3.3.0 release notes

[Adam Binford]

I don't think I see https://issues.apache.org/jira/browse/SPARK-37618 which
got merged post branch cut.

Adam

On Fri, Jun 3, 2022 at 4:25 AM Maxim Gekk 
wrote:

> Hi All,
>
> I am preparing the release notes of Spark 3.3.0. Here is a draft document:
>
> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>
> Please take a look and let me know if I missed any major changes or
> something.
>
> Maxim Gekk
>
> Software Engineer
>
> Databricks, Inc.
>


-- 
Adam Binford

The draft of the Spark 3.3.0 release notes

[Maxim Gekk]

Hi All,

I am preparing the release notes of Spark 3.3.0. Here is a draft document:
https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing

Please take a look and let me know if I missed any major changes or
something.

Maxim Gekk

Software Engineer

Databricks, Inc.