Re: [VOTE] SPIP: Structured Streaming - Arbitrary State API v2

[Wenchen Fan]

+1

On Thu, Jan 11, 2024 at 9:32 AM L. C. Hsieh  wrote:

> +1
>
> On Wed, Jan 10, 2024 at 9:06 AM Bhuwan Sahni
>  wrote:
>
>> +1. This is a good addition.
>>
>> 
>> *Bhuwan Sahni*
>> Staff Software Engineer
>>
>> bhuwan.sa...@databricks.com
>> 500 108th Ave. NE
>> Bellevue, WA 98004
>> USA
>>
>>
>> On Wed, Jan 10, 2024 at 9:00 AM Burak Yavuz  wrote:
>>
>>> +1. Excited to see more stateful workloads with Structured Streaming!
>>>
>>>
>>> Best,
>>> Burak
>>>
>>> On Wed, Jan 10, 2024 at 8:21 AM Praveen Gattu
>>>  wrote:
>>>
 +1. This brings Structured Streaming a good solution for
 customers wanting to build stateful stream processing applications.

 On Wed, Jan 10, 2024 at 7:30 AM Bartosz Konieczny <
 bartkoniec...@gmail.com> wrote:

> +1 :)
>
> On Wed, Jan 10, 2024 at 9:57 AM Shixiong Zhu 
> wrote:
>
>> +1 (binding)
>>
>> Best Regards,
>> Shixiong Zhu
>>
>>
>> On Tue, Jan 9, 2024 at 6:47 PM 刘唯  wrote:
>>
>>> This is a good addition! +1
>>>
>>> Raghu Angadi  于2024年1月9日周二
>>> 13:17写道：
>>>
 +1. This is a major improvement to the state API.

 Raghu.

 On Tue, Jan 9, 2024 at 1:42 AM Mich Talebzadeh <
 mich.talebza...@gmail.com> wrote:

> +1 for me as well
>
>
> Mich Talebzadeh,
> Dad | Technologist | Solutions Architect | Engineer
> London
> United Kingdom
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility
> for any loss, damage or destruction of data or any other property 
> which may
> arise from relying on this email's technical content is explicitly
> disclaimed. The author will in no case be liable for any monetary 
> damages
> arising from such loss, damage or destruction.
>
>
>
>
> On Tue, 9 Jan 2024 at 03:24, Anish Shrigondekar
>  wrote:
>
>> Thanks Jungtaek for creating the Vote thread.
>>
>> +1 (non-binding) from my side too.
>>
>> Thanks,
>> Anish
>>
>> On Tue, Jan 9, 2024 at 6:09 AM Jungtaek Lim <
>> kabhwan.opensou...@gmail.com> wrote:
>>
>>> Starting with my +1 (non-binding). Thanks!
>>>
>>> On Tue, Jan 9, 2024 at 9:37 AM Jungtaek Lim <
>>> kabhwan.opensou...@gmail.com> wrote:
>>>
 Hi all,

 I'd like to start the vote for SPIP: Structured Streaming -
 Arbitrary State API v2.

 References:

- JIRA ticket

- SPIP doc

 
- Discussion thread

 

 Please vote on the SPIP for the next 72 hours:

 [ ] +1: Accept the proposal as an official SPIP
 [ ] +0
 [ ] -1: I don’t think this is a good idea because …

 Thanks!
 Jungtaek Lim (HeartSaVioR)

>>>
>
> --
> Bartosz Konieczny
> freelance data engineer
> https://www.waitingforcode.com
> https://github.com/bartosz25/
> https://twitter.com/waitingforcode
>
>

Spark Kafka Rack Aware Consumer

[Schwager, Randall]

Hello Spark Devs!

Has there been discussion around adding the ability to dynamically set the 
‘client.rack’ Kafka parameter at the executor?
The Kafka SQL connector code on master doesn’t seem to support this feature. 
One can easily set the ‘client.rack’ parameter at the driver, but that just 
sets all executors to have the same rack. It seems that if we want each 
executor to set the correct rack, each executor will have to produce the 
setting dynamically on start-up.

Would this be a good area to consider contributing new functionality?

Sincerely,

Randall

E-MAIL CONFIDENTIALITY NOTICE: 
The contents of this e-mail message and any attachments are intended solely for 
the addressee(s) and may contain confidential and/or legally privileged 
information. If you are not the intended recipient of this message or if this 
message has been addressed to you in error, please immediately alert the sender 
by reply e-mail and then delete this message and any attachments. If you are 
not the intended recipient, you are notified that any use, dissemination, 
distribution, copying, or storage of this message or any attachment is strictly 
prohibited.

Install Ruby 3 to build the docs

[Nicholas Chammas]

Just a quick heads up that, while Ruby 2.7 will continue to work, you should 
plan to install Ruby 3 in the near future in order to build the docs. (I 
recommend using rbenv  to manage multiple Ruby 
versions.)

Ruby 2 reached EOL in March 2023 
. We will be 
unable to upgrade some of our Ruby dependencies to their latest versions until 
we are using Ruby 3.

This is not a problem today but will likely become a problem in the near future.

For more details, please refer to this pull request 
.

Best,
Nick

Re: [VOTE] SPIP: Structured Streaming - Arbitrary State API v2

[L. C. Hsieh]

+1

On Wed, Jan 10, 2024 at 9:06 AM Bhuwan Sahni
 wrote:

> +1. This is a good addition.
>
> 
> *Bhuwan Sahni*
> Staff Software Engineer
>
> bhuwan.sa...@databricks.com
> 500 108th Ave. NE
> Bellevue, WA 98004
> USA
>
>
> On Wed, Jan 10, 2024 at 9:00 AM Burak Yavuz  wrote:
>
>> +1. Excited to see more stateful workloads with Structured Streaming!
>>
>>
>> Best,
>> Burak
>>
>> On Wed, Jan 10, 2024 at 8:21 AM Praveen Gattu
>>  wrote:
>>
>>> +1. This brings Structured Streaming a good solution for
>>> customers wanting to build stateful stream processing applications.
>>>
>>> On Wed, Jan 10, 2024 at 7:30 AM Bartosz Konieczny <
>>> bartkoniec...@gmail.com> wrote:
>>>
 +1 :)

 On Wed, Jan 10, 2024 at 9:57 AM Shixiong Zhu  wrote:

> +1 (binding)
>
> Best Regards,
> Shixiong Zhu
>
>
> On Tue, Jan 9, 2024 at 6:47 PM 刘唯  wrote:
>
>> This is a good addition! +1
>>
>> Raghu Angadi  于2024年1月9日周二
>> 13:17写道：
>>
>>> +1. This is a major improvement to the state API.
>>>
>>> Raghu.
>>>
>>> On Tue, Jan 9, 2024 at 1:42 AM Mich Talebzadeh <
>>> mich.talebza...@gmail.com> wrote:
>>>
 +1 for me as well


 Mich Talebzadeh,
 Dad | Technologist | Solutions Architect | Engineer
 London
 United Kingdom


view my Linkedin profile
 


  https://en.everybodywiki.com/Mich_Talebzadeh



 *Disclaimer:* Use it at your own risk. Any and all responsibility
 for any loss, damage or destruction of data or any other property 
 which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in no case be liable for any monetary 
 damages
 arising from such loss, damage or destruction.




 On Tue, 9 Jan 2024 at 03:24, Anish Shrigondekar
  wrote:

> Thanks Jungtaek for creating the Vote thread.
>
> +1 (non-binding) from my side too.
>
> Thanks,
> Anish
>
> On Tue, Jan 9, 2024 at 6:09 AM Jungtaek Lim <
> kabhwan.opensou...@gmail.com> wrote:
>
>> Starting with my +1 (non-binding). Thanks!
>>
>> On Tue, Jan 9, 2024 at 9:37 AM Jungtaek Lim <
>> kabhwan.opensou...@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I'd like to start the vote for SPIP: Structured Streaming -
>>> Arbitrary State API v2.
>>>
>>> References:
>>>
>>>- JIRA ticket
>>>
>>>- SPIP doc
>>>
>>> 
>>>- Discussion thread
>>>
>>> 
>>>
>>> Please vote on the SPIP for the next 72 hours:
>>>
>>> [ ] +1: Accept the proposal as an official SPIP
>>> [ ] +0
>>> [ ] -1: I don’t think this is a good idea because …
>>>
>>> Thanks!
>>> Jungtaek Lim (HeartSaVioR)
>>>
>>

 --
 Bartosz Konieczny
 freelance data engineer
 https://www.waitingforcode.com
 https://github.com/bartosz25/
 https://twitter.com/waitingforcode

Re: [VOTE] SPIP: Structured Streaming - Arbitrary State API v2

[Bhuwan Sahni]

+1. This is a good addition.


*Bhuwan Sahni*
Staff Software Engineer

bhuwan.sa...@databricks.com
500 108th Ave. NE
Bellevue, WA 98004
USA


On Wed, Jan 10, 2024 at 9:00 AM Burak Yavuz  wrote:

> +1. Excited to see more stateful workloads with Structured Streaming!
>
>
> Best,
> Burak
>
> On Wed, Jan 10, 2024 at 8:21 AM Praveen Gattu
>  wrote:
>
>> +1. This brings Structured Streaming a good solution for
>> customers wanting to build stateful stream processing applications.
>>
>> On Wed, Jan 10, 2024 at 7:30 AM Bartosz Konieczny <
>> bartkoniec...@gmail.com> wrote:
>>
>>> +1 :)
>>>
>>> On Wed, Jan 10, 2024 at 9:57 AM Shixiong Zhu  wrote:
>>>
 +1 (binding)

 Best Regards,
 Shixiong Zhu


 On Tue, Jan 9, 2024 at 6:47 PM 刘唯  wrote:

> This is a good addition! +1
>
> Raghu Angadi  于2024年1月9日周二
> 13:17写道：
>
>> +1. This is a major improvement to the state API.
>>
>> Raghu.
>>
>> On Tue, Jan 9, 2024 at 1:42 AM Mich Talebzadeh <
>> mich.talebza...@gmail.com> wrote:
>>
>>> +1 for me as well
>>>
>>>
>>> Mich Talebzadeh,
>>> Dad | Technologist | Solutions Architect | Engineer
>>> London
>>> United Kingdom
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>> for any loss, damage or destruction of data or any other property which 
>>> may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary 
>>> damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 9 Jan 2024 at 03:24, Anish Shrigondekar
>>>  wrote:
>>>
 Thanks Jungtaek for creating the Vote thread.

 +1 (non-binding) from my side too.

 Thanks,
 Anish

 On Tue, Jan 9, 2024 at 6:09 AM Jungtaek Lim <
 kabhwan.opensou...@gmail.com> wrote:

> Starting with my +1 (non-binding). Thanks!
>
> On Tue, Jan 9, 2024 at 9:37 AM Jungtaek Lim <
> kabhwan.opensou...@gmail.com> wrote:
>
>> Hi all,
>>
>> I'd like to start the vote for SPIP: Structured Streaming -
>> Arbitrary State API v2.
>>
>> References:
>>
>>- JIRA ticket
>>
>>- SPIP doc
>>
>> 
>>- Discussion thread
>>
>>
>> Please vote on the SPIP for the next 72 hours:
>>
>> [ ] +1: Accept the proposal as an official SPIP
>> [ ] +0
>> [ ] -1: I don’t think this is a good idea because …
>>
>> Thanks!
>> Jungtaek Lim (HeartSaVioR)
>>
>
>>>
>>> --
>>> Bartosz Konieczny
>>> freelance data engineer
>>> https://www.waitingforcode.com
>>> https://github.com/bartosz25/
>>> https://twitter.com/waitingforcode
>>>
>>>

Re: [VOTE] SPIP: Structured Streaming - Arbitrary State API v2

[Burak Yavuz]

+1. Excited to see more stateful workloads with Structured Streaming!


Best,
Burak

On Wed, Jan 10, 2024 at 8:21 AM Praveen Gattu
 wrote:

> +1. This brings Structured Streaming a good solution for customers wanting
> to build stateful stream processing applications.
>
> On Wed, Jan 10, 2024 at 7:30 AM Bartosz Konieczny 
> wrote:
>
>> +1 :)
>>
>> On Wed, Jan 10, 2024 at 9:57 AM Shixiong Zhu  wrote:
>>
>>> +1 (binding)
>>>
>>> Best Regards,
>>> Shixiong Zhu
>>>
>>>
>>> On Tue, Jan 9, 2024 at 6:47 PM 刘唯  wrote:
>>>
 This is a good addition! +1

 Raghu Angadi  于2024年1月9日周二
 13:17写道：

> +1. This is a major improvement to the state API.
>
> Raghu.
>
> On Tue, Jan 9, 2024 at 1:42 AM Mich Talebzadeh <
> mich.talebza...@gmail.com> wrote:
>
>> +1 for me as well
>>
>>
>> Mich Talebzadeh,
>> Dad | Technologist | Solutions Architect | Engineer
>> London
>> United Kingdom
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>> for any loss, damage or destruction of data or any other property which 
>> may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Tue, 9 Jan 2024 at 03:24, Anish Shrigondekar
>>  wrote:
>>
>>> Thanks Jungtaek for creating the Vote thread.
>>>
>>> +1 (non-binding) from my side too.
>>>
>>> Thanks,
>>> Anish
>>>
>>> On Tue, Jan 9, 2024 at 6:09 AM Jungtaek Lim <
>>> kabhwan.opensou...@gmail.com> wrote:
>>>
 Starting with my +1 (non-binding). Thanks!

 On Tue, Jan 9, 2024 at 9:37 AM Jungtaek Lim <
 kabhwan.opensou...@gmail.com> wrote:

> Hi all,
>
> I'd like to start the vote for SPIP: Structured Streaming -
> Arbitrary State API v2.
>
> References:
>
>- JIRA ticket
>
>- SPIP doc
>
> 
>- Discussion thread
>
>
> Please vote on the SPIP for the next 72 hours:
>
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because …
>
> Thanks!
> Jungtaek Lim (HeartSaVioR)
>

>>
>> --
>> Bartosz Konieczny
>> freelance data engineer
>> https://www.waitingforcode.com
>> https://github.com/bartosz25/
>> https://twitter.com/waitingforcode
>>
>>

Re: [VOTE] SPIP: Structured Streaming - Arbitrary State API v2

[Praveen Gattu]

+1. This brings Structured Streaming a good solution for customers wanting
to build stateful stream processing applications.

On Wed, Jan 10, 2024 at 7:30 AM Bartosz Konieczny 
wrote:

> +1 :)
>
> On Wed, Jan 10, 2024 at 9:57 AM Shixiong Zhu  wrote:
>
>> +1 (binding)
>>
>> Best Regards,
>> Shixiong Zhu
>>
>>
>> On Tue, Jan 9, 2024 at 6:47 PM 刘唯  wrote:
>>
>>> This is a good addition! +1
>>>
>>> Raghu Angadi  于2024年1月9日周二 13:17写道：
>>>
 +1. This is a major improvement to the state API.

 Raghu.

 On Tue, Jan 9, 2024 at 1:42 AM Mich Talebzadeh <
 mich.talebza...@gmail.com> wrote:

> +1 for me as well
>
>
> Mich Talebzadeh,
> Dad | Technologist | Solutions Architect | Engineer
> London
> United Kingdom
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for
> any loss, damage or destruction of data or any other property which may
> arise from relying on this email's technical content is explicitly
> disclaimed. The author will in no case be liable for any monetary damages
> arising from such loss, damage or destruction.
>
>
>
>
> On Tue, 9 Jan 2024 at 03:24, Anish Shrigondekar
>  wrote:
>
>> Thanks Jungtaek for creating the Vote thread.
>>
>> +1 (non-binding) from my side too.
>>
>> Thanks,
>> Anish
>>
>> On Tue, Jan 9, 2024 at 6:09 AM Jungtaek Lim <
>> kabhwan.opensou...@gmail.com> wrote:
>>
>>> Starting with my +1 (non-binding). Thanks!
>>>
>>> On Tue, Jan 9, 2024 at 9:37 AM Jungtaek Lim <
>>> kabhwan.opensou...@gmail.com> wrote:
>>>
 Hi all,

 I'd like to start the vote for SPIP: Structured Streaming -
 Arbitrary State API v2.

 References:

- JIRA ticket

- SPIP doc

 
- Discussion thread


 Please vote on the SPIP for the next 72 hours:

 [ ] +1: Accept the proposal as an official SPIP
 [ ] +0
 [ ] -1: I don’t think this is a good idea because …

 Thanks!
 Jungtaek Lim (HeartSaVioR)

>>>
>
> --
> Bartosz Konieczny
> freelance data engineer
> https://www.waitingforcode.com
> https://github.com/bartosz25/
> https://twitter.com/waitingforcode
>
>

Re: [VOTE] SPIP: Structured Streaming - Arbitrary State API v2

[Bartosz Konieczny]

+1 :)

On Wed, Jan 10, 2024 at 9:57 AM Shixiong Zhu  wrote:

> +1 (binding)
>
> Best Regards,
> Shixiong Zhu
>
>
> On Tue, Jan 9, 2024 at 6:47 PM 刘唯  wrote:
>
>> This is a good addition! +1
>>
>> Raghu Angadi  于2024年1月9日周二 13:17写道：
>>
>>> +1. This is a major improvement to the state API.
>>>
>>> Raghu.
>>>
>>> On Tue, Jan 9, 2024 at 1:42 AM Mich Talebzadeh <
>>> mich.talebza...@gmail.com> wrote:
>>>
 +1 for me as well


 Mich Talebzadeh,
 Dad | Technologist | Solutions Architect | Engineer
 London
 United Kingdom


view my Linkedin profile
 


  https://en.everybodywiki.com/Mich_Talebzadeh



 *Disclaimer:* Use it at your own risk. Any and all responsibility for
 any loss, damage or destruction of data or any other property which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in no case be liable for any monetary damages
 arising from such loss, damage or destruction.




 On Tue, 9 Jan 2024 at 03:24, Anish Shrigondekar
  wrote:

> Thanks Jungtaek for creating the Vote thread.
>
> +1 (non-binding) from my side too.
>
> Thanks,
> Anish
>
> On Tue, Jan 9, 2024 at 6:09 AM Jungtaek Lim <
> kabhwan.opensou...@gmail.com> wrote:
>
>> Starting with my +1 (non-binding). Thanks!
>>
>> On Tue, Jan 9, 2024 at 9:37 AM Jungtaek Lim <
>> kabhwan.opensou...@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I'd like to start the vote for SPIP: Structured Streaming -
>>> Arbitrary State API v2.
>>>
>>> References:
>>>
>>>- JIRA ticket 
>>>- SPIP doc
>>>
>>> 
>>>- Discussion thread
>>>
>>>
>>> Please vote on the SPIP for the next 72 hours:
>>>
>>> [ ] +1: Accept the proposal as an official SPIP
>>> [ ] +0
>>> [ ] -1: I don’t think this is a good idea because …
>>>
>>> Thanks!
>>> Jungtaek Lim (HeartSaVioR)
>>>
>>

-- 
Bartosz Konieczny
freelance data engineer
https://www.waitingforcode.com
https://github.com/bartosz25/
https://twitter.com/waitingforcode