+1 Regards, Vaquar Khan
On Wed, Jan 21, 2026, 9:14 PM Anand Chinnakannan <[email protected]> wrote: > +1, Interesting problem to solve in python. Catalyst udf Optimization. > > Thanks > Anand > > On Wed, Jan 21, 2026, 5:15 PM Holden Karau <[email protected]> wrote: > >> This vote passes. I left it open for a bit longer because Monday was a >> holiday for some folks. >> >> The votes are: >> >> +1s: >> Huaxin Gao * >> Hyukjin Kwon* >> Felix Cheung* >> Jungtaek Lim >> Rishab Joshi >> Chao Sun* >> Liu Cao >> Wenchen Fan* >> Kent Yao* >> Kousuke Saruta* >> Peter Toth >> Dongjoon Hyun* >> John Zhuge >> >> 0s: >> >> -1s: >> >> None >> >> * Binding >> >> All of the work will be behind a feature flag so that if after the >> experiment it turns out not to be a success it can be removed later. >> >> Thank you everyone who voted, read the document, left feedback, etc. I'll >> start filing some smaller JIRAs to break the work down into smaller pieces. >> >> On Wed, Jan 14, 2026 at 5:54 PM John Zhuge <[email protected]> wrote: >> >>> +1 (non-binding) Promising. Let's maintain focus while recognizing the >>> potential architectural shift. >>> >>> On Wed, Jan 14, 2026 at 3:44 PM Dongjoon Hyun <[email protected]> >>> wrote: >>> >>>> +1 >>>> >>>> On 2026/01/14 11:24:50 Peter Toth wrote: >>>> > +1 (non-binding) >>>> > >>>> > On Wed, Jan 14, 2026 at 7:25 AM Kousuke Saruta <[email protected]> >>>> wrote: >>>> > >>>> > > +1 >>>> > > >>>> > > 2026年1月14日(水) 15:03 Kent Yao <[email protected]>: >>>> > > >>>> > >> +1 (binding) >>>> > >> >>>> > >> Wenchen Fan <[email protected]> 于2026年1月14日周三 13:06写道： >>>> > >> >>>> > >>> +1 (binding) >>>> > >>> >>>> > >>> On Wed, Jan 14, 2026 at 11:54 AM Liu Cao <[email protected]> >>>> wrote: >>>> > >>> >>>> > >>>> +1 (non-binding) >>>> > >>>> >>>> > >>>> The initial scope seems reasonable; very interested in the long >>>> term >>>> > >>>> aspect in accelerator integrations. >>>> > >>>> >>>> > >>>> On Tue, Jan 13, 2026 at 19:18 Rishab Joshi < >>>> [email protected]> >>>> > >>>> wrote: >>>> > >>>> >>>> > >>>>> +1 (non-binding). >>>> > >>>>> Interesting problem. Looking forward to seeing the project >>>> mature and >>>> > >>>>> evolve. >>>> > >>>>> >>>> > >>>>> On Tue, Jan 13, 2026 at 7:05 PM Jungtaek Lim < >>>> > >>>>> [email protected]> wrote: >>>> > >>>>> >>>> > >>>>>> +1 (non-binding) >>>> > >>>>>> >>>> > >>>>>> I can't imagine the coverage of UDF for this SPIP at this >>>> point, but >>>> > >>>>>> I think this is the way to go if things work out, so >>>> definitely good to >>>> > >>>>>> explore. >>>> > >>>>>> >>>> > >>>>>> On Wed, Jan 14, 2026 at 12:00 PM Felix Cheung < >>>> > >>>>>> [email protected]> wrote: >>>> > >>>>>> >>>> > >>>>>>> +1 worth a shot, it would be great to have some improvement >>>> in this >>>> > >>>>>>> space >>>> > >>>>>>> >>>> > >>>>>>> >>>> > >>>>>>> ------------------------------ >>>> > >>>>>>> *From:* Hyukjin Kwon <[email protected]> >>>> > >>>>>>> *Sent:* Tuesday, January 13, 2026 6:50:19 PM >>>> > >>>>>>> *To:* [email protected] <[email protected]> >>>> > >>>>>>> *Subject:* Re: [VOTE] SPIP: Improving Spark SQL UDFs with >>>> > >>>>>>> Transpilation >>>> > >>>>>>> >>>> > >>>>>>> I'm +1 on this as long as we can isolate the implementation >>>> and turn >>>> > >>>>>>> it off by default till it's proven that it works and it's >>>> ready. >>>> > >>>>>>> >>>> > >>>>>>> I am not fully confident that it will work but I think it >>>> should be >>>> > >>>>>>> worth trying. >>>> > >>>>>>> >>>> > >>>>>>> On Tue, Jan 13, 2026 at 6:29 AM huaxin gao < >>>> [email protected]> >>>> > >>>>>>> wrote: >>>> > >>>>>>> >>>> > >>>>>>> +1 >>>> > >>>>>>> >>>> > >>>>>>> On Mon, Jan 12, 2026 at 12:24 PM Holden Karau < >>>> > >>>>>>> [email protected]> wrote: >>>> > >>>>>>> >>>> > >>>>>>> Hi Folks, >>>> > >>>>>>> >>>> > >>>>>>> Discussion on the SPIP Spark SQL UDF transpilation >>>> > >>>>>>> < >>>> https://lists.apache.org/thread/xj8qfqvo5f9o188984mwh2kcg0fnqs9c> >>>> > >>>>>>> seems to have settled, so I'm now bringing it for a vote. The >>>> normal >>>> > >>>>>>> requirements is that an SPIP vote is open for at least 72 >>>> hours, but given >>>> > >>>>>>> that some are just returning from the winter holidays, I plan >>>> to leave this >>>> > >>>>>>> vote open until Sunday Jan 18th. >>>> > >>>>>>> >>>> > >>>>>>> From the original discussion: >>>> > >>>>>>> >>>> > >>>>>>> It's been a few years since we last looked at transpilation, >>>> and >>>> > >>>>>>> with the growth of Pandas on Spark I think it's time we >>>> revisit it. I've >>>> > >>>>>>> got a JIRA filed < >>>> https://issues.apache.org/jira/browse/SPARK-54783> >>>> > >>>>>>> some rough proof of concept code >>>> > >>>>>>> <https://github.com/apache/spark/pull/53547> (I think doing >>>> the >>>> > >>>>>>> transpilation Python side instead of Scala side makes more >>>> sense, but was >>>> > >>>>>>> interesting to play with), and of course everyones favourite >>>> a >>>> > >>>>>>> design doc. >>>> > >>>>>>> < >>>> https://docs.google.com/document/d/1cHc6tiR4yO3hppTzrK1F1w9RwyEPMvaeEuL2ub2LURg/edit?usp=sharing> >>>> (I >>>> > >>>>>>> also have a collection of YouTube streams playing with the >>>> idea >>>> > >>>>>>> <https://www.youtube.com/@HoldenKarau/streams> if anyone >>>> wants to >>>> > >>>>>>> follow along on that journey). >>>> > >>>>>>> >>>> > >>>>>>> Cheers, >>>> > >>>>>>> >>>> > >>>>>>> Holden :) >>>> > >>>>>>> >>>> > >>>>>>> -- >>>> > >>>>>>> Twitter: https://twitter.com/holdenkarau >>>> > >>>>>>> Fight Health Insurance: https://www.fighthealthinsurance.com/ >>>> > >>>>>>> <https://www.fighthealthinsurance.com/?q=hk_email> >>>> > >>>>>>> Books (Learning Spark, High Performance Spark, etc.): >>>> > >>>>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>>> > >>>>>>> YouTube Live Streams: >>>> https://www.youtube.com/user/holdenkarau >>>> > >>>>>>> Pronouns: she/her >>>> > >>>>>>> >>>> > >>>>>>> >>>> > >>>>> >>>> > >>>>> -- >>>> > >>>>> Regards >>>> > >>>>> Rishab Joshi >>>> > >>>>> >>>> > >>>> -- >>>> > >>>> >>>> > >>>> Liu Cao >>>> > >>>> >>>> > >>>> >>>> > >>>> >>>> > >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe e-mail: [email protected] >>>> >>>> >>> >>> -- >>> John Zhuge >>> >> >> >> -- >> Twitter: https://twitter.com/holdenkarau >> Fight Health Insurance: https://www.fighthealthinsurance.com/ >> <https://www.fighthealthinsurance.com/?q=hk_email> >> Books (Learning Spark, High Performance Spark, etc.): >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >> Pronouns: she/her >> >
