+1 (non-binding). Interesting problem. Looking forward to seeing the project mature and evolve.
On Tue, Jan 13, 2026 at 7:05 PM Jungtaek Lim <[email protected]> wrote: > +1 (non-binding) > > I can't imagine the coverage of UDF for this SPIP at this point, but I > think this is the way to go if things work out, so definitely good to > explore. > > On Wed, Jan 14, 2026 at 12:00 PM Felix Cheung <[email protected]> > wrote: > >> +1 worth a shot, it would be great to have some improvement in this space >> >> >> ------------------------------ >> *From:* Hyukjin Kwon <[email protected]> >> *Sent:* Tuesday, January 13, 2026 6:50:19 PM >> *To:* [email protected] <[email protected]> >> *Subject:* Re: [VOTE] SPIP: Improving Spark SQL UDFs with Transpilation >> >> I'm +1 on this as long as we can isolate the implementation and turn it >> off by default till it's proven that it works and it's ready. >> >> I am not fully confident that it will work but I think it should be worth >> trying. >> >> On Tue, Jan 13, 2026 at 6:29 AM huaxin gao <[email protected]> >> wrote: >> >> +1 >> >> On Mon, Jan 12, 2026 at 12:24 PM Holden Karau <[email protected]> >> wrote: >> >> Hi Folks, >> >> Discussion on the SPIP Spark SQL UDF transpilation >> <https://lists.apache.org/thread/xj8qfqvo5f9o188984mwh2kcg0fnqs9c> seems >> to have settled, so I'm now bringing it for a vote. The normal requirements >> is that an SPIP vote is open for at least 72 hours, but given that some are >> just returning from the winter holidays, I plan to leave this vote open >> until Sunday Jan 18th. >> >> From the original discussion: >> >> It's been a few years since we last looked at transpilation, and with the >> growth of Pandas on Spark I think it's time we revisit it. I've got a JIRA >> filed <https://issues.apache.org/jira/browse/SPARK-54783> some rough >> proof of concept code <https://github.com/apache/spark/pull/53547> (I >> think doing the transpilation Python side instead of Scala side makes more >> sense, but was interesting to play with), and of course everyones favourite >> a design doc. >> <https://docs.google.com/document/d/1cHc6tiR4yO3hppTzrK1F1w9RwyEPMvaeEuL2ub2LURg/edit?usp=sharing> >> (I >> also have a collection of YouTube streams playing with the idea >> <https://www.youtube.com/@HoldenKarau/streams> if anyone wants to follow >> along on that journey). >> >> Cheers, >> >> Holden :) >> >> -- >> Twitter: https://twitter.com/holdenkarau >> Fight Health Insurance: https://www.fighthealthinsurance.com/ >> <https://www.fighthealthinsurance.com/?q=hk_email> >> Books (Learning Spark, High Performance Spark, etc.): >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >> Pronouns: she/her >> >> -- Regards Rishab Joshi
