Re: I'm going to be out starting Nov 5th

[rahul kumar]

Come back strong and healthy Holden!
On Sun, Nov 1, 2020 at 9:01 AM Holden Karau  wrote:

> Thanks everyone, these kind words mean a lot :) I hope everyone is doing
> as well as possible with our very challenging 2020. Community is what keeps
> me going in open source :)
>
> On Sun, Nov 1, 2020 at 7:58 AM Xiao Li  wrote:
>
>> Take care, Holden!
>>
>> Bests,
>>
>> Xiao
>>
>> On Sat, Oct 31, 2020 at 9:53 PM 郑瑞峰  wrote:
>>
>>> Take care, Holden! Best wishes!
>>>
>>>
>>> -- 原始邮件 --
>>> *发件人:* "Hyukjin Kwon" ;
>>> *发送时间:* 2020年11月1日(星期天) 上午10:24
>>> *收件人:* "Denny Lee";
>>> *抄送:* "Dongjoon Hyun";"Holden Karau"<
>>> hol...@pigscanfly.ca>;"dev";
>>> *主题:* Re: I'm going to be out starting Nov 5th
>>>
>>> Oh, take care Holden!
>>>
>>> On Sun, 1 Nov 2020, 03:04 Denny Lee,  wrote:
>>>
 Best wishes Holden! :)

 On Sat, Oct 31, 2020 at 11:00 Dongjoon Hyun 
 wrote:

> Take care, Holden! I believe everything goes well.
>
> Bests,
> Dongjoon.
>
> On Sat, Oct 31, 2020 at 10:24 AM Reynold Xin 
> wrote:
>
>> Take care Holden and best of luck with everything!
>>
>>
>> On Sat, Oct 31 2020 at 10:21 AM, Holden Karau 
>> wrote:
>>
>>> Hi Folks,
>>>
>>> Just a heads up so folks working on decommissioning or other areas
>>> I've been active in don't block on me, I'm going to be out for at least 
>>> a
>>> week and possibly more starting on November 5th. If there is anything 
>>> that
>>> folks want me to review before then please let me know and I'll make the
>>> time for it. If you are curious I've got more details at
>>> http://blog.holdenkarau.com/2020/10/taking-break-surgery.html
>>>
>>> Happy Sparking Everyone,
>>>
>>> Holden :)
>>>
>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>> Books (Learning Spark, High Performance Spark, etc.):
>>> https://amzn.to/2MaRAG9  
>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>
>>
>>
>> --
>>
>>
>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9  
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>

Exposing custom functions defined over a scala Dataframe to Python.

[rahul kumar]

Hello Devs,

I have been working on a custom spark connector. This connector is
implemented in Scala. So, far I have defined a custom join-kinda. function
over dataframe. Now I want to expose it in pyspark as well. I did some
investigation in the spark project and it appears to me that all attributes
are exposed through python module. So, rebuilding spark library just to
expose this function would be impractical. So my question is, is there any
other way I could expose it in python, so that folks can use it as native
functionality.

Any directions/help is deeply appreciated.

Best,
Rahul