Re: [VOTE] SPIP: Add FunctionCatalog

[Ryan Blue]

And a late +1 from me.

On Fri, Mar 12, 2021 at 5:46 AM Takeshi Yamamuro 
wrote:

> +1, too.
>
> On Fri, Mar 12, 2021 at 8:51 PM kordex  wrote:
>
>> +1 (for what it's worth). It will definitely help our efforts.
>>
>> On Fri, Mar 12, 2021 at 12:14 PM Gengliang Wang  wrote:
>> >
>> > +1 (non-binding)
>> >
>> > On Fri, Mar 12, 2021 at 3:00 PM Hyukjin Kwon 
>> wrote:
>> >>
>> >> +1
>> >>
>> >> 2021년 3월 12일 (금) 오후 2:54, Jungtaek Lim 님이
>> 작성:
>> >>>
>> >>> +1 (non-binding) Excellent description on SPIP doc! Thanks for the
>> amazing effort!
>> >>>
>> >>> On Wed, Mar 10, 2021 at 3:19 AM Liang-Chi Hsieh 
>> wrote:
>> 
>> 
>>  +1 (non-binding).
>> 
>>  Thanks for the work!
>> 
>> 
>>  Erik Krogen wrote
>>  > +1 from me (non-binding)
>>  >
>>  > On Tue, Mar 9, 2021 at 9:27 AM huaxin gao 
>> 
>>  > huaxin.gao11@
>> 
>>  >  wrote:
>>  >
>>  >> +1 (non-binding)
>> 
>> 
>> 
>> 
>> 
>>  --
>>  Sent from:
>> http://apache-spark-developers-list.1001551.n3.nabble.com/
>> 
>>  -
>>  To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>> 
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>
>
> --
> ---
> Takeshi Yamamuro
>


-- 
Ryan Blue
Software Engineer
Netflix

Re: [VOTE] SPIP: Add FunctionCatalog

[Takeshi Yamamuro]

+1, too.

On Fri, Mar 12, 2021 at 8:51 PM kordex  wrote:

> +1 (for what it's worth). It will definitely help our efforts.
>
> On Fri, Mar 12, 2021 at 12:14 PM Gengliang Wang  wrote:
> >
> > +1 (non-binding)
> >
> > On Fri, Mar 12, 2021 at 3:00 PM Hyukjin Kwon 
> wrote:
> >>
> >> +1
> >>
> >> 2021년 3월 12일 (금) 오후 2:54, Jungtaek Lim 님이
> 작성:
> >>>
> >>> +1 (non-binding) Excellent description on SPIP doc! Thanks for the
> amazing effort!
> >>>
> >>> On Wed, Mar 10, 2021 at 3:19 AM Liang-Chi Hsieh 
> wrote:
> 
> 
>  +1 (non-binding).
> 
>  Thanks for the work!
> 
> 
>  Erik Krogen wrote
>  > +1 from me (non-binding)
>  >
>  > On Tue, Mar 9, 2021 at 9:27 AM huaxin gao 
> 
>  > huaxin.gao11@
> 
>  >  wrote:
>  >
>  >> +1 (non-binding)
> 
> 
> 
> 
> 
>  --
>  Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
> 
>  -
>  To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> 
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

-- 
---
Takeshi Yamamuro

Re: [VOTE] SPIP: Add FunctionCatalog

[kordex]

+1 (for what it's worth). It will definitely help our efforts.

On Fri, Mar 12, 2021 at 12:14 PM Gengliang Wang  wrote:
>
> +1 (non-binding)
>
> On Fri, Mar 12, 2021 at 3:00 PM Hyukjin Kwon  wrote:
>>
>> +1
>>
>> 2021년 3월 12일 (금) 오후 2:54, Jungtaek Lim 님이 작성:
>>>
>>> +1 (non-binding) Excellent description on SPIP doc! Thanks for the amazing 
>>> effort!
>>>
>>> On Wed, Mar 10, 2021 at 3:19 AM Liang-Chi Hsieh  wrote:


 +1 (non-binding).

 Thanks for the work!


 Erik Krogen wrote
 > +1 from me (non-binding)
 >
 > On Tue, Mar 9, 2021 at 9:27 AM huaxin gao 

 > huaxin.gao11@

 >  wrote:
 >
 >> +1 (non-binding)





 --
 Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

 -
 To unsubscribe e-mail: dev-unsubscr...@spark.apache.org


-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] SPIP: Add FunctionCatalog

[Gengliang Wang]

+1 (non-binding)

On Fri, Mar 12, 2021 at 3:00 PM Hyukjin Kwon  wrote:

> +1
>
> 2021년 3월 12일 (금) 오후 2:54, Jungtaek Lim 님이
> 작성:
>
>> +1 (non-binding) Excellent description on SPIP doc! Thanks for the
>> amazing effort!
>>
>> On Wed, Mar 10, 2021 at 3:19 AM Liang-Chi Hsieh  wrote:
>>
>>>
>>> +1 (non-binding).
>>>
>>> Thanks for the work!
>>>
>>>
>>> Erik Krogen wrote
>>> > +1 from me (non-binding)
>>> >
>>> > On Tue, Mar 9, 2021 at 9:27 AM huaxin gao 
>>>
>>> > huaxin.gao11@
>>>
>>> >  wrote:
>>> >
>>> >> +1 (non-binding)
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>>
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>>

Re: [VOTE] SPIP: Add FunctionCatalog

[Hyukjin Kwon]

+1

2021년 3월 12일 (금) 오후 2:54, Jungtaek Lim 님이 작성:

> +1 (non-binding) Excellent description on SPIP doc! Thanks for the amazing
> effort!
>
> On Wed, Mar 10, 2021 at 3:19 AM Liang-Chi Hsieh  wrote:
>
>>
>> +1 (non-binding).
>>
>> Thanks for the work!
>>
>>
>> Erik Krogen wrote
>> > +1 from me (non-binding)
>> >
>> > On Tue, Mar 9, 2021 at 9:27 AM huaxin gao 
>>
>> > huaxin.gao11@
>>
>> >  wrote:
>> >
>> >> +1 (non-binding)
>>
>>
>>
>>
>>
>> --
>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: [VOTE] SPIP: Add FunctionCatalog

[Jungtaek Lim]

+1 (non-binding) Excellent description on SPIP doc! Thanks for the amazing
effort!

On Wed, Mar 10, 2021 at 3:19 AM Liang-Chi Hsieh  wrote:

>
> +1 (non-binding).
>
> Thanks for the work!
>
>
> Erik Krogen wrote
> > +1 from me (non-binding)
> >
> > On Tue, Mar 9, 2021 at 9:27 AM huaxin gao 
>
> > huaxin.gao11@
>
> >  wrote:
> >
> >> +1 (non-binding)
>
>
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: [VOTE] SPIP: Add FunctionCatalog

[Liang-Chi Hsieh]

+1 (non-binding).

Thanks for the work!


Erik Krogen wrote
> +1 from me (non-binding)
> 
> On Tue, Mar 9, 2021 at 9:27 AM huaxin gao 

> huaxin.gao11@

>  wrote:
> 
>> +1 (non-binding)





--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] SPIP: Add FunctionCatalog

[Erik Krogen]

+1 from me (non-binding)

On Tue, Mar 9, 2021 at 9:27 AM huaxin gao  wrote:

> +1 (non-binding)
>
> On Tue, Mar 9, 2021 at 1:12 AM Kent Yao  wrote:
>
>> +1, looks great!
>>
>> *Kent Yao *
>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>> *a spark enthusiast*
>> *kyuubi is a
>> unified multi-tenant JDBC interface for large-scale data processing and
>> analytics, built on top of Apache Spark .*
>> *spark-authorizer A Spark
>> SQL extension which provides SQL Standard Authorization for **Apache
>> Spark .*
>> *spark-postgres  A library
>> for reading data from and transferring data to Postgres / Greenplum with
>> Spark SQL and DataFrames, 10~100x faster.*
>> *spark-func-extras A
>> library that brings excellent and useful functions from various modern
>> database management systems to Apache Spark .*
>>
>>
>>
>> On 03/9/2021 17:10，Wenchen Fan 
>> wrote：
>>
>> +1 (binding)
>>
>> On Tue, Mar 9, 2021 at 1:47 PM Russell Spitzer 
>> wrote:
>>
>>> +1 (for what it's worth)
>>>
>>> Thanks for making such a robust proposal, i'm excited to see the new
>>> work coming from this
>>>
>>> On Mar 8, 2021, at 11:44 PM, Dongjoon Hyun 
>>> wrote:
>>>
>>> +1 (binding)
>>>
>>> Thank you, Ryan.
>>>
>>> Bests,
>>> Dongjoon.
>>>
>>>
>>> On Mon, Mar 8, 2021 at 5:20 PM Chao Sun  wrote:
>>>
 +1 (non-binding)

 On Mon, Mar 8, 2021 at 5:13 PM John Zhuge  wrote:

> +1 (non-binding)
>
> On Mon, Mar 8, 2021 at 4:32 PM Holden Karau 
> wrote:
>
>> +1 (binding)
>>
>> On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:
>>
>>> Hi everyone, I’d like to start a vote for the FunctionCatalog design
>>> proposal (SPIP).
>>>
>>> The proposal is to add a FunctionCatalog interface that can be used
>>> to load and list functions for Spark to call. There are interfaces for
>>> scalar and aggregate functions.
>>>
>>> In the discussion we’ve come to consensus and I’ve updated the
>>> design doc to match how functions will be called:
>>>
>>> In addition to produceResult(InternalRow), which is optional,
>>> functions can define produceResult methods with arguments that are
>>> Spark’s internal data types, like UTF8String. Spark will prefer
>>> these methods when calling the UDF using codgen.
>>>
>>> I’ve also updated the AggregateFunction interface and merged it
>>> with the partial aggregate interface because Spark doesn’t support
>>> non-partial aggregates.
>>>
>>> The full SPIP doc is here:
>>> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>>>
>>> Please vote on the SPIP in the next 72 hours. Once it is approved,
>>> I’ll do a final update of the PR and we can merge the API.
>>>
>>> [ ] +1: Accept the proposal as an official SPIP
>>> [ ] +0
>>> [ ] -1: I don’t think this is a good idea because …
>>> --
>>> Ryan Blue
>>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9  
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
>
>
> --
> John Zhuge
>

>>> - To
>> unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: [VOTE] SPIP: Add FunctionCatalog

[huaxin gao]

+1 (non-binding)

On Tue, Mar 9, 2021 at 1:12 AM Kent Yao  wrote:

> +1, looks great!
>
> *Kent Yao *
> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> *a spark enthusiast*
> *kyuubi is a unified multi-tenant JDBC
> interface for large-scale data processing and analytics, built on top
> of Apache Spark .*
> *spark-authorizer A Spark
> SQL extension which provides SQL Standard Authorization for **Apache
> Spark .*
> *spark-postgres  A library for
> reading data from and transferring data to Postgres / Greenplum with Spark
> SQL and DataFrames, 10~100x faster.*
> *spark-func-extras A
> library that brings excellent and useful functions from various modern
> database management systems to Apache Spark .*
>
>
>
> On 03/9/2021 17:10，Wenchen Fan 
> wrote：
>
> +1 (binding)
>
> On Tue, Mar 9, 2021 at 1:47 PM Russell Spitzer 
> wrote:
>
>> +1 (for what it's worth)
>>
>> Thanks for making such a robust proposal, i'm excited to see the new work
>> coming from this
>>
>> On Mar 8, 2021, at 11:44 PM, Dongjoon Hyun 
>> wrote:
>>
>> +1 (binding)
>>
>> Thank you, Ryan.
>>
>> Bests,
>> Dongjoon.
>>
>>
>> On Mon, Mar 8, 2021 at 5:20 PM Chao Sun  wrote:
>>
>>> +1 (non-binding)
>>>
>>> On Mon, Mar 8, 2021 at 5:13 PM John Zhuge  wrote:
>>>
 +1 (non-binding)

 On Mon, Mar 8, 2021 at 4:32 PM Holden Karau 
 wrote:

> +1 (binding)
>
> On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:
>
>> Hi everyone, I’d like to start a vote for the FunctionCatalog design
>> proposal (SPIP).
>>
>> The proposal is to add a FunctionCatalog interface that can be used
>> to load and list functions for Spark to call. There are interfaces for
>> scalar and aggregate functions.
>>
>> In the discussion we’ve come to consensus and I’ve updated the design
>> doc to match how functions will be called:
>>
>> In addition to produceResult(InternalRow), which is optional,
>> functions can define produceResult methods with arguments that are
>> Spark’s internal data types, like UTF8String. Spark will prefer
>> these methods when calling the UDF using codgen.
>>
>> I’ve also updated the AggregateFunction interface and merged it with
>> the partial aggregate interface because Spark doesn’t support non-partial
>> aggregates.
>>
>> The full SPIP doc is here:
>> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>>
>> Please vote on the SPIP in the next 72 hours. Once it is approved,
>> I’ll do a final update of the PR and we can merge the API.
>>
>> [ ] +1: Accept the proposal as an official SPIP
>> [ ] +0
>> [ ] -1: I don’t think this is a good idea because …
>> --
>> Ryan Blue
>>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9  
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>


 --
 John Zhuge

>>>
>> - To
> unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] SPIP: Add FunctionCatalog

[Kent Yao]

+1, looks great!






  



















Kent Yao @ Data Science Center, Hangzhou Research Institute, NetEase Corp.a spark enthusiastkyuubiis a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark.spark-authorizerA Spark SQL extension which provides SQL Standard Authorization for Apache Spark.spark-postgres A library for reading data from and transferring data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.spark-func-extrasA library that brings excellent and useful functions from various modern database management systems to Apache Spark.















 


On 03/9/2021 17:10，Wenchen Fan wrote： 


+1 (binding)On Tue, Mar 9, 2021 at 1:47 PM Russell Spitzer  wrote:+1 (for what it's worth)Thanks for making such a robust proposal, i'm excited to see the new work coming from thisOn Mar 8, 2021, at 11:44 PM, Dongjoon Hyun  wrote:+1 (binding)Thank you, Ryan.Bests,Dongjoon.On Mon, Mar 8, 2021 at 5:20 PM Chao Sun  wrote:+1 (non-binding)On Mon, Mar 8, 2021 at 5:13 PM John Zhuge  wrote:+1 (non-binding)On Mon, Mar 8, 2021 at 4:32 PM Holden Karau  wrote:+1 (binding)On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:Hi everyone, I’d like to start a vote for the FunctionCatalog design proposal (SPIP).The proposal is to add a FunctionCatalog interface that can be used to load and list functions for Spark to call. There are interfaces for scalar and aggregate functions.In the discussion we’ve come to consensus and I’ve updated the design doc to match how functions will be called:
In addition to produceResult(InternalRow), which is optional, functions can define produceResult methods with arguments that are Spark’s internal data types, like UTF8String. Spark will prefer these methods when calling the UDF using codgen.
I’ve also updated the AggregateFunction interface and merged it with the partial aggregate interface because Spark doesn’t support non-partial aggregates.The full SPIP doc is here: https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwlPlease vote on the SPIP in the next 72 hours. Once it is approved, I’ll do a final update of the PR and we can merge the API.[ ] +1: Accept the proposal as an official SPIP[ ] +0[ ] -1: I don’t think this is a good idea because …
-- Ryan Blue
-- Twitter: https://twitter.com/holdenkarauBooks (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 YouTube Live Streams: https://www.youtube.com/user/holdenkarau
-- John Zhuge








-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] SPIP: Add FunctionCatalog

[Wenchen Fan]

+1 (binding)

On Tue, Mar 9, 2021 at 1:47 PM Russell Spitzer 
wrote:

> +1 (for what it's worth)
>
> Thanks for making such a robust proposal, i'm excited to see the new work
> coming from this
>
> On Mar 8, 2021, at 11:44 PM, Dongjoon Hyun 
> wrote:
>
> +1 (binding)
>
> Thank you, Ryan.
>
> Bests,
> Dongjoon.
>
>
> On Mon, Mar 8, 2021 at 5:20 PM Chao Sun  wrote:
>
>> +1 (non-binding)
>>
>> On Mon, Mar 8, 2021 at 5:13 PM John Zhuge  wrote:
>>
>>> +1 (non-binding)
>>>
>>> On Mon, Mar 8, 2021 at 4:32 PM Holden Karau 
>>> wrote:
>>>
 +1 (binding)

 On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:

> Hi everyone, I’d like to start a vote for the FunctionCatalog design
> proposal (SPIP).
>
> The proposal is to add a FunctionCatalog interface that can be used to
> load and list functions for Spark to call. There are interfaces for scalar
> and aggregate functions.
>
> In the discussion we’ve come to consensus and I’ve updated the design
> doc to match how functions will be called:
>
> In addition to produceResult(InternalRow), which is optional,
> functions can define produceResult methods with arguments that are
> Spark’s internal data types, like UTF8String. Spark will prefer these
> methods when calling the UDF using codgen.
>
> I’ve also updated the AggregateFunction interface and merged it with
> the partial aggregate interface because Spark doesn’t support non-partial
> aggregates.
>
> The full SPIP doc is here:
> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>
> Please vote on the SPIP in the next 72 hours. Once it is approved,
> I’ll do a final update of the PR and we can merge the API.
>
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because …
> --
> Ryan Blue
>
 --
 Twitter: https://twitter.com/holdenkarau
 Books (Learning Spark, High Performance Spark, etc.):
 https://amzn.to/2MaRAG9  
 YouTube Live Streams: https://www.youtube.com/user/holdenkarau

>>>
>>>
>>> --
>>> John Zhuge
>>>
>>
>

Re: [VOTE] SPIP: Add FunctionCatalog

[Russell Spitzer]

+1 (for what it's worth)

Thanks for making such a robust proposal, i'm excited to see the new work 
coming from this

> On Mar 8, 2021, at 11:44 PM, Dongjoon Hyun  wrote:
> 
> +1 (binding)
> 
> Thank you, Ryan.
> 
> Bests,
> Dongjoon.
> 
> 
> On Mon, Mar 8, 2021 at 5:20 PM Chao Sun  > wrote:
> +1 (non-binding)
> 
> On Mon, Mar 8, 2021 at 5:13 PM John Zhuge  > wrote:
> +1 (non-binding)
> 
> On Mon, Mar 8, 2021 at 4:32 PM Holden Karau  > wrote:
> +1 (binding)
> 
> On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  > wrote:
> Hi everyone, I’d like to start a vote for the FunctionCatalog design proposal 
> (SPIP).
> 
> The proposal is to add a FunctionCatalog interface that can be used to load 
> and list functions for Spark to call. There are interfaces for scalar and 
> aggregate functions.
> 
> In the discussion we’ve come to consensus and I’ve updated the design doc to 
> match how functions will be called:
> 
> In addition to produceResult(InternalRow), which is optional, functions can 
> define produceResult methods with arguments that are Spark’s internal data 
> types, like UTF8String. Spark will prefer these methods when calling the UDF 
> using codgen.
> 
> I’ve also updated the AggregateFunction interface and merged it with the 
> partial aggregate interface because Spark doesn’t support non-partial 
> aggregates.
> 
> The full SPIP doc is here: 
> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>  
> 
> Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll do a 
> final update of the PR and we can merge the API.
> 
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because …
> 
> -- 
> Ryan Blue
> -- 
> Twitter: https://twitter.com/holdenkarau 
> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
>  
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau 
> 
> 
> -- 
> John Zhuge

Re: [VOTE] SPIP: Add FunctionCatalog

[Dongjoon Hyun]

+1 (binding)

Thank you, Ryan.

Bests,
Dongjoon.


On Mon, Mar 8, 2021 at 5:20 PM Chao Sun  wrote:

> +1 (non-binding)
>
> On Mon, Mar 8, 2021 at 5:13 PM John Zhuge  wrote:
>
>> +1 (non-binding)
>>
>> On Mon, Mar 8, 2021 at 4:32 PM Holden Karau  wrote:
>>
>>> +1 (binding)
>>>
>>> On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:
>>>
 Hi everyone, I’d like to start a vote for the FunctionCatalog design
 proposal (SPIP).

 The proposal is to add a FunctionCatalog interface that can be used to
 load and list functions for Spark to call. There are interfaces for scalar
 and aggregate functions.

 In the discussion we’ve come to consensus and I’ve updated the design
 doc to match how functions will be called:

 In addition to produceResult(InternalRow), which is optional,
 functions can define produceResult methods with arguments that are
 Spark’s internal data types, like UTF8String. Spark will prefer these
 methods when calling the UDF using codgen.

 I’ve also updated the AggregateFunction interface and merged it with
 the partial aggregate interface because Spark doesn’t support non-partial
 aggregates.

 The full SPIP doc is here:
 https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl

 Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll
 do a final update of the PR and we can merge the API.

 [ ] +1: Accept the proposal as an official SPIP
 [ ] +0
 [ ] -1: I don’t think this is a good idea because …
 --
 Ryan Blue

>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>> Books (Learning Spark, High Performance Spark, etc.):
>>> https://amzn.to/2MaRAG9  
>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>
>>
>>
>> --
>> John Zhuge
>>
>

Re: [VOTE] SPIP: Add FunctionCatalog

[Chao Sun]

+1 (non-binding)

On Mon, Mar 8, 2021 at 5:13 PM John Zhuge  wrote:

> +1 (non-binding)
>
> On Mon, Mar 8, 2021 at 4:32 PM Holden Karau  wrote:
>
>> +1 (binding)
>>
>> On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:
>>
>>> Hi everyone, I’d like to start a vote for the FunctionCatalog design
>>> proposal (SPIP).
>>>
>>> The proposal is to add a FunctionCatalog interface that can be used to
>>> load and list functions for Spark to call. There are interfaces for scalar
>>> and aggregate functions.
>>>
>>> In the discussion we’ve come to consensus and I’ve updated the design
>>> doc to match how functions will be called:
>>>
>>> In addition to produceResult(InternalRow), which is optional, functions
>>> can define produceResult methods with arguments that are Spark’s
>>> internal data types, like UTF8String. Spark will prefer these methods
>>> when calling the UDF using codgen.
>>>
>>> I’ve also updated the AggregateFunction interface and merged it with
>>> the partial aggregate interface because Spark doesn’t support non-partial
>>> aggregates.
>>>
>>> The full SPIP doc is here:
>>> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>>>
>>> Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll
>>> do a final update of the PR and we can merge the API.
>>>
>>> [ ] +1: Accept the proposal as an official SPIP
>>> [ ] +0
>>> [ ] -1: I don’t think this is a good idea because …
>>> --
>>> Ryan Blue
>>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9  
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
>
>
> --
> John Zhuge
>

Re: [VOTE] SPIP: Add FunctionCatalog

[John Zhuge]

+1 (non-binding)

On Mon, Mar 8, 2021 at 4:32 PM Holden Karau  wrote:

> +1 (binding)
>
> On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:
>
>> Hi everyone, I’d like to start a vote for the FunctionCatalog design
>> proposal (SPIP).
>>
>> The proposal is to add a FunctionCatalog interface that can be used to
>> load and list functions for Spark to call. There are interfaces for scalar
>> and aggregate functions.
>>
>> In the discussion we’ve come to consensus and I’ve updated the design doc
>> to match how functions will be called:
>>
>> In addition to produceResult(InternalRow), which is optional, functions
>> can define produceResult methods with arguments that are Spark’s
>> internal data types, like UTF8String. Spark will prefer these methods
>> when calling the UDF using codgen.
>>
>> I’ve also updated the AggregateFunction interface and merged it with the
>> partial aggregate interface because Spark doesn’t support non-partial
>> aggregates.
>>
>> The full SPIP doc is here:
>> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>>
>> Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll
>> do a final update of the PR and we can merge the API.
>>
>> [ ] +1: Accept the proposal as an official SPIP
>> [ ] +0
>> [ ] -1: I don’t think this is a good idea because …
>> --
>> Ryan Blue
>>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9  
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>


-- 
John Zhuge

Re: [VOTE] SPIP: Add FunctionCatalog

[Holden Karau]

+1 (binding)

On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue  wrote:

> Hi everyone, I’d like to start a vote for the FunctionCatalog design
> proposal (SPIP).
>
> The proposal is to add a FunctionCatalog interface that can be used to
> load and list functions for Spark to call. There are interfaces for scalar
> and aggregate functions.
>
> In the discussion we’ve come to consensus and I’ve updated the design doc
> to match how functions will be called:
>
> In addition to produceResult(InternalRow), which is optional, functions
> can define produceResult methods with arguments that are Spark’s internal
> data types, like UTF8String. Spark will prefer these methods when calling
> the UDF using codgen.
>
> I’ve also updated the AggregateFunction interface and merged it with the
> partial aggregate interface because Spark doesn’t support non-partial
> aggregates.
>
> The full SPIP doc is here:
> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>
> Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll do
> a final update of the PR and we can merge the API.
>
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because …
> --
> Ryan Blue
>
-- 
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  
YouTube Live Streams: https://www.youtube.com/user/holdenkarau