Hi Timo,
Thanks for your reply.
If we aim for the option 1, it makes sense for me to include the change in this
FLIP as the option 1 does not change any public API. I'll update the FLIP page
to illustrate this.
Best,
Wei
> 在 2020年3月9日，17:58，Timo Walther <twal...@apache.org> 写道：
>
> Hi Wei,
>
> I agree with Dawid that we should defer the instantiation of temporary
> functions to compile time. In the long-term, we would like to integrate
> FunctionCatalog as a component of CatalogManager and unify the handling of
> catalog objects as much as possible.
>
> We should aim for your proposed option 1. For fluent definition of functions
> in Table API, we would still like to offer passing instances like
> `t.select(call(new ScalarFunction() { ... }))` that would be registered as
> temporary system functions.
>
> Regrds,
> Timo
>
>
> On 09.03.20 09:24, Wei Zhong wrote:
>> Hi Dawid,
>> I think defering the instantiation of temporary functions to compile time is
>> quite a good idea but needs further discussion. As it is orthogonal with
>> this FLIP, we could continue the discussion in a new thread later. What do
>> you think?
>> Best,
>> Wei
>>> 在 2020年3月5日，21:11，Wei Zhong <weizhong0...@gmail.com> 写道：
>>>
>>> Hi Dawid,
>>>
>>> Thanks for your suggestion.
>>>
>>> After some investigation, there are two designs in my mind about how to
>>> defer the instantiation of temporary system function and temporary catalog
>>> function to compile time.
>>>
>>> 1. FunctionCatalog accepts both FunctionDefinitions and uninstantiated
>>> temporary functions. The uninstantiated temporary functions will be
>>> instantiated when compiling. There is no public API change in this design,
>>> but the FunctionCatalog needs to store and process both FunctionDefinitions
>>> and uninstantiated temporary functions.
>>>
>>> 2. FunctionCatalog accepts only uninstantiated temporary functions. In this
>>> design we need to remove those APIs that accepts FunctionDefinitions from
>>> TableEnvironment, i.e. `void createTemporaryFunction(String path,
>>> UserDefinedFunction functionInstance)` and `void
>>> createTemporarySystemFunction(String name, UserDefinedFunction
>>> functionInstance)`. But the FunctionCatalog only needs to store and process
>>> uninstantiated temporary functions.
>>>
>>> As I don't know the details about the plan to store temporary functions as
>>> catalog functions instead of FunctionDefinitions, I'm not sure which
>>> solution fits more. It would be great if you could share more details or
>>> share some thoughts on these two solutions?
>>>
>>> Best,
>>> Wei
>>>
>>>> 在 2020年3月4日，16:17，Dawid Wysakowicz <dwysakow...@apache.org> 写道：
>>>>
>>>> Hi all,
>>>> I had a really quick look and from my perspective the proposal looks fine.
>>>> I share Jarks opinion that the instantiation could be done at a later
>>>> stage. I agree with Wei it requires some changes in the internal
>>>> implementation of the FunctionCatalog, to store temporary functions as
>>>> catalog functions instead of FunctionDefinitions, but we have that on our
>>>> agenda anyway. I would suggest investigating if we could do that as part of
>>>> this flip already. Nevertheless this in theory can be also done later.
>>>>
>>>> Best,
>>>> Dawid
>>>>
>>>> On Mon, 2 Mar 2020, 14:58 Jark Wu, <imj...@gmail.com> wrote:
>>>>
>>>>> Thanks for the explanation, Wei!
>>>>>
>>>>> On Mon, 2 Mar 2020 at 20:59, Wei Zhong <weizhong0...@gmail.com> wrote:
>>>>>
>>>>>> Hi Jark,
>>>>>>
>>>>>> Thanks for your suggestion.
>>>>>>
>>>>>> Actually, the timing of starting a Python process depends on the UDF
>>>>> type,
>>>>>> because the Python process is used to provide the necessary information
>>>>> to
>>>>>> instantiate the FunctionDefinition object of the Python UDF. For catalog
>>>>>> function, the FunctionDefinition will be instantiated when compiling the
>>>>>> job, which means the Python process is required during the compilation
>>>>>> instead of the registeration. For temporary system function and temporary
>>>>>> catalog function, the FunctionDefinition will be instantiated during the
>>>>>> UDF registeration, so the Python process need to be started at that time.
>>>>>>
>>>>>> But this FLIP will only support registering the temporary system function
>>>>>> and temporary catalog function in SQL DDL because registering Python UDF
>>>>> to
>>>>>> catalog is not supported yet. We plan to support the registeration of
>>>>>> Python catalog function (via Table API and SQL DDL) in a separate FLIP.
>>>>>> I'll add a non-goal section to the FLIP page to illustrate this.
>>>>>>
>>>>>> Best,
>>>>>> Wei
>>>>>>
>>>>>>
>>>>>>> 在 2020年3月2日，15:11，Jark Wu <imj...@gmail.com> 写道：
>>>>>>>
>>>>>>> Hi Weizhong,
>>>>>>>
>>>>>>> Thanks for proposing this feature. In geneal, I'm +1 from the table's
>>>>>> view.
>>>>>>>
>>>>>>> I have one suggestion: I think the register python function into
>>>>> catalog
>>>>>>> doesn't need to startup python process (the "High Level Sequence
>>>>> Diagram"
>>>>>>> in your FLIP).
>>>>>>> Because only meta-information is persisted into catalog, we don't need
>>>>> to
>>>>>>> store "return type", "input types" into catalog.
>>>>>>> I guess the python process is required when compiling a SQL job.
>>>>>>>
>>>>>>> Best,
>>>>>>> Jark
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, 28 Feb 2020 at 19:04, Benchao Li <libenc...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Big +1 for this feature.
>>>>>>>>
>>>>>>>> We built our SQL platform on Java Table API, and most common UDF are
>>>>>>>> implemented in Java. However some python developers are not familiar
>>>>>> with
>>>>>>>> Java/Scala, and it's very inconvenient for these users to use UDF in
>>>>>> SQL.
>>>>>>>>
>>>>>>>> Wei Zhong <weizhong0...@gmail.com> 于2020年2月28日周五 下午6:58写道：
>>>>>>>>
>>>>>>>>> Thank for your reply Dan!
>>>>>>>>>
>>>>>>>>> By the way, this FLIP is closely related to the SQL API. @Jark Wu <
>>>>>>>>> imj...@gmail.com> @Timo <twal...@apache.org> could you please take a
>>>>>>>>> look?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Wei
>>>>>>>>>
>>>>>>>>>> 在 2020年2月25日，16:25，zoudan <zoud...@163.com> 写道：
>>>>>>>>>>
>>>>>>>>>> +1 for supporting Python UDF in Java/Scala Table API.
>>>>>>>>>> This is a great feature and would be helpful for python users!
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Dan Zou
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Benchao Li
>>>>>>>> School of Electronics Engineering and Computer Science, Peking
>>>>>> University
>>>>>>>> Tel:+86-15650713730
>>>>>>>> Email: libenc...@gmail.com; libenc...@pku.edu.cn
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>
