Hi Kurt, Re: > What I want to propose is we can merge #3 and #4, make them both under >"catalog" concept, by extending catalog function to make it have ability to >have built-in catalog functions. Some benefits I can see from this approach: >1. We don't have to introduce new concept like external built-in functions. >Actually I don't see a full story about how to treat a built-in functions, and it >seems a little bit disrupt with catalog. As a result, you have to make some restriction >like "hive built-in functions can only be used when current catalog is hive catalog".
Yes, I've unified #3 and #4 but it seems I didn't update some part of the doc. I've modified those sections, and they are up to date now. In short, now built-in function of external systems are defined as a special kind of catalog function in Flink, and handled by Flink as following: - An external built-in function must be associated with a catalog for the purpose of decoupling flink-table and external systems. - It always resides in front of catalog functions in ambiguous function reference order, just like in its own external system - It is a special catalog function that doesn’t have a schema/database namespace - It goes thru the same instantiation logic as other user defined catalog functions in the external system Please take another look at the doc, and let me know if you have more questions. On Tue, Sep 3, 2019 at 7:28 AM Timo Walther <twal...@apache.org> wrote: > Hi Kurt, > > it should not affect the functions and operations we currently have in > SQL. It just categorizes the available built-in functions. It is kind of > an orthogonal concept to the catalog API but built-in functions deserve > this special kind of treatment. CatalogFunction still fits perfectly in > there because the regular catalog object resolution logic is not > affected. So tables and functions are resolved in the same way but with > built-in functions that have priority as in the original design. > > Regards, > Timo > > > On 03.09.19 15:26, Kurt Young wrote: > > Does this only affect the functions and operations we currently have in > SQL > > and > > have no effect on tables, right? Looks like this is an orthogonal concept > > with Catalog? > > If the answer are both yes, then the catalog function will be a weird > > concept? > > > > Best, > > Kurt > > > > > > On Tue, Sep 3, 2019 at 8:10 PM Danny Chan <yuzhao....@gmail.com> wrote: > > > >> The way you proposed are basically the same as what Calcite does, I > think > >> we are in the same line. > >> > >> Best, > >> Danny Chan > >> 在 2019年9月3日 +0800 PM7:57，Timo Walther <twal...@apache.org>，写道： > >>> This sounds exactly as the module approach I mentioned, no? > >>> > >>> Regards, > >>> Timo > >>> > >>> On 03.09.19 13:42, Danny Chan wrote: > >>>> Thanks Bowen for bring up this topic, I think it’s a useful > >> refactoring to make our function usage more user friendly. > >>>> For the topic of how to organize the builtin operators and operators > >> of Hive, here is a solution from Apache Calcite, the Calcite way is to > make > >> every dialect operators a “Library”, user can specify which libraries > they > >> want to use for a sql query. The builtin operators always comes as the > >> first class objects and the others are used from the order they appears. > >> Maybe you can take a reference. > >>>> [1] > >> > https://github.com/apache/calcite/commit/9a4eab5240d96379431d14a1ac33bfebaf6fbb28 > >>>> Best, > >>>> Danny Chan > >>>> 在 2019年8月28日 +0800 AM2:50，Bowen Li <bowenl...@gmail.com>，写道： > >>>>> Hi folks, > >>>>> > >>>>> I'd like to kick off a discussion on reworking Flink's > >> FunctionCatalog. > >>>>> It's critically helpful to improve function usability in SQL. > >>>>> > >>>>> > >> > https://docs.google.com/document/d/1w3HZGj9kry4RsKVCduWp82HkW6hhgi2unnvOAUS72t8/edit?usp=sharing > >>>>> In short, it: > >>>>> - adds support for precise function reference with fully/partially > >>>>> qualified name > >>>>> - redefines function resolution order for ambiguous function > >> reference > >>>>> - adds support for Hive's rich built-in functions (support for Hive > >> user > >>>>> defined functions was already added in 1.9.0) > >>>>> - clarifies the concept of temporary functions > >>>>> > >>>>> Would love to hear your thoughts. > >>>>> > >>>>> Bowen > >>> > >
