Hi everyone,
@Xuefu: I would like to avoid adding too many things incrementally.
Users should be able to override all catalog objects consistently
according to FLIP-64 (Support for Temporary Objects in Table module). If
functions are treated completely different, we need more code and
special cases. From an implementation perspective, this topic only
affects the lookup logic which is rather low implementation effort which
is why I would like to clarify the remaining items. As you said, we have
a slight consenus on overriding built-in functions; we should also
strive for reaching consensus on the remaining topics.
@Dawid: I like your idea as it ensures registering catalog objects
consistent and the overriding of built-in functions more explicit.
Thanks,
Timo
On 17.09.19 11:59, kai wang wrote:
hi, everyone
I think this flip is very meaningful. it supports functions that can be
shared by different catalogs and dbs, reducing the duplication of functions.
Our group based on flink's sql parser module implements create function
feature, stores the parsed function metadata and schema into mysql, and
also customizes the catalog, customizes sql-client to support custom
schemas and functions. Loaded, but the function is currently global, and is
not subdivided according to catalog and db.
In addition, I very much hope to participate in the development of this
flip, I have been paying attention to the community, but found it is more
difficult to join.
thank you.
Xuefu Z <usxu...@gmail.com> 于2019年9月17日周二 上午11:19写道：
Thanks to Tmo and Dawid for sharing thoughts.
It seems to me that there is a general consensus on having temp functions
that have no namespaces and overwrite built-in functions. (As a side note
for comparability, the current user defined functions are all temporary and
having no namespaces.)
Nevertheless, I can also see the merit of having namespaced temp functions
that can overwrite functions defined in a specific cat/db. However, this
idea appears orthogonal to the former and can be added incrementally.
How about we first implement non-namespaced temp functions now and leave
the door open for namespaced ones for later releases as the requirement
might become more crystal? This also helps shorten the debate and allow us
to make some progress along this direction.
As to Dawid's idea of having a dedicated cat/db to host the temporary temp
functions that don't have namespaces, my only concern is the special
treatment for a cat/db, which makes code less clean, as evident in treating
the built-in catalog currently.
Thanks,
Xuefiu
On Mon, Sep 16, 2019 at 5:07 PM Dawid Wysakowicz <
wysakowicz.da...@gmail.com>
wrote:
Hi,
Another idea to consider on top of Timo's suggestion. How about we have a
special namespace (catalog + database) for built-in objects? This catalog
would be invisible for users as Xuefu was suggesting.
Then users could still override built-in functions, if they fully qualify
object with the built-in namespace, but by default the common logic of
current dB & cat would be used.
CREATE TEMPORARY FUNCTION func ...
registers temporary function in current cat & dB
CREATE TEMPORARY FUNCTION cat.db.func ...
registers temporary function in cat db
CREATE TEMPORARY FUNCTION system.system.func ...
Overrides built-in function with temporary function
The built-in/system namespace would not be writable for permanent
objects.
WDYT?
This way I think we can have benefits of both solutions.
Best,
Dawid
On Tue, 17 Sep 2019, 07:24 Timo Walther, <twal...@apache.org> wrote:
Hi Bowen,
I understand the potential benefit of overriding certain built-in
functions. I'm open to such a feature if many people agree. However, it
would be great to still support overriding catalog functions with
temporary functions in order to prototype a query even though a
catalog/database might not be available currently or should not be
modified yet. How about we support both cases?
CREATE TEMPORARY FUNCTION abs
-> creates/overrides a built-in function and never consideres current
catalog and database; inconsistent with other DDL but acceptable for
functions I guess.
CREATE TEMPORARY FUNCTION cat.db.fun
-> creates/overrides a catalog function
Regarding "Flink don't have any other built-in objects (tables, views)
except functions", this might change in the near future. Take
https://issues.apache.org/jira/browse/FLINK-13900 as an example.
Thanks,
Timo
On 14.09.19 01:40, Bowen Li wrote:
Hi Fabian,
Yes, I agree 1-part/no-override is the least favorable thus I didn't
include that as a voting option, and the discussion is mainly between
1-part/override builtin and 3-part/not override builtin.
Re > However, it means that temp functions are differently treated
than
other db objects.
IMO, the treatment difference results from the fact that functions
are
a
bit different from other objects - Flink don't have any other
built-in
objects (tables, views) except functions.
Cheers,
Bowen
--
Xuefu Zhang
"In Honey We Trust!"
