[
https://issues.apache.org/jira/browse/FLINK-13755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bowen Li updated FLINK-13755:
-----------------------------
Description:
Unlike UDFs that are persisted in Hive Metastore, Hive builtin functions are
registered into in-memory function catalog at runtime.
cc [~xuefuz] [~lirui] [~Terry1897]
was:
Unlike UDFs that are persisted in Hive Metastore, Hive builtin functions are
registered into in-memory function catalog at runtime, which makes it hard for
Flink to integrate with architecturely.
First and basic option is to do it the hard way by integrating hive's function
registry, which architecturely can be hard.
Second option to support rich Hive built-in functions is to develop builtin
functions in Flink with the same logic. I did a simple comparison. With Flink
1.10.0 and Hive 2.3.4, they have 56 common (of same name) built-in functions;
there are 195 functions in Hive 2.3.4 that don't exist in Flink 1.10.0. Please
see attached files. According my sampling in the 195 functions, some are
straight-forward to rewrite, some don't seem to be frequently used.
Besides rewriting all of them, another option for users is to manually register
those builtin functions in Hive metastore, so Flink can load them thru
HiveCatalog at runtime.
Lastly, we can load and hold hive builtin functions in an in-memory map of
HiveCatalog as if it's from hive function registry
cc [~xuefuz] [~lirui] [~Terry1897]
> support Hive built-in functions in Flink
> ----------------------------------------
>
> Key: FLINK-13755
> URL: https://issues.apache.org/jira/browse/FLINK-13755
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Hive
> Affects Versions: 1.10.0
> Reporter: Bowen Li
> Assignee: Bowen Li
> Priority: Major
> Fix For: 1.10.0
>
> Attachments: common builtin functions is flink and hive.txt, hive
> builtin functions that are missing in flink.txt
>
>
> Unlike UDFs that are persisted in Hive Metastore, Hive builtin functions are
> registered into in-memory function catalog at runtime.
> cc [~xuefuz] [~lirui] [~Terry1897]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
