[
https://issues.apache.org/jira/browse/FLINK-13755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bowen Li updated FLINK-13755:
-----------------------------
Description:
Unlike UDFs that are persisted in Hive Metastore, Hive builtin functions are
registered into in-memory function catalog at runtime, which makes it hard for
Flink to integrate with architecturely.
One option to support rich Hive built-in functions is to develop builtin
functions in Flink with the same logic. I did a simple comparison. With Flink
1.10.0 and Hive 2.3.4, they have 56 common (of same name) built-in functions;
there are 195 functions in Hive 2.3.4 that don't exist in Flink 1.10.0. Please
see attached files. According my sampling in the 195 functions, some are
straight-forward to rewrite, some don't seem to be frequently used. Besides
rewriting all of them, another option for users is to manually register those
builtin functions in Hive metastore, so Flink can load them thru HiveCatalog at
runtime.
cc [~xuefuz] [~lirui] [~Terry1897]
was:
Unlike UDFs that are persisted in Hive Metastore, Hive builtin functions are
registered into in-memory function catalog at runtime, which makes it hard for
Flink to integrate with architecturely.
One option to support rich Hive built-in functions is to develop builtin
functions in Flink with the same logic. I did a simple comparison. With Flink
1.10.0 and Hive 2.3.4, they have 56 common (of same name) built-in functions;
there are 195 functions in Hive 2.3.4 that don't exist in Flink 1.10.0. Please
see attached files
cc [~xuefuz] [~lirui] [~Terry1897]
> support Hive built-in functions in Flink
> ----------------------------------------
>
> Key: FLINK-13755
> URL: https://issues.apache.org/jira/browse/FLINK-13755
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Hive
> Affects Versions: 1.10.0
> Reporter: Bowen Li
> Assignee: Bowen Li
> Priority: Major
> Fix For: 1.10.0
>
> Attachments: common builtin functions is flink and hive.txt, hive
> builtin functions that are missing in flink.txt
>
>
> Unlike UDFs that are persisted in Hive Metastore, Hive builtin functions are
> registered into in-memory function catalog at runtime, which makes it hard
> for Flink to integrate with architecturely.
> One option to support rich Hive built-in functions is to develop builtin
> functions in Flink with the same logic. I did a simple comparison. With Flink
> 1.10.0 and Hive 2.3.4, they have 56 common (of same name) built-in functions;
> there are 195 functions in Hive 2.3.4 that don't exist in Flink 1.10.0.
> Please see attached files. According my sampling in the 195 functions, some
> are straight-forward to rewrite, some don't seem to be frequently used.
> Besides rewriting all of them, another option for users is to manually
> register those builtin functions in Hive metastore, so Flink can load them
> thru HiveCatalog at runtime.
>
> cc [~xuefuz] [~lirui] [~Terry1897]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
