[
https://issues.apache.org/jira/browse/IMPALA-7741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659759#comment-16659759
]
Paul Rogers edited comment on IMPALA-7741 at 10/22/18 10:45 PM:
----------------------------------------------------------------
{{gen_builtins_catalog.py}} generates {{ScalarBuiltins.java}} from information
in {{impala_functions.py}}. This presents a difficulty. {{impala_functions.py}}
lists functions implemented in the backend. However, the FE rewrites multiple
functions out of existence, so no BE implementation is necessary.
{{impala_functions.py}} divides functions into two categories:
{{visible_functions}} and {{invisible_functions}}. The {{visible_functions}}
category already includes some FE-only functions, so we can expand on this idea.
It turns out that {{nvl2(expr, ifNotNull, ifNull)}} is a bit nasty. The two
second arguments can be of any type, but must be the same type. The first
argument can be of any type, not necessarily the same as the other two. This
can lead to an n^2 table in types.
An easy answer is to leverage the {{NULL}} type for the first argument since we
only care about its nullness, not its type or value.
was (Author: paul.rogers):
{{gen_builtins_catalog.py}} generates {{ScalarBuiltins.java}} from information
in {{impala_functions.py}}. This presents a difficulty. {{impala_functions.py}}
lists functions implemented in the backend. However, the FE rewrites multiple
functions out of existence, so no BE implementation is necessary.
{{impala_functions.py}} divides functions into two categories:
{{visible_functions}} and {{invisible_functions}}. Perhaps the cleanest
solution is to add a third category: {{fe_only_functions}} to describe those
available to the user, but which happen to be rewritten away by the FE.
{{gen_builtins_catalog.py}} would then combine the existing
{{visible_functions}} with the new {{fe_only_functions}} to create the final
set of functions in {{ScalarBuiltins.java}}. The two existing categories would
continue to drive the BE code generation.
> Functions nvl2(), decode(), nullif() not listed in _impala_builtins
> -------------------------------------------------------------------
>
> Key: IMPALA-7741
> URL: https://issues.apache.org/jira/browse/IMPALA-7741
> Project: IMPALA
> Issue Type: Improvement
> Affects Versions: Impala 3.0
> Reporter: Paul Rogers
> Priority: Major
>
> The
> [docs|https://impala.apache.org/docs/build3x/html/topics/impala_show.html]
> for {{SHOW FUNCTIONS}} says that we can use the following to list all
> built-in functions:
> {code:sql}
> show functions in _impala_builtins like '*week*';
> {code}
> However several Impala functions are removed early in the FE planning process
> and thus do not appear in the FE's function table in {{ScalarBuiltins}}:
> {{nvl2()}}, {{decode()}}, and {{nullif()}}. For example:
> {noformat}
> show functions in _impala_builtins like '*decode**'
> +-------------+------------------------------+-------------+---------------+
> | return type | signature | binary type | is persistent |
> +-------------+------------------------------+-------------+---------------+
> | STRING | base64decode(STRING) | BUILTIN | true |
> | STRING | madlib_decode_vector(STRING) | BUILTIN | true |
> +-------------+------------------------------+-------------+---------------+
> {noformat}
> However, since these three are perfectly valid functions, would have expected
> them to appear in the table. How they are processed internally is an
> implementation detail unimportant to the end user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
