[
https://issues.apache.org/jira/browse/SPARK-4867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin updated SPARK-4867:
-------------------------------
Issue Type: Sub-task (was: Bug)
Parent: SPARK-5166
> UDF clean up
> ------------
>
> Key: SPARK-4867
> URL: https://issues.apache.org/jira/browse/SPARK-4867
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Michael Armbrust
> Priority: Blocker
>
> Right now our support and internal implementation of many functions has a few
> issues. Specifically:
> - UDFS don't know their input types and thus don't do type coercion.
> - We hard code a bunch of built in functions into the parser. This is bad
> because in SQL it creates new reserved words for things that aren't actually
> keywords. Also it means that for each function we need to add support to
> both SQLContext and HiveContext separately.
> For this JIRA I propose we do the following:
> - Change the interfaces for registerFunction and ScalaUdf to include types
> for the input arguments as well as the output type.
> - Add a rule to analysis that does type coercion for UDFs.
> - Add a parse rule for functions to SQLParser.
> - Rewrite all the UDFs that are currently hacked into the various parsers
> using this new functionality.
> Depending on how big this refactoring becomes we could split parts 1&2 from
> part 3 above.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
