Jianshi Huang created SPARK-6382:
------------------------------------
Summary: withUDF(...) {...} for supporting temporary UDF
definitions in the scope
Key: SPARK-6382
URL: https://issues.apache.org/jira/browse/SPARK-6382
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 1.3.0, 1.3.1
Reporter: Jianshi Huang
Currently the scope of UDF registration is global. It's unsuitable for
libraries that's built on top of DataFrame, as many operations has to done by
registering a UDF first.
Please provide a way for binding temporary UDFs.
e.g.
{code}
withUDF(("merge_map", (m1: Map[String, Double], m2: Map[String, Double]) => m2
++ m2),
...) {
sql("select merge_map(d1.map, d2.map) from d1, d2 where d1.id = d2.id")
}
{code}
Also UDF registry is a mutable Hashmap, refactoring it to a immutable one makes
more sense.
Jianshi
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
