spark git commit: [SPARK-23122][PYSPARK][FOLLOW-UP] Update the docs for UDF Registration
Repository: spark
Updated Branches:
refs/heads/branch-2.3 cf078a205 -> 743b9173f
[SPARK-23122][PYSPARK][FOLLOW-UP] Update the docs for UDF Registration
## What changes were proposed in this pull request?
This PR is to update the docs for UDF registration
## How was this patch tested?
N/A
Author: gatorsmile
Closes #20348 from gatorsmile/testUpdateDoc.
(cherry picked from commit 73281161fc7fddd645c712986ec376ac2b1bd213)
Signed-off-by: gatorsmile
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/743b9173
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/743b9173
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/743b9173
Branch: refs/heads/branch-2.3
Commit: 743b9173f8feaed8e594961aa85d61fb3f8e5e70
Parents: cf078a2
Author: gatorsmile
Authored: Mon Jan 22 04:27:59 2018 -0800
Committer: gatorsmile
Committed: Mon Jan 22 04:28:08 2018 -0800
--
python/pyspark/sql/udf.py | 12
1 file changed, 8 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/spark/blob/743b9173/python/pyspark/sql/udf.py
--
diff --git a/python/pyspark/sql/udf.py b/python/pyspark/sql/udf.py
index c77f19f8..134badb 100644
--- a/python/pyspark/sql/udf.py
+++ b/python/pyspark/sql/udf.py
@@ -199,8 +199,8 @@ class UDFRegistration(object):
@ignore_unicode_prefix
@since("1.3.1")
def register(self, name, f, returnType=None):
-"""Registers a Python function (including lambda function) or a
user-defined function
-in SQL statements.
+"""Register a Python function (including lambda function) or a
user-defined function
+as a SQL function.
:param name: name of the user-defined function in SQL statements.
:param f: a Python function, or a user-defined function. The
user-defined function can
@@ -210,6 +210,10 @@ class UDFRegistration(object):
be either a :class:`pyspark.sql.types.DataType` object or a
DDL-formatted type string.
:return: a user-defined function.
+To register a nondeterministic Python function, users need to first
build
+a nondeterministic user-defined function for the Python function and
then register it
+as a SQL function.
+
`returnType` can be optionally specified when `f` is a Python function
but not
when `f` is a user-defined function. Please see below.
@@ -297,7 +301,7 @@ class UDFRegistration(object):
@ignore_unicode_prefix
@since(2.3)
def registerJavaFunction(self, name, javaClassName, returnType=None):
-"""Register a Java user-defined function so it can be used in SQL
statements.
+"""Register a Java user-defined function as a SQL function.
In addition to a name and the function itself, the return type can be
optionally specified.
When the return type is not specified we would infer it via reflection.
@@ -334,7 +338,7 @@ class UDFRegistration(object):
@ignore_unicode_prefix
@since(2.3)
def registerJavaUDAF(self, name, javaClassName):
-"""Register a Java user-defined aggregate function so it can be used
in SQL statements.
+"""Register a Java user-defined aggregate function as a SQL function.
:param name: name of the user-defined aggregate function
:param javaClassName: fully qualified name of java class
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
spark git commit: [SPARK-23122][PYSPARK][FOLLOW-UP] Update the docs for UDF Registration
Repository: spark
Updated Branches:
refs/heads/master 60175e959 -> 73281161f
[SPARK-23122][PYSPARK][FOLLOW-UP] Update the docs for UDF Registration
## What changes were proposed in this pull request?
This PR is to update the docs for UDF registration
## How was this patch tested?
N/A
Author: gatorsmile
Closes #20348 from gatorsmile/testUpdateDoc.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/73281161
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/73281161
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/73281161
Branch: refs/heads/master
Commit: 73281161fc7fddd645c712986ec376ac2b1bd213
Parents: 60175e959
Author: gatorsmile
Authored: Mon Jan 22 04:27:59 2018 -0800
Committer: gatorsmile
Committed: Mon Jan 22 04:27:59 2018 -0800
--
python/pyspark/sql/udf.py | 12
1 file changed, 8 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/spark/blob/73281161/python/pyspark/sql/udf.py
--
diff --git a/python/pyspark/sql/udf.py b/python/pyspark/sql/udf.py
index c77f19f8..134badb 100644
--- a/python/pyspark/sql/udf.py
+++ b/python/pyspark/sql/udf.py
@@ -199,8 +199,8 @@ class UDFRegistration(object):
@ignore_unicode_prefix
@since("1.3.1")
def register(self, name, f, returnType=None):
-"""Registers a Python function (including lambda function) or a
user-defined function
-in SQL statements.
+"""Register a Python function (including lambda function) or a
user-defined function
+as a SQL function.
:param name: name of the user-defined function in SQL statements.
:param f: a Python function, or a user-defined function. The
user-defined function can
@@ -210,6 +210,10 @@ class UDFRegistration(object):
be either a :class:`pyspark.sql.types.DataType` object or a
DDL-formatted type string.
:return: a user-defined function.
+To register a nondeterministic Python function, users need to first
build
+a nondeterministic user-defined function for the Python function and
then register it
+as a SQL function.
+
`returnType` can be optionally specified when `f` is a Python function
but not
when `f` is a user-defined function. Please see below.
@@ -297,7 +301,7 @@ class UDFRegistration(object):
@ignore_unicode_prefix
@since(2.3)
def registerJavaFunction(self, name, javaClassName, returnType=None):
-"""Register a Java user-defined function so it can be used in SQL
statements.
+"""Register a Java user-defined function as a SQL function.
In addition to a name and the function itself, the return type can be
optionally specified.
When the return type is not specified we would infer it via reflection.
@@ -334,7 +338,7 @@ class UDFRegistration(object):
@ignore_unicode_prefix
@since(2.3)
def registerJavaUDAF(self, name, javaClassName):
-"""Register a Java user-defined aggregate function so it can be used
in SQL statements.
+"""Register a Java user-defined aggregate function as a SQL function.
:param name: name of the user-defined aggregate function
:param javaClassName: fully qualified name of java class
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
