Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18792#discussion_r130512174
--- Diff: docs/sql-programming-guide.md ---
@@ -1903,6 +1903,25 @@ releases of Spark SQL.
Hive can optionally merge the small files into fewer large files to
avoid overflowing the HDFS
metadata. Spark SQL does not support that.
+**Hive UDF/UDTF/UDAF**
+
+Spark SQL implements the basic functionality of the Hive UDF/UDTF/UDAF,
but does not support all the APIs for users.
+Some of them are meaningless in Spark and the others are rarely used by
users.
+Below is a list of major APIs we don't support in Spark SQL:
+
+* `getRequiredJars` and `getRequiredFiles` (`UDF` and `GenericUDF`) are
functions to to automatically
+ include additional resources required by this UDF.
+* `initialize(StructObjectInspector)` in `GenericUDTF` is not supported
yet. Spark SQL currently uses
+ a deprecated interface `initialize(ObjectInspector[])` only.
+* `configure` (`GenericUDF`, `GenericUDTF`, and `GenericUDAFEvaluator`) is
a function to initialize
+ functions with `MapredContext`. But, Spark SQL does not use
`MapredContext` internally.
+* `close` (`GenericUDF` and `GenericUDAFEvaluator`) is a function to
release associated resources.
+ Spark SQL does not call this function when tasks finished.
--- End diff --
`finished` -> `finish`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
