This is an automated email from the ASF dual-hosted git repository.

pan3793 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 7640b3df675d [SPARK-60447][SQL] Support V2 functions in CREATE 
FUNCTION on session catalog
7640b3df675d is described below

commit 7640b3df675d479e3f41ac2bc33dcccf0620c015
Author: Cheng Pan <[email protected]>
AuthorDate: Tue Nov 3 10:11:15 2026 +0800

    [SPARK-60447][SQL] Support V2 functions in CREATE FUNCTION on session 
catalog
    
    ### What changes were proposed in this pull request?
    
    This PR allows `CREATE [TEMPORARY] FUNCTION ... AS 'class' [USING JAR ...]` 
on the session
    catalog to register a DataSource V2 function, i.e. a class implementing
    `org.apache.spark.sql.connector.catalog.functions.UnboundFunction`, the 
same way as Hive UDFs.
    
    ```sql
    ADD JAR /path/to/my-udfs.jar;
    CREATE TEMPORARY FUNCTION strlen AS 'com.example.StrLen';
    -- or persistent, with the jar loaded on first use
    CREATE FUNCTION strlen AS 'com.example.StrLen' USING JAR 
'hdfs:///udfs/my-udfs.jar';
    SELECT strlen(name) FROM t;
    ```
    
    Persistent functions are stored in the external catalog in the same shape 
as Hive UDFs (real
    class name and resource URIs), with no new metadata format; the function 
kind is decided by the
    class when loaded. They behave like Hive UDFs: a missing class, an 
inaccessible jar, or a class
    that cannot be instantiated only fails when the function is invoked, not on 
CREATE, SHOW,
    DESCRIBE or listing, which do not initialize the class of a persistent 
function. Column names use
    the function name. HMS calls are the same as for a Hive UDF.
    
    How other engines sharing the HMS react to such an entry:
    
    1. Hive: an entry it cannot register is logged and skipped.
    2. Flink: lists it, and fails only when it is called.
    3. Impala: skips it with an error log.
    4. Trino, Doris, StarRocks: do not read HMS function entries created by 
other engines.
    
    Flink and Impala store their own Java functions in this same shape. A class 
implementing both a
    Hive UDF interface and `UnboundFunction` runs as a V2 function in Spark and 
as a Hive UDF in
    other engines.
    
    ### Known limitation
    
    A scalar V2 function that implements the magic method (`invoke`), and whose 
jar is only added
    at runtime (`ADD JAR`, or `CREATE FUNCTION ... USING JAR`), may fail to 
compile in generated code
    when the context classloader of the driver thread running the query cannot 
see the jar. For
    example:
    
    1. In spark-shell, the jar is loaded by one line, and a later line calls 
the function, since the
       REPL resets the context classloader after each line.
    2. In a multi-threaded driver application, a thread other than the one that 
loaded the jar runs
       the query.
    3. Spark Connect enabled via `spark.plugins` without
       `spark.sql.artifact.isolation.alwaysApplyClassloader=true`, where 
requests run on different
       threads.
    
    With default configs, whole-stage codegen falls back with a warning and the 
query returns correct
    results more slowly, but operators without a fallback fail the query, e.g.
    `SELECT c FROM t ORDER BY f(c) LIMIT 10`.
    
    Not affected, for the query's main thread:
    
    1. Spark Thrift Server, which sets the jar classloader for each operation.
    2. Spark Connect server started by `start-connect-server.sh`, which applies 
the session
       classloader to each request.
    3. Functions implementing `produceResult` only, and aggregate functions.
    4. Jars provided at startup via `--jars` / `spark.jars`.
    
    This is a pre-existing issue for V2 functions served by a `FunctionCatalog` 
from runtime-added
    jars, tracked by 
[SPARK-60811](https://issues.apache.org/jira/browse/SPARK-60811); this feature
    makes it easier to trigger.
    
    ### Why are the changes needed?
    
    The V2 function API is the modern, codegen-friendly way to write Spark 
functions, but it is only
    usable through a custom `FunctionCatalog`. This PR lets session catalog 
users, including HMS,
    use V2 functions with the familiar `ADD JAR` / `CREATE FUNCTION` workflow, 
without implementing
    a catalog plugin.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes.
    
    1. `CREATE [TEMPORARY] FUNCTION ... AS 'class'` accepts a class 
implementing `UnboundFunction`.
       Previously, calling such a function failed with `NO_HANDLER_FOR_UDAF`, 
or with
       `_LEGACY_ERROR_TEMP_2450` when Hive support is enabled.
    2. `DESCRIBE FUNCTION` on such a function shows the function's description; 
for a persistent
       function, once it has been invoked in the session.
    3. A class implementing `UnboundFunction` that also extends a Hive UDF 
class or
       `UserDefinedAggregateFunction` now runs as a V2 function, including 
functions created in
       earlier versions. See the migration guide.
    4. New error condition `CANNOT_INSTANTIATE_FUNCTION_CLASS`, raised when 
such a class cannot be
       instantiated.
    
    ### How was this patch tested?
    
    New `CreateV2FunctionSuite`, and a new test in `HiveUDFSuite`.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Opus 5.5
    
    Closes #59697 from pan3793/v2-udf.
    
    Authored-by: Cheng Pan <[email protected]>
    Signed-off-by: Cheng Pan <[email protected]>
---
 .../src/main/resources/error/error-conditions.json |   6 +
 docs/sql-migration-guide.md                        |   2 +-
 docs/sql-ref-syntax-ddl-create-function.md         |  11 +-
 .../sql/catalyst/expressions/ExpressionInfo.java   |   2 +-
 .../sql/catalyst/analysis/FunctionResolution.scala |  86 +--
 .../sql/catalyst/catalog/SessionCatalog.scala      | 104 +++-
 .../catalyst/expressions/V2ExpressionUtils.scala   |  54 +-
 .../apache/spark/sql/catalyst/util/package.scala   |   7 +-
 .../spark/sql/errors/QueryCompilationErrors.scala  |  27 +-
 .../catalog/functions/JavaInitTrackingStrLen.java  |  66 +++
 .../functions/JavaStaticInitFailingStrLen.java     |  61 ++
 .../sql/connector/CreateV2FunctionSuite.scala      | 635 +++++++++++++++++++++
 .../sql/expressions/ExpressionInfoSuite.scala      |   2 +-
 .../spark/sql/hive/execution/HiveUDFSuite.scala    |  35 +-
 14 files changed, 997 insertions(+), 101 deletions(-)

diff --git a/common/utils/src/main/resources/error/error-conditions.json 
b/common/utils/src/main/resources/error/error-conditions.json
index 17253772da03..ec42354a2be1 100644
--- a/common/utils/src/main/resources/error/error-conditions.json
+++ b/common/utils/src/main/resources/error/error-conditions.json
@@ -641,6 +641,12 @@
     ],
     "sqlState" : "22546"
   },
+  "CANNOT_INSTANTIATE_FUNCTION_CLASS" : {
+    "message" : [
+      "Cannot instantiate class <className> when registering the function 
<functionName>: <msg>. Please make sure the class has a public no-arg 
constructor that succeeds."
+    ],
+    "sqlState" : "46103"
+  },
   "CANNOT_INSTANTIATE_HIVE_FUNCTION" : {
     "message" : [
       "Cannot instantiate the Hive UDF/UDAF/UDTF wrapper class <clazz>. Check 
that the function arguments and their types are supported."
diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md
index 38341b7c6fc4..bbe256bba9c0 100644
--- a/docs/sql-migration-guide.md
+++ b/docs/sql-migration-guide.md
@@ -67,7 +67,6 @@ license: |
 - Since Spark 4.4, when `spark.sql.parquet.fieldId.read.enabled` is true, a 
Parquet read fails with `PARQUET_FIELD_REQUESTED_MORE_THAN_ONCE` if two or more 
requested fields of a struct (or of the top-level schema) resolve to the same 
Parquet field. This happens when one requested field is matched by field ID and 
another by name, or when two requested fields have the same field ID. In Spark 
4.3 and earlier, such a read returned wrong values without an error: with the 
vectorized reader one [...]
 - Since Spark 4.4, when `spark.sql.groupByOrdinal` is true (the default), the 
pipe `AGGREGATE` operator rejects an aliased integer literal in its `GROUP BY` 
clause, such as `TABLE t |> AGGREGATE sum(x) AS s GROUP BY 2 AS grp`, with 
`UNSUPPORTED_FEATURE.PIPE_OPERATOR_AGGREGATE_UNSUPPORTED_CASE`, because a bare 
integer literal in the same position is an ordinal. Previously the aliased 
literal was grouped as a constant. Bare ordinals and aliases on other grouping 
expressions are not affecte [...]
 - Since Spark 4.4, with whole-stage codegen enabled, a filter condition, 
including the `FILTER` clause of an aggregate function, evaluates the remaining 
top-level conjuncts of the condition on a row after one of them evaluates to 
`NULL`, as the SQL standard requires, and does so in the order they appear in 
the condition, as Spark already did with whole-stage codegen disabled, which 
also evaluates a subexpression shared by several conjuncts before the 
condition. Only a `FALSE` conjunct st [...]
-
 - Since Spark 4.4, when a format is specified, `to_timestamp`, 
`try_to_timestamp`, and `to_timestamp_ltz` cast a numeric first argument to a 
string and parse it with the format, for example `to_timestamp(20201231, 
'yyyyMMdd')` returns `2020-12-31 00:00:00`. This matches the existing behavior 
when `spark.sql.timestampType` is `TIMESTAMP_NTZ`. Previously, with the default 
`TIMESTAMP_LTZ` timestamp type, such calls failed with `INTERNAL_ERROR`. Note 
that the string form of the number is par [...]
 - Since Spark 4.4, the Spark Thrift Server logs a warning at startup when 
`hive.server2.enable.doAs` is `true` and `hive.server2.authentication` verifies 
user identities (any recognized type other than `NONE`/`NOSASL`), because 
executor-side data access still runs as the server's own service identity 
rather than the connecting user (SPARK-5159). Set 
`spark.sql.hive.thriftServer.allowIneffectiveDoAs` to `true` to acknowledge the 
limitation and silence the warning. Spark 5.0 is expected to [...]
 - Since Spark 4.4, `UPDATE` and `MERGE` on a Data Source V2 table fail with 
`DATATYPE_MISMATCH.INVALID_ROW_LEVEL_OPERATION_ASSIGNMENTS` when they assign to 
a metadata column of the table. This includes `UPDATE SET *` and `INSERT *` in 
`MERGE WITH SCHEMA EVOLUTION` when the source has a column with the name of a 
metadata column of the target table. Previously, these assignments passed 
analysis and were usually ignored. To restore the previous behavior, set 
`spark.sql.legacy.ignoreMetadata [...]
@@ -75,6 +74,7 @@ license: |
 - Since Spark 4.4, `CREATE TABLE` and `REPLACE TABLE` on a Data Source V2 
table fail with `AGGREGATE_OR_WINDOW_FUNCTION_IN_CHECK_CONSTRAINT` when the 
condition of a CHECK constraint calls an aggregate or window function, 
including a user-defined aggregate function, `grouping`, and `grouping_id`. 
`ALTER TABLE ... ADD CONSTRAINT` keeps failing with its existing errors, such 
as `INVALID_WHERE_CONDITION` and `WINDOW_FUNCTION_NOT_ALLOWED_IN_CLAUSE`. 
Previously, these statements succeeded and  [...]
 - Since Spark 4.4, a write to a Data Source V2 table that parses the SQL text 
of a CHECK constraint binds an unqualified built-in function name to 
`system.builtin`, regardless of the SQL path and 
`spark.sql.functionResolution.sessionOrder` (SPARK-60777). Previously, an 
existing constraint could call another function with the same name.
 - Since Spark 4.4, `CREATE TABLE`, `REPLACE TABLE`, and `ALTER TABLE ... ADD 
CONSTRAINT` on a Data Source V2 table fail with `SUBQUERY_IN_CHECK_CONSTRAINT` 
when the condition of a CHECK constraint contains a subquery. `ALTER TABLE ... 
ADD CONSTRAINT` keeps failing with 
`UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.HIGHER_ORDER_FUNCTION` on a subquery 
in a lambda function. Previously, most of these statements failed with 
`INTERNAL_ERROR`, but a subquery that needed no resolution, such as `id  [...]
+- Since Spark 4.4, a function created by `CREATE FUNCTION ... AS 'class'` 
whose class implements 
`org.apache.spark.sql.connector.catalog.functions.UnboundFunction` and extends 
either a Hive UDF class (such as `UDF` or `GenericUDF`) or 
`org.apache.spark.sql.expressions.UserDefinedAggregateFunction` runs as a 
DataSource V2 function instead of a Hive UDF or Spark UDAF, including functions 
created in earlier versions. Its result type, argument coercion, and null 
handling are defined by the V [...]
 
 ## Upgrading from Spark SQL 4.2 to 4.3
 
diff --git a/docs/sql-ref-syntax-ddl-create-function.md 
b/docs/sql-ref-syntax-ddl-create-function.md
index 256587049441..3367d539a0c3 100644
--- a/docs/sql-ref-syntax-ddl-create-function.md
+++ b/docs/sql-ref-syntax-ddl-create-function.md
@@ -80,13 +80,22 @@ CREATE [ OR REPLACE ] [ TEMPORARY ] FUNCTION [ IF NOT 
EXISTS ]
 * **class_name**
 
     Specifies the name of the class that provides the implementation for 
function to be created.
-    The implementing class should extend one of the base classes as follows:
+    The implementing class should have a public no-arg constructor, and extend 
one of the base
+    classes as follows:
 
+    * Should implement `UnboundFunction` in 
`org.apache.spark.sql.connector.catalog.functions`
+      package.
     * Should extend `UDF` or `UDAF` in `org.apache.hadoop.hive.ql.exec` 
package.
     * Should extend `AbstractGenericUDAFResolver`, `GenericUDF`, or
       `GenericUDTF` in `org.apache.hadoop.hive.ql.udf.generic` package.
     * Should extend `UserDefinedAggregateFunction` in 
`org.apache.spark.sql.expressions` package.
 
+    **Note:** If an `UnboundFunction` binds to a `ScalarFunction` that 
implements the magic method
+    `invoke`, and its jar is added at runtime (`ADD JAR`, or `USING JAR`), 
codegen may fail to find
+    the class, e.g., in spark-shell. Whole-stage codegen falls back to slower 
interpreted
+    execution, other codegen fails the query. To avoid it, add the jar at 
startup via `--jars`, or
+    implement `produceResult` instead.
+
 * **resource_locations**
 
     Specifies the list of resources that contain the implementation of the 
function
diff --git 
a/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionInfo.java
 
b/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionInfo.java
index 4fd6e543cf09..89b65422b186 100644
--- 
a/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionInfo.java
+++ 
b/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionInfo.java
@@ -53,7 +53,7 @@ public class ExpressionInfo {
 
     private static final Set<String> validSources =
             new HashSet<>(Arrays.asList("built-in", "hive", "python_udf", 
"scala_udf", "sql_udf",
-                    "java_udf", "python_udtf", "internal"));
+                    "java_udf", "python_udtf", "internal", "v2_function"));
 
     public String getClassName() {
         return className;
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionResolution.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionResolution.scala
index ebaecdb117e8..9bb099b13c09 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionResolution.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionResolution.scala
@@ -54,11 +54,7 @@ object FunctionType {
   case object NotFound extends FunctionType
 }
 import org.apache.spark.sql.connector.catalog.CatalogV2Implicits._
-import org.apache.spark.sql.connector.catalog.functions.{
-  AggregateFunction => V2AggregateFunction,
-  ScalarFunction,
-  UnboundFunction
-}
+import org.apache.spark.sql.connector.catalog.functions.UnboundFunction
 import org.apache.spark.sql.errors.{DataTypeErrorsBase, QueryCompilationErrors}
 import org.apache.spark.sql.internal.connector.V1Function
 import org.apache.spark.sql.types._
@@ -686,7 +682,7 @@ class FunctionResolution(
         // clauses (DISTINCT / FILTER / ORDER BY / IGNORE NULLS). They have 
dedicated physical
         // operators that do not honor them, so reject rather than silently 
drop the clause.
         if (agg.isInstanceOf[PythonUDAF] || agg.isInstanceOf[PythonAggregate]) 
{
-          checkUnsupportedAggregateClause(agg, unresolvedFunc)
+          checkUnsupportedAggregateClause(agg.prettyName, unresolvedFunc)
         }
         // After parse, the functions not set the ordering within group yet.
         val newAgg = agg match {
@@ -718,7 +714,7 @@ class FunctionResolution(
         aggFunc.toAggregateExpression(unresolvedFunc.isDistinct, 
unresolvedFunc.filter)
       // This function is not an aggregate function, just return the resolved 
one.
       case other =>
-        checkUnsupportedAggregateClause(other, unresolvedFunc)
+        checkUnsupportedAggregateClause(other.prettyName, unresolvedFunc)
         if (other.isInstanceOf[String2TrimExpression] && numArgs == 2) {
           if (trimWarningEnabled.get) {
             log.warn(
@@ -734,21 +730,21 @@ class FunctionResolution(
   }
 
   private def checkUnsupportedAggregateClause(
-      func: Expression,
+      funcName: String,
       unresolvedFunc: UnresolvedFunction): Unit = {
     if (unresolvedFunc.isDistinct) {
-      throw 
QueryCompilationErrors.functionWithUnsupportedSyntaxError(func.prettyName, 
"DISTINCT")
+      throw 
QueryCompilationErrors.functionWithUnsupportedSyntaxError(funcName, "DISTINCT")
     }
     if (unresolvedFunc.filter.isDefined) {
       throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
-        func.prettyName,
+        funcName,
         "FILTER clause"
       )
     }
     // Only fail for IGNORE NULLS; RESPECT NULLS is the default behavior
     if (unresolvedFunc.ignoreNulls.contains(true)) {
       throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
-        func.prettyName,
+        funcName,
         "IGNORE NULLS"
       )
     }
@@ -796,80 +792,28 @@ class FunctionResolution(
       unbound: UnboundFunction,
       arguments: Seq[Expression],
       unresolvedFunc: UnresolvedFunction): Expression = {
-    val inputType = StructType(arguments.zipWithIndex.map {
-      case (exp, pos) => StructField(s"_$pos", exp.dataType, exp.nullable)
-    })
-    val bound = try {
-      unbound.bind(inputType)
-    } catch {
-      case unsupported: UnsupportedOperationException =>
-        throw QueryCompilationErrors.functionCannotProcessInputError(
-          unbound,
-          arguments,
-          unsupported
-        )
-    }
-
-    if (bound.inputTypes().length != arguments.length) {
-      throw 
QueryCompilationErrors.v2FunctionInvalidInputTypeLengthError(bound, arguments)
-    }
-
-    bound match {
-      case scalarFunc: ScalarFunction[_] =>
-        processV2ScalarFunction(scalarFunc, arguments, unresolvedFunc)
-      case aggFunc: V2AggregateFunction[_, _] =>
-        processV2AggregateFunction(aggFunc, arguments, unresolvedFunc)
-      case _ =>
-        failAnalysis(
-          errorClass = "INVALID_UDF_IMPLEMENTATION",
-          messageParameters = Map("funcName" -> toSQLId(bound.name()))
-        )
-    }
-  }
-
-  private def processV2ScalarFunction(
-      scalarFunc: ScalarFunction[_],
-      arguments: Seq[Expression],
-      unresolvedFunc: UnresolvedFunction): Expression = {
-    if (unresolvedFunc.isDistinct) {
-      throw 
QueryCompilationErrors.functionWithUnsupportedSyntaxError(scalarFunc.name(), 
"DISTINCT")
-    } else if (unresolvedFunc.filter.isDefined) {
-      throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
-        scalarFunc.name(),
-        "FILTER clause"
-      )
-    } else if (unresolvedFunc.ignoreNulls.contains(true)) {
-      // Only fail for IGNORE NULLS; RESPECT NULLS is the default behavior
-      throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
-        scalarFunc.name(),
-        "IGNORE NULLS"
-      )
-    } else {
-      V2ExpressionUtils.resolveScalarFunction(scalarFunc, arguments)
+    V2ExpressionUtils.bindFunction(unbound, arguments) match {
+      case aggregator: V2Aggregator[_, _] =>
+        processV2AggregateFunction(aggregator, unresolvedFunc)
+      case scalarExpr =>
+        checkUnsupportedAggregateClause(unbound.name(), unresolvedFunc)
+        scalarExpr
     }
   }
 
   private def processV2AggregateFunction(
-      aggFunc: V2AggregateFunction[_, _],
-      arguments: Seq[Expression],
+      aggregator: V2Aggregator[_, _],
       unresolvedFunc: UnresolvedFunction): Expression = {
     // Only fail for IGNORE NULLS; RESPECT NULLS is the default behavior
     if (unresolvedFunc.ignoreNulls.contains(true)) {
       throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
-        aggFunc.name(),
+        aggregator.aggrFunc.name(),
         "IGNORE NULLS"
       )
     }
-    val aggregator = V2Aggregator(aggFunc, arguments)
     aggregator.toAggregateExpression(unresolvedFunc.isDistinct, 
unresolvedFunc.filter)
   }
 
-  private def failAnalysis(errorClass: String, messageParameters: Map[String, 
String]): Nothing = {
-    throw new AnalysisException(
-      errorClass = errorClass,
-      messageParameters = messageParameters)
-  }
-
   /**
    * Resolves [[UnresolvedProcedure]] for `CALL` / `DESCRIBE PROCEDURE` using 
the same multipart
    * candidates as SQL functions and relations ([[resolutionCandidates]] /
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
index 96a36ef59e15..5c58eef2c72e 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
@@ -17,12 +17,14 @@
 
 package org.apache.spark.sql.catalyst.catalog
 
+import java.lang.reflect.InvocationTargetException
 import java.net.URI
 import java.util.Locale
 import java.util.concurrent.{Callable, ExecutionException, TimeUnit}
 import javax.annotation.concurrent.GuardedBy
 
 import scala.collection.mutable
+import scala.util.control.NonFatal
 
 import com.google.common.cache.{Cache, CacheBuilder}
 import com.google.common.util.concurrent.UncheckedExecutionException
@@ -40,6 +42,7 @@ import 
org.apache.spark.sql.catalyst.catalog.SQLFunction.{padArgumentsWithDefaul
 import org.apache.spark.sql.catalyst.expressions.{Alias, Attribute, 
AttributeReference, Cast, Expression, ExpressionInfo, LateralSubquery, 
NamedArgumentExpression, NamedExpression, OuterReference, ScalarSubquery, 
UpCast}
 import org.apache.spark.sql.catalyst.expressions.NamedLambdaVariable
 import org.apache.spark.sql.catalyst.expressions.UnresolvedNamedLambdaVariable
+import org.apache.spark.sql.catalyst.expressions.V2ExpressionUtils
 import org.apache.spark.sql.catalyst.parser.{CatalystSqlParser, 
ParserInterface}
 import org.apache.spark.sql.catalyst.plans.Inner
 import org.apache.spark.sql.catalyst.plans.logical.{FunctionSignature, 
InputParameter, LateralJoin, LogicalPlan, NamedParametersSupport, 
OneRowRelation, Project, SubqueryAlias, View}
@@ -47,6 +50,7 @@ import org.apache.spark.sql.catalyst.trees.CurrentOrigin
 import org.apache.spark.sql.catalyst.util.{CharVarcharUtils, StringUtils}
 import org.apache.spark.sql.connector.catalog.CatalogManager
 import 
org.apache.spark.sql.connector.catalog.CatalogManager.SESSION_CATALOG_NAME
+import org.apache.spark.sql.connector.catalog.functions.UnboundFunction
 import org.apache.spark.sql.errors.{QueryCompilationErrors, 
QueryExecutionErrors}
 import org.apache.spark.sql.internal.SQLConf
 import org.apache.spark.sql.internal.StaticSQLConf.GLOBAL_TEMP_DATABASE
@@ -2073,17 +2077,69 @@ class SessionCatalog(
   }
 
   /**
-   * Constructs a [[FunctionBuilder]] based on the provided function metadata.
+   * Constructs a [[FunctionBuilder]] and the [[ExpressionInfo]] based on the 
provided function
+   * metadata.
    */
-  private def makeFunctionBuilder(func: CatalogFunction): FunctionBuilder = {
+  private def makeFunctionBuilderAndInfo(
+      func: CatalogFunction): (FunctionBuilder, ExpressionInfo) = {
+    val clazz = loadFunctionClass(func)
+    val name = func.identifier.unquotedString
+    if (isV2FunctionClass(clazz)) {
+      // A V2 function: bind it to the arguments at each call site. An 
aggregate function is
+      // returned as a bare `V2Aggregator`, which the caller wraps into an 
`AggregateExpression`
+      // carrying DISTINCT / FILTER. This check goes first so that a class 
also implementing a
+      // Hive UDF interface, for other engines sharing the metastore, runs as 
a V2 function in
+      // Spark. The alias makes errors and column names use the function name, 
like the Hive UDF
+      // wrappers do.
+      val unbound = newV2FunctionInstance(clazz, func.identifier)
+      val builder = (input: Seq[Expression]) => {
+        val expr = V2ExpressionUtils.bindFunction(
+          unbound, input, displayName = Some(func.identifier))
+        expr.setTagValue(FunctionRegistry.FUNC_ALIAS, name)
+        expr
+      }
+      (builder, makeExprInfoForV2Function(func, unbound))
+    } else {
+      val builder = (input: Seq[Expression]) =>
+        functionExpressionBuilder.makeExpression(name, clazz, input)
+      (builder, makeExprInfoForHiveFunction(func))
+    }
+  }
+
+  /**
+   * Loads the class a class-based function (`CREATE FUNCTION ... AS 'class'`) 
refers to.
+   */
+  private def loadFunctionClass(func: CatalogFunction): Class[_] = {
     val className = func.className
     if (!Utils.classIsLoadable(className)) {
       throw QueryCompilationErrors.cannotLoadClassWhenRegisteringFunctionError(
         className, func.identifier)
     }
-    val clazz = Utils.classForName(className)
-    val name = func.identifier.unquotedString
-    (input: Seq[Expression]) => functionExpressionBuilder.makeExpression(name, 
clazz, input)
+    Utils.classForName(className)
+  }
+
+  private def isV2FunctionClass(clazz: Class[_]): Boolean = {
+    classOf[UnboundFunction].isAssignableFrom(clazz)
+  }
+
+  private def newV2FunctionInstance(
+      clazz: Class[_],
+      ident: FunctionIdentifier): UnboundFunction = {
+    try {
+      
clazz.getDeclaredConstructor().newInstance().asInstanceOf[UnboundFunction]
+    } catch {
+      case e: InvocationTargetException =>
+        val cause = e.getCause
+        if (NonFatal(cause) || cause.isInstanceOf[LinkageError]) {
+          throw QueryCompilationErrors.cannotInstantiateFunctionClassError(
+            clazz.getName, ident, cause)
+        } else {
+          throw cause
+        }
+      case NonFatal(e) =>
+        throw QueryCompilationErrors.cannotInstantiateFunctionClassError(
+          clazz.getName, ident, e)
+    }
   }
 
   private def makeUserDefinedScalarFuncBuilder(func: UserDefinedFunction): 
FunctionBuilder = {
@@ -2119,8 +2175,13 @@ class SessionCatalog(
       funcDefinition: CatalogFunction,
       overrideIfExists: Boolean,
       functionBuilder: Option[FunctionBuilder] = None): Unit = {
-    val builder = 
functionBuilder.getOrElse(makeFunctionBuilder(funcDefinition))
-    registerFunction(funcDefinition, overrideIfExists, functionRegistry, 
builder, info = None)
+    functionBuilder match {
+      case Some(builder) =>
+        registerFunction(funcDefinition, overrideIfExists, functionRegistry, 
builder, info = None)
+      case None =>
+        val (builder, info) = makeFunctionBuilderAndInfo(funcDefinition)
+        registerFunction(funcDefinition, overrideIfExists, functionRegistry, 
builder, Some(info))
+    }
   }
 
   private def registerFunction[T](
@@ -2173,6 +2234,28 @@ class SessionCatalog(
     registry.registerFunction(identToRegister, resolvedInfo, functionBuilder)
   }
 
+  private def makeExprInfoForV2Function(
+      func: CatalogFunction,
+      unbound: UnboundFunction): ExpressionInfo = {
+    val description = try {
+      unbound.description()
+    } catch {
+      case NonFatal(_) | _: LinkageError => null
+    }
+    new ExpressionInfo(
+      func.className,
+      func.identifier.database.orNull,
+      func.identifier.funcName,
+      description,
+      "",
+      "",
+      "",
+      "",
+      "",
+      "",
+      "v2_function")
+  }
+
   private def makeExprInfoForHiveFunction(func: CatalogFunction): 
ExpressionInfo = {
     new ExpressionInfo(
       func.className,
@@ -2736,12 +2819,7 @@ class SessionCatalog(
                 makeUserDefinedScalarFuncBuilder(udf))
             } else {
               loadFunctionResources(funcMetadata.resources)
-              registerFunction(
-                funcMetadata,
-                overrideIfExists = false,
-                functionRegistry,
-                makeFunctionBuilder(funcMetadata),
-                info = None)
+              registerFunction(funcMetadata, overrideIfExists = false)
             }
             functionRegistry.lookupFunctionBuilder(qualifiedIdent).get
           }
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/V2ExpressionUtils.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/V2ExpressionUtils.scala
index 2ca5a499b84e..8e1e2be16a9a 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/V2ExpressionUtils.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/V2ExpressionUtils.scala
@@ -22,9 +22,10 @@ import java.lang.reflect.{Method, Modifier}
 import org.apache.spark.internal.Logging
 import org.apache.spark.internal.LogKeys.{FUNCTION_NAME, FUNCTION_PARAM}
 import org.apache.spark.sql.AnalysisException
-import org.apache.spark.sql.catalyst.{InternalRow, SQLConfHelper}
+import org.apache.spark.sql.catalyst.{FunctionIdentifier, InternalRow, 
SQLConfHelper}
 import org.apache.spark.sql.catalyst.analysis.{NoSuchFunctionException, 
UnresolvedAttribute}
 import org.apache.spark.sql.catalyst.encoders.EncoderUtils
+import org.apache.spark.sql.catalyst.expressions.aggregate.V2Aggregator
 import org.apache.spark.sql.catalyst.expressions.objects.{Invoke, StaticInvoke}
 import org.apache.spark.sql.catalyst.plans.logical.{LocalRelation, 
LogicalPlan, SampleMethod}
 import org.apache.spark.sql.catalyst.types.DataTypeUtils.fromAttributes
@@ -210,9 +211,54 @@ object V2ExpressionUtils extends SQLConfHelper with 
Logging {
     case V2SampleMethod.SYSTEM => SampleMethod.System
   }
 
+  /**
+   * Binds a V2 [[UnboundFunction]] to the given arguments and turns the 
resulting
+   * [[BoundFunction]] into a Catalyst expression. The optional display name 
preserves identifier
+   * parts in diagnostics.
+   *
+   * An aggregate function is returned as a bare [[V2Aggregator]] so that 
callers can wrap it into
+   * an `AggregateExpression` carrying DISTINCT / FILTER, which is not known 
here.
+   */
+  def bindFunction(
+      unbound: UnboundFunction,
+      arguments: Seq[Expression],
+      displayName: Option[FunctionIdentifier] = None): Expression = {
+    val inputType = StructType(arguments.zipWithIndex.map {
+      case (exp, pos) => StructField(s"_$pos", exp.dataType, exp.nullable)
+    })
+    val bound = try {
+      unbound.bind(inputType)
+    } catch {
+      case unsupported: UnsupportedOperationException =>
+        throw QueryCompilationErrors.functionCannotProcessInputError(
+          displayName.map(_.unquotedString).getOrElse(unbound.name()), 
arguments, unsupported)
+    }
+
+    val inputTypesLength = bound.inputTypes().length
+    if (inputTypesLength != arguments.length) {
+      throw QueryCompilationErrors.v2FunctionInvalidInputTypeLengthError(
+        displayName.map(_.unquotedString).getOrElse(bound.name()),
+        arguments.length,
+        inputTypesLength)
+    }
+
+    bound match {
+      case scalarFunc: ScalarFunction[_] =>
+        resolveScalarFunction(scalarFunc, arguments, displayName)
+      case aggFunc: AggregateFunction[_, _] =>
+        V2Aggregator(aggFunc, arguments)
+      case _ =>
+        throw new AnalysisException(
+          errorClass = "INVALID_UDF_IMPLEMENTATION",
+          messageParameters = Map(
+            "funcName" -> 
displayName.map(_.quotedString).getOrElse(toSQLId(bound.name()))))
+    }
+  }
+
   def resolveScalarFunction(
       scalarFunc: ScalarFunction[_],
-      arguments: Seq[Expression]): Expression = {
+      arguments: Seq[Expression],
+      displayName: Option[FunctionIdentifier] = None): Expression = {
     val declaredInputTypes = scalarFunc.inputTypes().toImmutableArraySeq
     val argClasses = declaredInputTypes.map(EncoderUtils.dataTypeJavaClass)
     findMethod(scalarFunc, MAGIC_METHOD_NAME, argClasses) match {
@@ -238,7 +284,9 @@ object V2ExpressionUtils extends SQLConfHelper with Logging 
{
           case _ =>
             throw new AnalysisException(
               errorClass = "SCALAR_FUNCTION_NOT_FULLY_IMPLEMENTED",
-              messageParameters = Map("scalarFunc" -> 
toSQLId(scalarFunc.name())))
+              messageParameters = Map(
+                "scalarFunc" -> displayName.map(_.quotedString).getOrElse(
+                  toSQLId(scalarFunc.name()))))
         }
     }
   }
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/package.scala 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/package.scala
index 77e8dcc7d127..5842282c9010 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/package.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/package.scala
@@ -21,8 +21,9 @@ import java.io._
 import java.nio.charset.StandardCharsets.UTF_8
 
 import org.apache.spark.internal.Logging
-import org.apache.spark.sql.catalyst.analysis.TempResolvedColumn
+import org.apache.spark.sql.catalyst.analysis.{FunctionRegistry, 
TempResolvedColumn}
 import org.apache.spark.sql.catalyst.expressions._
+import org.apache.spark.sql.catalyst.expressions.objects.InvokeLike
 import 
org.apache.spark.sql.catalyst.util.FieldMetadataUtils.FIELD_ID_METADATA_KEY
 import org.apache.spark.sql.connector.catalog.MetadataColumn
 import org.apache.spark.sql.types.{MetadataBuilder, NumericType, StringType, 
StructType}
@@ -136,6 +137,10 @@ package object util extends Logging {
       )
     case c: Cast if !c.containsTag(Cast.USER_SPECIFIED_CAST) =>
       PrettyAttribute(usePrettyExpression(c.child, 
shouldTrimTempResolvedColumn).sql, c.dataType)
+    // A V2 function's magic method registered under a function name, e.g. by 
`CREATE FUNCTION`.
+    case i: InvokeLike if i.getTagValue(FunctionRegistry.FUNC_ALIAS).isDefined 
=>
+      val args = i.arguments.map(toPrettySQL(_, shouldTrimTempResolvedColumn))
+      PrettyAttribute(s"${i.prettyName}(${args.mkString(", ")})", i.dataType)
     case j: RoutedSqlJsonExpression =>
       // Column names are for display and never reparsed, so render the clean 
clause-free form, not
       // the canonical `sql` that appends a round-trip-only default clause. 
Render children in place
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
index 181398ca1590..4af1c7d1a13d 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
@@ -35,7 +35,6 @@ import org.apache.spark.sql.catalyst.trees.{Origin, TreeNode}
 import org.apache.spark.sql.catalyst.util.{quoteIdentifier, ParseMode}
 import org.apache.spark.sql.connector.catalog._
 import org.apache.spark.sql.connector.catalog.CatalogV2Implicits._
-import org.apache.spark.sql.connector.catalog.functions.{BoundFunction, 
UnboundFunction}
 import org.apache.spark.sql.connector.expressions.filter.Predicate
 import org.apache.spark.sql.internal.SQLConf
 import org.apache.spark.sql.internal.SQLConf.LEGACY_CTE_PRECEDENCE_POLICY
@@ -1448,6 +1447,17 @@ private[sql] object QueryCompilationErrors extends 
QueryErrorsBase with Compilat
     new FunctionAlreadyExistsException(func.nameParts)
   }
 
+  def cannotInstantiateFunctionClassError(
+      className: String, func: FunctionIdentifier, cause: Throwable): 
Throwable = {
+    new AnalysisException(
+      errorClass = "CANNOT_INSTANTIATE_FUNCTION_CLASS",
+      messageParameters = Map(
+        "className" -> className,
+        "functionName" -> toSQLId(func.toString),
+        "msg" -> Option(cause.getMessage).getOrElse(cause.getClass.getName)),
+      cause = Some(cause))
+  }
+
   def cannotLoadClassWhenRegisteringFunctionError(
       className: String, func: FunctionIdentifier): Throwable = {
     new AnalysisException(
@@ -2794,27 +2804,28 @@ private[sql] object QueryCompilationErrors extends 
QueryErrorsBase with Compilat
   }
 
   def functionCannotProcessInputError(
-      unbound: UnboundFunction,
+      functionName: String,
       arguments: Seq[Expression],
       unsupported: UnsupportedOperationException): Throwable = {
     new AnalysisException(
       errorClass = "_LEGACY_ERROR_TEMP_1198",
       messageParameters = Map(
-        "unbound" -> unbound.name,
+        "unbound" -> functionName,
         "arguments" -> arguments.map(_.dataType.simpleString).mkString(", "),
         "unsupported" -> unsupported.getMessage),
       cause = Some(unsupported))
   }
 
   def v2FunctionInvalidInputTypeLengthError(
-      bound: BoundFunction,
-      args: Seq[Expression]): Throwable = {
+      functionName: String,
+      argsLength: Int,
+      inputTypesLength: Int): Throwable = {
     new AnalysisException(
       errorClass = "_LEGACY_ERROR_TEMP_1199",
       messageParameters = Map(
-        "bound" -> bound.name(),
-        "argsLen" -> args.length.toString,
-        "inputTypesLen" -> bound.inputTypes().length.toString))
+        "bound" -> functionName,
+        "argsLen" -> argsLength.toString,
+        "inputTypesLen" -> inputTypesLength.toString))
   }
 
   def cannotResolveColumnNameAmongAttributesError(
diff --git 
a/sql/core/src/test/java/test/org/apache/spark/sql/connector/catalog/functions/JavaInitTrackingStrLen.java
 
b/sql/core/src/test/java/test/org/apache/spark/sql/connector/catalog/functions/JavaInitTrackingStrLen.java
new file mode 100644
index 000000000000..778346832183
--- /dev/null
+++ 
b/sql/core/src/test/java/test/org/apache/spark/sql/connector/catalog/functions/JavaInitTrackingStrLen.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package test.org.apache.spark.sql.connector.catalog.functions;
+
+import org.apache.spark.sql.catalyst.InternalRow;
+import org.apache.spark.sql.connector.catalog.functions.ScalarFunction;
+import org.apache.spark.sql.connector.catalog.functions.SimpleFunction;
+import org.apache.spark.sql.types.DataType;
+import org.apache.spark.sql.types.DataTypes;
+
+/**
+ * A string length function whose class initialization requires the class
+ * `test.v2udf.UsingResourceStrLen`, which only exists in the function's 
`USING` jar, and records
+ * the initialization in a system property.
+ */
+public class JavaInitTrackingStrLen implements SimpleFunction, 
ScalarFunction<Integer> {
+  static {
+    try {
+      Class.forName(
+        "test.v2udf.UsingResourceStrLen", true, 
Thread.currentThread().getContextClassLoader());
+    } catch (ClassNotFoundException e) {
+      throw new IllegalStateException(e);
+    }
+    System.setProperty("spark.test.JavaInitTrackingStrLen.initialized", 
"true");
+  }
+
+  @Override
+  public DataType[] inputTypes() {
+    return new DataType[] { DataTypes.StringType };
+  }
+
+  @Override
+  public DataType resultType() {
+    return DataTypes.IntegerType;
+  }
+
+  @Override
+  public String name() {
+    return "strlen";
+  }
+
+  @Override
+  public String description() {
+    return "strlen(string) -> int";
+  }
+
+  @Override
+  public Integer produceResult(InternalRow input) {
+    return input.getUTF8String(0).numChars();
+  }
+}
diff --git 
a/sql/core/src/test/java/test/org/apache/spark/sql/connector/catalog/functions/JavaStaticInitFailingStrLen.java
 
b/sql/core/src/test/java/test/org/apache/spark/sql/connector/catalog/functions/JavaStaticInitFailingStrLen.java
new file mode 100644
index 000000000000..c75d75aa1bbe
--- /dev/null
+++ 
b/sql/core/src/test/java/test/org/apache/spark/sql/connector/catalog/functions/JavaStaticInitFailingStrLen.java
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package test.org.apache.spark.sql.connector.catalog.functions;
+
+import org.apache.spark.sql.catalyst.InternalRow;
+import org.apache.spark.sql.connector.catalog.functions.ScalarFunction;
+import org.apache.spark.sql.connector.catalog.functions.SimpleFunction;
+import org.apache.spark.sql.types.DataType;
+import org.apache.spark.sql.types.DataTypes;
+
+/**
+ * A string length function whose class fails to initialize.
+ */
+public class JavaStaticInitFailingStrLen implements SimpleFunction, 
ScalarFunction<Integer> {
+  static {
+    // javac rejects a static initializer that cannot complete normally; `if` 
avoids that.
+    if (true) {
+      throw new IllegalStateException("static initializer failed");
+    }
+  }
+
+  @Override
+  public DataType[] inputTypes() {
+    return new DataType[] { DataTypes.StringType };
+  }
+
+  @Override
+  public DataType resultType() {
+    return DataTypes.IntegerType;
+  }
+
+  @Override
+  public String name() {
+    return "strlen";
+  }
+
+  @Override
+  public String description() {
+    return "strlen(string) -> int";
+  }
+
+  @Override
+  public Integer produceResult(InternalRow input) {
+    return input.getUTF8String(0).numChars();
+  }
+}
diff --git 
a/sql/core/src/test/scala/org/apache/spark/sql/connector/CreateV2FunctionSuite.scala
 
b/sql/core/src/test/scala/org/apache/spark/sql/connector/CreateV2FunctionSuite.scala
new file mode 100644
index 000000000000..675bcb6803ca
--- /dev/null
+++ 
b/sql/core/src/test/scala/org/apache/spark/sql/connector/CreateV2FunctionSuite.scala
@@ -0,0 +1,635 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.connector
+
+import java.io.File
+import java.lang.management.ManagementFactory
+import java.util.concurrent.atomic.AtomicInteger
+
+import scala.reflect.ClassTag
+
+import 
test.org.apache.spark.sql.connector.catalog.functions.{JavaInitTrackingStrLen, 
JavaStaticInitFailingStrLen}
+import 
test.org.apache.spark.sql.connector.catalog.functions.JavaStrLen.{JavaStrLenNoImpl,
 JavaStrLenStaticMagic}
+
+import org.apache.spark.sql.{AnalysisException, QueryTest, Row}
+import org.apache.spark.sql.catalyst.{FunctionIdentifier, InternalRow}
+import org.apache.spark.sql.connector.catalog.functions.{BoundFunction, 
ScalarFunction, SimpleFunction, UnboundFunction}
+import org.apache.spark.sql.test.SharedSparkSession
+import org.apache.spark.sql.types.{DataType, IntegerType, StringType, 
StructType}
+import org.apache.spark.unsafe.types.UTF8String
+import org.apache.spark.util.{SparkTestUtils, Utils}
+
+/**
+ * Tests `CREATE [TEMPORARY] FUNCTION name AS 'class'` where the class is a V2 
function, i.e. it
+ * implements [[UnboundFunction]].
+ */
+class CreateV2FunctionSuite extends QueryTest with SharedSparkSession {
+
+  private val strLenClass = classOf[V2StrLen].getName
+  private val strLenMagicClass = classOf[V2StrLenMagic].getName
+  private val avgClass = classOf[V2IntegralAverage].getName
+
+  // Runs on executors, which must load the class from the jar too.
+  private val jarStrLenQuery = "SELECT jar_strlen(CAST(id AS STRING)) FROM 
range(9, 11)"
+
+  /**
+   * Compiles a V2 string length function named `className` into a jar that is 
not on the
+   * classpath, and returns the jar URI. The jar is named after the class, as 
Spark rejects
+   * registering the same jar name with a different path. With `magic`, the 
function implements
+   * the magic `invoke` method, otherwise `produceResult`. With `requires`, 
initializing the class
+   * loads that class through the context classloader.
+   */
+  private def createStrLenJar(
+      className: String,
+      magic: Boolean = true,
+      requires: Option[String] = None): String = {
+    val dir = Utils.createTempDir()
+    val cp = ManagementFactory.getRuntimeMXBean.getClassPath
+      .split(File.pathSeparator).map(p => new File(p).toURI.toURL).toSeq
+    val simpleName = className.split('.').last
+    val method = if (magic) {
+      "public int invoke(UTF8String s) { return s.numChars(); }"
+    } else {
+      "public Integer produceResult(InternalRow r) { return 
r.getUTF8String(0).numChars(); }"
+    }
+    // scalastyle:off classforname
+    val staticInit = requires.map { required =>
+      s"""static {
+         |    try {
+         |      Class.forName("$required", true, 
Thread.currentThread().getContextClassLoader());
+         |    } catch (ClassNotFoundException e) {
+         |      throw new IllegalStateException(e);
+         |    }
+         |  }""".stripMargin
+    }.getOrElse("")
+    // scalastyle:on classforname
+    val source =
+      s"""package ${className.stripSuffix("." + simpleName)};
+         |
+         |import org.apache.spark.sql.catalyst.InternalRow;
+         |import 
org.apache.spark.sql.connector.catalog.functions.ScalarFunction;
+         |import 
org.apache.spark.sql.connector.catalog.functions.SimpleFunction;
+         |import org.apache.spark.sql.types.DataType;
+         |import org.apache.spark.sql.types.DataTypes;
+         |import org.apache.spark.unsafe.types.UTF8String;
+         |
+         |public class $simpleName implements SimpleFunction, 
ScalarFunction<Integer> {
+         |  $staticInit
+         |  public DataType[] inputTypes() { return new DataType[] { 
DataTypes.StringType }; }
+         |  public DataType resultType() { return DataTypes.IntegerType; }
+         |  public String name() { return "jar_strlen"; }
+         |  public String description() { return "jar_strlen(string) -> int"; }
+         |  $method
+         |}
+         |""".stripMargin
+    SparkTestUtils.createJarWithJavaSources(
+      Map(className -> source), new File(dir, s"$simpleName.jar"), 
cp).toURI.toString
+  }
+
+  test("temporary scalar function") {
+    withUserDefinedFunction("strlen" -> true) {
+      sql(s"CREATE TEMPORARY FUNCTION strlen AS '$strLenClass'")
+      checkAnswer(sql("SELECT strlen('abc')"), Row(3))
+    }
+  }
+
+  test("temporary scalar function with magic method") {
+    withUserDefinedFunction("strlen" -> true) {
+      sql(s"CREATE TEMPORARY FUNCTION strlen AS '$strLenMagicClass'")
+      checkAnswer(sql("SELECT strlen('abcd')"), Row(4))
+    }
+  }
+
+  test("persistent scalar function") {
+    withUserDefinedFunction("default.strlen" -> false) {
+      sql(s"CREATE FUNCTION strlen AS '$strLenClass'")
+      checkAnswer(sql("SELECT strlen('abc')"), Row(3))
+      checkAnswer(sql("SELECT default.strlen('abcde')"), Row(5))
+      checkAnswer(sql("SELECT spark_catalog.default.strlen('ab')"), Row(2))
+    }
+  }
+
+  test("CREATE OR REPLACE FUNCTION") {
+    withUserDefinedFunction("default.f" -> false) {
+      sql(s"CREATE FUNCTION f AS '$strLenClass'")
+      checkAnswer(sql("SELECT f('abc')"), Row(3))
+      sql(s"CREATE OR REPLACE FUNCTION f AS '$avgClass'")
+      checkAnswer(sql("SELECT f(id) FROM VALUES (1), (3) AS t(id)"), Row(2))
+    }
+  }
+
+  test("aggregate function") {
+    withUserDefinedFunction("iavg" -> true) {
+      sql(s"CREATE TEMPORARY FUNCTION iavg AS '$avgClass'")
+      checkAnswer(sql("SELECT iavg(id) FROM VALUES (1), (2), (3), (5) AS 
t(id)"), Row(2))
+      // DISTINCT and FILTER are supported by aggregate functions.
+      checkAnswer(
+        sql("SELECT iavg(DISTINCT id) FROM VALUES (1), (1), (5) AS t(id)"), 
Row(3))
+      checkAnswer(
+        sql("SELECT iavg(id) FILTER (WHERE id > 1) FROM VALUES (1), (2), (4) 
AS t(id)"), Row(3))
+      // The bound function is chosen by the input type.
+      checkAnswer(sql("SELECT iavg(id) FROM VALUES (1L), (4L) AS t(id)"), 
Row(2L))
+    }
+  }
+
+  test("SimpleFunction") {
+    withUserDefinedFunction("strlen" -> true) {
+      sql(s"CREATE TEMPORARY FUNCTION strlen AS 
'${classOf[V2SimpleStrLen].getName}'")
+      checkAnswer(sql("SELECT strlen('abc')"), Row(3))
+      // Without `bind`, the arguments are implicitly cast to the declared 
input types.
+      checkAnswer(sql("SELECT strlen(1234)"), Row(4))
+      val e = intercept[AnalysisException](sql("SELECT strlen('a', 'b')"))
+      assert(e.getCondition == "_LEGACY_ERROR_TEMP_1199")
+    }
+  }
+
+  test("temporary function USING JAR") {
+    val className = "test.v2udf.TempStrLen"
+    assert(!Utils.classIsLoadable(className))
+    val jar = createStrLenJar(className)
+    withUserDefinedFunction("jar_strlen" -> true) {
+      sql(s"CREATE TEMPORARY FUNCTION jar_strlen AS '$className' USING JAR 
'$jar'")
+      checkAnswer(sql(jarStrLenQuery), Row(1) :: Row(2) :: Nil)
+    }
+  }
+
+  test("temporary function after ADD JAR") {
+    val className = "test.v2udf.AddJarStrLen"
+    assert(!Utils.classIsLoadable(className))
+    sql(s"ADD JAR '${createStrLenJar(className)}'")
+    withUserDefinedFunction("jar_strlen" -> true) {
+      sql(s"CREATE TEMPORARY FUNCTION jar_strlen AS '$className'")
+      checkAnswer(sql(jarStrLenQuery), Row(1) :: Row(2) :: Nil)
+    }
+  }
+
+  test("persistent function USING JAR") {
+    val className = "test.v2udf.PersistentStrLen"
+    assert(!Utils.classIsLoadable(className))
+    // Without the magic method, so that the generated code does not refer to 
the class.
+    val jar = createStrLenJar(className, magic = false)
+    Utils.withContextClassLoader(Utils.getSparkClassLoader) {
+      withUserDefinedFunction("default.jar_strlen" -> false) {
+        sql(s"CREATE FUNCTION jar_strlen AS '$className' USING JAR '$jar'")
+        def describe(): Seq[String] =
+          sql("DESCRIBE FUNCTION 
jar_strlen").collect().map(_.getString(0)).toSeq
+        // The jar is loaded on first invocation, so the description is not 
available yet.
+        assert(!describe().exists(_.contains("jar_strlen(string) -> int")))
+
+        checkAnswer(sql(jarStrLenQuery), Row(1) :: Row(2) :: Nil)
+        assert(describe().exists(_.contains("jar_strlen(string) -> int")))
+
+        // Loading the jar switches the context classloader; the function must 
still work
+        // after it is rolled back, as spark-shell does.
+        Thread.currentThread().setContextClassLoader(Utils.getSparkClassLoader)
+        checkAnswer(sql(jarStrLenQuery), Row(1) :: Row(2) :: Nil)
+      }
+    }
+  }
+
+  test("magic method from a runtime jar after the context classloader is 
reset") {
+    val className = "test.v2udf.ResetLoaderStrLen"
+    val jar = createStrLenJar(className)
+    Utils.withContextClassLoader(Utils.getSparkClassLoader) {
+      withUserDefinedFunction("default.jar_strlen" -> false) {
+        sql(s"CREATE FUNCTION jar_strlen AS '$className' USING JAR '$jar'")
+        checkAnswer(sql(jarStrLenQuery), Row(1) :: Row(2) :: Nil)
+        // Resets the context classloader, as spark-shell does after each 
line. Generated code
+        // cannot see the jar then.
+        // TODO(SPARK-60811): the query should succeed once generated code can 
see the jar.
+        Thread.currentThread().setContextClassLoader(Utils.getSparkClassLoader)
+        val e = intercept[Exception](sql(jarStrLenQuery).collect())
+        assert(e.getCause.getMessage.contains("Cannot determine simple type 
name"))
+      }
+    }
+  }
+
+  test("DESCRIBE FUNCTION reports the V2 function description once it is 
loaded") {
+    withUserDefinedFunction("tstrlen" -> true, "default.strlen" -> false) {
+      def describe(name: String): Seq[String] =
+        sql(s"DESCRIBE FUNCTION $name").collect().map(_.getString(0)).toSeq
+
+      sql(s"CREATE TEMPORARY FUNCTION tstrlen AS '$strLenClass'")
+      assert(describe("tstrlen").exists(_.contains("strlen(string) -> int")))
+
+      sql(s"CREATE FUNCTION strlen AS '$strLenClass'")
+      // A persistent function is loaded on first invocation in the session.
+      assert(describe("default.strlen").exists(_.contains(strLenClass)))
+      assert(!describe("default.strlen").exists(_.contains("strlen(string) -> 
int")))
+      checkAnswer(sql("SELECT strlen('abc')"), Row(3))
+      assert(describe("default.strlen").exists(_.contains("strlen(string) -> 
int")))
+    }
+  }
+
+  test("metadata reads do not construct a persistent function before its first 
invocation") {
+    V2MetadataTrackingStrLen.constructions.set(0)
+    val className = classOf[V2MetadataTrackingStrLen].getName
+    withUserDefinedFunction("default.metadata_len" -> false) {
+      sql(s"CREATE FUNCTION metadata_len AS '$className'")
+      assert(spark.catalog.functionExists("metadata_len"))
+      assert(spark.catalog.functionExists("default", "metadata_len"))
+      assert(spark.catalog.listFunctions().collect().exists(_.name == 
"metadata_len"))
+      assert(spark.catalog.getFunction("metadata_len").className == className)
+      sql("DESCRIBE FUNCTION EXTENDED metadata_len").collect()
+      assert(V2MetadataTrackingStrLen.constructions.get() == 0)
+      checkAnswer(sql("SELECT metadata_len('abc')"), Row(3))
+      assert(V2MetadataTrackingStrLen.constructions.get() == 1)
+      assert(spark.catalog.getFunction("metadata_len").className == className)
+      assert(V2MetadataTrackingStrLen.constructions.get() == 1)
+    }
+  }
+
+  test("metadata reads do not initialize the class of a persistent function") {
+    val className = "test.v2udf.RequiresLaterJarStrLen"
+    val requiredClassName = "test.v2udf.LaterJarStrLen"
+    // Initializing the class fails until the jar of the class it requires is 
added. A failed
+    // initialization would make the class unusable for the rest of the JVM's 
life.
+    sql(s"ADD JAR '${createStrLenJar(className, requires = 
Some(requiredClassName))}'")
+    withUserDefinedFunction("default.metadata_len" -> false) {
+      sql(s"CREATE FUNCTION metadata_len AS '$className'")
+      assert(spark.catalog.listFunctions().collect().exists(_.name == 
"metadata_len"))
+      assert(spark.catalog.getFunction("metadata_len").description == "N/A.")
+      sql("DESCRIBE FUNCTION EXTENDED metadata_len").collect()
+
+      sql(s"ADD JAR '${createStrLenJar(requiredClassName)}'")
+      checkAnswer(sql("SELECT metadata_len('abc')"), Row(3))
+      assert(spark.catalog.getFunction("metadata_len").description == 
"jar_strlen(string) -> int")
+    }
+  }
+
+  gridTest("class whose description throws")(
+      Seq(classOf[V2FunctionThrowingDescription].getName,
+        classOf[V2FunctionLinkageErrorDescription].getName)) { className =>
+    withUserDefinedFunction("tstrlen" -> true, "default.pstrlen" -> false) {
+      sql(s"CREATE TEMPORARY FUNCTION tstrlen AS '$className'")
+      sql(s"CREATE FUNCTION pstrlen AS '$className'")
+
+      def checkUnavailableDescription(name: String): Unit = {
+        val output = sql(s"DESCRIBE FUNCTION 
$name").collect().map(_.getString(0))
+        assert(output.contains(s"Class: $className"))
+        assert(output.contains("Usage: N/A."))
+        assert(spark.catalog.getFunction(name).className == className)
+        val info = 
spark.sessionState.catalog.lookupFunctionInfo(FunctionIdentifier(name))
+        assert(info.getSource == "v2_function")
+      }
+
+      checkUnavailableDescription("tstrlen")
+      checkAnswer(sql("SELECT tstrlen(CAST(id AS STRING)) FROM range(9, 11)"),
+        Row(1) :: Row(2) :: Nil)
+      checkAnswer(sql("SELECT pstrlen(CAST(id AS STRING)) FROM range(9, 11)"),
+        Row(1) :: Row(2) :: Nil)
+      checkUnavailableDescription("pstrlen")
+
+      sql("REFRESH FUNCTION pstrlen")
+      checkAnswer(sql("SELECT pstrlen('abc')"), Row(3))
+      checkUnavailableDescription("pstrlen")
+    }
+  }
+
+  test("DROP FUNCTION") {
+    sql(s"CREATE FUNCTION strlen AS '$strLenClass'")
+    checkAnswer(sql("SELECT strlen('abc')"), Row(3))
+    sql("DROP FUNCTION strlen")
+    intercept[AnalysisException](sql("SELECT strlen('abc')"))
+  }
+
+  test("unsupported clauses and column names use the function name") {
+    withUserDefinedFunction("strlen" -> true, "iavg" -> true, 
"default.pstrlen" -> false) {
+      sql(s"CREATE TEMPORARY FUNCTION strlen AS '$strLenClass'")
+      sql(s"CREATE TEMPORARY FUNCTION iavg AS '$avgClass'")
+      sql(s"CREATE FUNCTION pstrlen AS '$strLenClass'")
+      assert(sql("SELECT strlen('a'), pstrlen('a'), iavg(id) FROM 
range(3)").columns.toSeq ==
+        Seq("strlen(a)", "spark_catalog.default.pstrlen(a)", "iavg(id)"))
+
+      def checkUnsupported(call: String, name: String, syntax: String): Unit = 
{
+        checkError(
+          exception = intercept[AnalysisException](sql(s"SELECT $call FROM 
range(3)")),
+          condition = "INVALID_SQL_SYNTAX.FUNCTION_WITH_UNSUPPORTED_SYNTAX",
+          parameters = Map("prettyName" -> name, "syntax" -> syntax),
+          context = ExpectedContext(call, 7, 6 + call.length))
+      }
+      checkUnsupported("strlen(DISTINCT 'a')", "`strlen`", "DISTINCT")
+      checkUnsupported("pstrlen(DISTINCT 'a')", 
"`spark_catalog`.`default`.`pstrlen`", "DISTINCT")
+      checkUnsupported("strlen('a') FILTER (WHERE true)", "`strlen`", "FILTER 
CLAUSE")
+      checkUnsupported("iavg(id) IGNORE NULLS", "`iavg`", "IGNORE NULLS")
+      checkUnsupported(
+        "iavg(id) WITHIN GROUP (ORDER BY id)", "`iavg`", "WITHIN GROUP (ORDER 
BY ...)")
+    }
+  }
+
+  test("column names of magic methods use the function name") {
+    val simpleMagicClass = classOf[V2SimpleStrLenMagic].getName
+    withUserDefinedFunction(
+        "strlen" -> true, "sstrlen" -> true, "simple_strlen" -> true, 
"default.pstrlen" -> false) {
+      sql(s"CREATE TEMPORARY FUNCTION strlen AS '$strLenMagicClass'")
+      sql(s"CREATE TEMPORARY FUNCTION sstrlen AS 
'${classOf[V2StrLenStaticMagic].getName}'")
+      sql(s"CREATE TEMPORARY FUNCTION simple_strlen AS '$simpleMagicClass'")
+      sql(s"CREATE FUNCTION pstrlen AS '$strLenMagicClass'")
+      val df = sql("SELECT strlen('a'), sstrlen('a'), simple_strlen('a'), 
pstrlen('a')")
+      assert(df.columns.toSeq ==
+        Seq("strlen(a)", "sstrlen(a)", "simple_strlen(a)", 
"spark_catalog.default.pstrlen(a)"))
+
+      // The column name does not depend on the function instance.
+      withTempView("v") {
+        sql("CREATE TEMPORARY VIEW v AS SELECT simple_strlen(s) FROM VALUES 
('abc') t(s)")
+        sql(s"CREATE OR REPLACE TEMPORARY FUNCTION simple_strlen AS 
'$simpleMagicClass'")
+        checkAnswer(spark.table("v"), Row(3))
+      }
+    }
+  }
+
+  gridTest("bind-time errors use the registered function name")(for {
+    isTemp <- Seq(true, false)
+    funcName <- Seq("my_len", "my.len", "my`len")
+  } yield (isTemp, funcName)) { case (isTemp, funcName) =>
+    val ident = FunctionIdentifier(funcName, if (isTemp) None else 
Some("default"))
+    val name = ident.quotedString
+    val registeredIdent = if (isTemp) ident else ident.copy(catalog = 
Some("spark_catalog"))
+    val registeredName = registeredIdent.unquotedString
+    val quotedName = registeredIdent.quotedString
+    val create = if (isTemp) "CREATE TEMPORARY FUNCTION" else "CREATE FUNCTION"
+
+    def checkBindingError(
+        className: String,
+        args: String,
+        condition: String,
+        parameters: Map[String, String]): Unit = {
+      withUserDefinedFunction(name -> isTemp) {
+        sql(s"$create $name AS '$className'")
+        val call = s"$name($args)"
+        checkError(
+          exception = intercept[AnalysisException](sql(s"SELECT $call")),
+          condition = condition,
+          parameters = parameters,
+          context = ExpectedContext(call, 7, 6 + call.length))
+      }
+    }
+
+    checkBindingError(strLenClass, "1", "_LEGACY_ERROR_TEMP_1198",
+      Map("unbound" -> registeredName, "arguments" -> "int", "unsupported" -> 
"Expect StringType"))
+    checkBindingError(classOf[V2SimpleStrLen].getName, "'a', 'b'", 
"_LEGACY_ERROR_TEMP_1199",
+      Map("bound" -> registeredName, "argsLen" -> "2", "inputTypesLen" -> "1"))
+    checkBindingError(classOf[V2StrLenBadBound].getName, "'abc'", 
"INVALID_UDF_IMPLEMENTATION",
+      Map("funcName" -> quotedName))
+    checkBindingError(
+      classOf[V2StrLenNoImpl].getName, "'abc'", 
"SCALAR_FUNCTION_NOT_FULLY_IMPLEMENTED",
+      Map("scalarFunc" -> quotedName))
+  }
+
+  test("input type that the function cannot bind") {
+    withUserDefinedFunction("strlen" -> true) {
+      sql(s"CREATE TEMPORARY FUNCTION strlen AS '$strLenClass'")
+      val e = intercept[AnalysisException](sql("SELECT strlen(1)"))
+      assert(e.getMessage.contains("Expect StringType"))
+      val e2 = intercept[AnalysisException](sql("SELECT strlen('a', 'b')"))
+      assert(e2.getMessage.contains("Expect exactly one argument"))
+    }
+  }
+
+  /**
+   * Checks that a broken persistent function `f` can be created, listed and 
described like a
+   * Hive UDF, and only fails when invoked.
+   */
+  private def checkFailsOnlyWhenInvoked[T <: Throwable : ClassTag](createSql: 
String)(
+      check: T => Unit): Unit = {
+    withUserDefinedFunction("default.f" -> false) {
+      sql(createSql)
+      assert(sql("SHOW USER FUNCTIONS").collect().map(_.getString(0)).contains(
+        "spark_catalog.default.f"))
+      assert(spark.catalog.listFunctions().collect().exists(_.name == "f"))
+      assert(spark.catalog.getFunction("f").className == 
createSql.split("'")(1))
+      sql("DESCRIBE FUNCTION EXTENDED f").collect()
+      check(intercept[T](sql("SELECT f('abc')").collect()))
+    }
+  }
+
+  /** Checks that a class that cannot be instantiated fails as 
`CANNOT_INSTANTIATE_...`. */
+  private def checkCannotInstantiate(className: String, msg: String): Unit = {
+    checkError(
+      exception = intercept[AnalysisException] {
+        sql(s"CREATE TEMPORARY FUNCTION f AS '$className'")
+      },
+      condition = "CANNOT_INSTANTIATE_FUNCTION_CLASS",
+      parameters = Map("className" -> className, "functionName" -> "`f`", 
"msg" -> msg))
+
+    checkFailsOnlyWhenInvoked[AnalysisException](s"CREATE FUNCTION f AS 
'$className'") { e =>
+      checkError(
+        exception = e,
+        condition = "CANNOT_INSTANTIATE_FUNCTION_CLASS",
+        parameters = Map(
+          "className" -> className,
+          "functionName" -> "`spark_catalog`.`default`.`f`",
+          "msg" -> msg),
+        context = ExpectedContext("f('abc')", 7, 14))
+    }
+  }
+
+  test("class without a public no-arg constructor") {
+    val className = classOf[V2FunctionWithoutNoArgConstructor].getName
+    checkCannotInstantiate(className, s"$className.<init>()")
+  }
+
+  test("class whose constructor throws") {
+    checkCannotInstantiate(classOf[V2FunctionThrowingConstructor].getName, 
"constructor failed")
+  }
+
+  test("class whose constructor throws a linkage error") {
+    checkCannotInstantiate(
+      classOf[V2FunctionLinkageErrorConstructor].getName, "constructor 
dependency")
+  }
+
+  gridTest("fatal constructor failures propagate")(
+      Seq(
+        (classOf[V2FunctionOutOfMemoryConstructor].getName, 
classOf[OutOfMemoryError]),
+        (classOf[V2FunctionStackOverflowConstructor].getName, 
classOf[StackOverflowError]),
+        (classOf[V2FunctionInterruptedConstructor].getName, 
classOf[InterruptedException]))) {
+    case (className, errorClass) =>
+      def checkFatal(operation: => Any): Unit = {
+        val e = intercept[Throwable](operation)
+        assert(e.getClass == errorClass)
+        assert(e.getMessage == "constructor failed")
+      }
+
+      checkFatal(sql(s"CREATE TEMPORARY FUNCTION f AS '$className'"))
+      withUserDefinedFunction("default.f" -> false) {
+        sql(s"CREATE FUNCTION f AS '$className'")
+        sql("DESCRIBE FUNCTION f").collect()
+        spark.catalog.getFunction("f")
+        spark.catalog.listFunctions().collect()
+        checkFatal(sql("SELECT f('abc')").collect())
+        checkFatal(sql("REFRESH FUNCTION f"))
+      }
+  }
+
+  test("class that fails to initialize") {
+    val className = classOf[JavaStaticInitFailingStrLen].getName
+    checkFailsOnlyWhenInvoked[LinkageError](s"CREATE FUNCTION f AS 
'$className'") { _ => }
+  }
+
+  test("USING resources are loaded before the class is initialized") {
+    val className = classOf[JavaInitTrackingStrLen].getName
+    val initialized = "spark.test.JavaInitTrackingStrLen.initialized"
+    // Initializing the class requires a class in this jar, so it fails if 
done before the jar
+    // is loaded.
+    val jar = createStrLenJar("test.v2udf.UsingResourceStrLen")
+    withUserDefinedFunction("default.f" -> false) {
+      sql(s"CREATE FUNCTION f AS '$className' USING JAR '$jar'")
+      assert(spark.catalog.functionExists("f"))
+      spark.catalog.listFunctions().collect()
+      spark.catalog.getFunction("f")
+      val describe = sql("DESCRIBE FUNCTION f").collect().map(_.getString(0))
+      assert(!describe.exists(_.contains("strlen(string) -> int")))
+      assert(System.getProperty(initialized) == null)
+
+      checkAnswer(sql("SELECT f('abc')"), Row(3))
+      assert(System.getProperty(initialized) == "true")
+      assert(sql("DESCRIBE FUNCTION f").collect().map(_.getString(0))
+        .exists(_.contains("strlen(string) -> int")))
+    }
+  }
+
+  test("class that is not on the classpath") {
+    checkFailsOnlyWhenInvoked[AnalysisException]("CREATE FUNCTION f AS 
'not.a.Class'") { e =>
+      checkError(
+        exception = e,
+        condition = "CANNOT_LOAD_FUNCTION_CLASS",
+        parameters = Map(
+          "className" -> "not.a.Class",
+          "functionName" -> "`spark_catalog`.`default`.`f`"),
+        context = ExpectedContext("f('abc')", 7, 14))
+    }
+  }
+
+  test("jar that is not accessible") {
+    val jar = new File(Utils.createTempDir(), "missing.jar").toURI.toString
+    checkFailsOnlyWhenInvoked[AnalysisException](
+        s"CREATE FUNCTION f AS 'test.v2udf.MissingJarStrLen' USING JAR 
'$jar'") { e =>
+      assert(e.getCondition == "CANNOT_LOAD_FUNCTION_CLASS", e.getMessage)
+    }
+  }
+}
+
+class V2StrLen extends UnboundFunction {
+  override def name(): String = "strlen"
+
+  override def bind(inputType: StructType): BoundFunction = {
+    if (inputType.fields.length != 1) {
+      throw new UnsupportedOperationException("Expect exactly one argument")
+    }
+    inputType.fields(0).dataType match {
+      case _: StringType => StrLenDefault
+      case _ => throw new UnsupportedOperationException("Expect StringType")
+    }
+  }
+
+  override def description(): String = "strlen: strlen(string) -> int"
+}
+
+object V2MetadataTrackingStrLen {
+  val constructions = new AtomicInteger
+}
+
+class V2MetadataTrackingStrLen extends V2StrLen {
+  V2MetadataTrackingStrLen.constructions.incrementAndGet()
+}
+
+class V2StrLenMagic extends V2StrLen {
+  override def bind(inputType: StructType): BoundFunction = {
+    super.bind(inputType)
+    StrLenMagic
+  }
+}
+
+class V2StrLenBadBound extends V2StrLen {
+  override def bind(inputType: StructType): BoundFunction = BadBoundFunction
+}
+
+class V2StrLenNoImpl extends V2StrLen {
+  override def bind(inputType: StructType): BoundFunction = new 
JavaStrLenNoImpl
+}
+
+class V2SimpleStrLen extends SimpleFunction with ScalarFunction[Int] {
+  override def inputTypes(): Array[DataType] = Array(StringType)
+  override def resultType(): DataType = IntegerType
+  override def name(): String = "strlen"
+  override def description(): String = "strlen: strlen(string) -> int"
+  override def produceResult(input: InternalRow): Int = 
input.getUTF8String(0).numChars()
+}
+
+class V2StrLenStaticMagic extends V2StrLen {
+  override def bind(inputType: StructType): BoundFunction = {
+    super.bind(inputType)
+    new JavaStrLenStaticMagic
+  }
+}
+
+class V2SimpleStrLenMagic extends SimpleFunction with ScalarFunction[Int] {
+  override def inputTypes(): Array[DataType] = Array(StringType)
+  override def resultType(): DataType = IntegerType
+  override def name(): String = "strlen"
+  override def description(): String = "strlen: strlen(string) -> int"
+  def invoke(s: UTF8String): Int = s.numChars()
+}
+
+class V2IntegralAverage extends UnboundFunction {
+  override def name(): String = IntegralAverage.name()
+  override def bind(inputType: StructType): BoundFunction = 
IntegralAverage.bind(inputType)
+  override def description(): String = IntegralAverage.description()
+}
+
+class V2FunctionThrowingDescription extends V2StrLen {
+  override def description(): String = throw new 
UnsupportedOperationException("description failed")
+}
+
+class V2FunctionLinkageErrorDescription extends V2StrLen {
+  // scalastyle:off throwerror
+  override def description(): String = throw new 
NoClassDefFoundError("description dependency")
+  // scalastyle:on throwerror
+}
+
+class V2FunctionThrowingConstructor extends V2StrLen {
+  throw new IllegalStateException("constructor failed")
+}
+
+class V2FunctionLinkageErrorConstructor extends V2StrLen {
+  // scalastyle:off throwerror
+  throw new NoClassDefFoundError("constructor dependency")
+  // scalastyle:on throwerror
+}
+
+class V2FunctionOutOfMemoryConstructor extends V2StrLen {
+  // scalastyle:off throwerror
+  throw new OutOfMemoryError("constructor failed")
+  // scalastyle:on throwerror
+}
+
+class V2FunctionStackOverflowConstructor extends V2StrLen {
+  // scalastyle:off throwerror
+  throw new StackOverflowError("constructor failed")
+  // scalastyle:on throwerror
+}
+
+class V2FunctionInterruptedConstructor extends V2StrLen {
+  throw new InterruptedException("constructor failed")
+}
+
+class V2FunctionWithoutNoArgConstructor(name: String) extends UnboundFunction {
+  override def name(): String = name
+  override def bind(inputType: StructType): BoundFunction = StrLenDefault
+  override def description(): String = name
+}
diff --git 
a/sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
 
b/sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
index 7b5402119b40..49d242ad4511 100644
--- 
a/sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
+++ 
b/sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
@@ -83,7 +83,7 @@ class ExpressionInfoSuite extends SharedSparkSession {
 
     val validSources = Seq(
       "built-in", "hive", "python_udf", "scala_udf", "java_udf", 
"python_udtf", "internal",
-      "sql_udf")
+      "sql_udf", "v2_function")
     validSources.foreach { source =>
       val info = new ExpressionInfo(
         "testClass", null, "testName", null, "", "", "", "", "", "", source)
diff --git 
a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala
 
b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala
index 89edb53048a3..5b5474512f29 100644
--- 
a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala
+++ 
b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala
@@ -42,9 +42,10 @@ import org.apache.hadoop.io.{LongWritable, Writable}
 import org.apache.spark.{SparkException, SparkFiles, SparkRuntimeException, 
TestUtils}
 import org.apache.spark.sql.{AnalysisException, QueryTest, Row}
 import org.apache.spark.sql.catalyst.InternalRow
-import org.apache.spark.sql.catalyst.expressions.{AttributeReference, 
BindReferences, CodegenObjectFactoryMode, Literal}
+import org.apache.spark.sql.catalyst.expressions.{ApplyFunctionExpression, 
AttributeReference, BindReferences, CodegenObjectFactoryMode, Literal}
 import org.apache.spark.sql.catalyst.plans.logical.{Filter, Project}
 import org.apache.spark.sql.catalyst.util.{DateTimeUtils, GenericArrayData}
+import org.apache.spark.sql.connector.catalog.functions.{ScalarFunction, 
SimpleFunction}
 import org.apache.spark.sql.execution.WholeStageCodegenExec
 import org.apache.spark.sql.functions.{call_function, max}
 import org.apache.spark.sql.hive.{HiveGenericUDF, HiveGenericUDTF}
@@ -1250,6 +1251,38 @@ class HiveUDFSuite extends QueryTest with 
TestHiveSingleton {
     }
     hiveContext.reset()
   }
+
+  test("class implementing both GenericUDF and a V2 function runs as a V2 
function") {
+    withUserDefinedFunction("dual_strlen" -> false) {
+      sql(s"CREATE FUNCTION dual_strlen AS 
'${classOf[GenericUDFAndV2StrLen].getName}'")
+      val df = sql("SELECT dual_strlen('abc')")
+      val analyzed = df.queryExecution.analyzed
+      
assert(!analyzed.exists(_.expressions.exists(_.exists(_.isInstanceOf[HiveGenericUDF]))))
+      assert(analyzed.exists(_.expressions.exists(_.exists(
+        _.isInstanceOf[ApplyFunctionExpression]))))
+      checkAnswer(df, Row(3))
+    }
+  }
+}
+
+/**
+ * A function that runs in Hive as a [[GenericUDF]] and in Spark as a V2 
function.
+ */
+class GenericUDFAndV2StrLen extends GenericUDF with SimpleFunction with 
ScalarFunction[Int] {
+  override def initialize(arguments: Array[ObjectInspector]): ObjectInspector =
+    PrimitiveObjectInspectorFactory.javaIntObjectInspector
+
+  override def evaluate(arguments: Array[DeferredObject]): AnyRef =
+    Int.box(arguments(0).get().toString.length)
+
+  override def getDisplayString(children: Array[String]): String =
+    s"dual_strlen(${children.mkString(", ")})"
+
+  override def inputTypes(): Array[DataType] = Array(StringType)
+  override def resultType(): DataType = IntegerType
+  override def name(): String = "dual_strlen"
+  override def description(): String = "dual_strlen(string) -> int"
+  override def produceResult(input: InternalRow): Int = 
input.getUTF8String(0).numChars()
 }
 
 class TestPair(x: Int, y: Int) extends Writable with Serializable {


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to