[
https://issues.apache.org/jira/browse/SPARK-56764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pin_zhang updated SPARK-56764:
------------------------------
Description:
1. ./sbin/start-thriftserver.sh with async=false and create a UDF
spark.sql.hive.thriftServer.async false
spark.jars file:///home/test/spark-4.1.1-bin-hadoop3-connect/custom/udf.jar
CREATE FUNCTION IF NOT EXISTS hello AS 'com.test.HelloUDF';
2. restart thriftserver
./bin/beeline -u jdbc:hive2://localhost:10000 -e 'select 1;'
./bin/beeline -u jdbc:hive2://localhost:10000 -e 'select hello("a");'
Connecting to jdbc:hive2://localhost:10000
Connected to: Spark SQL (version 4.1.1)
Driver: Hive JDBC (version 2.3.10)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Error: Error running query: [CANNOT_LOAD_FUNCTION_CLASS]
org.apache.spark.sql.AnalysisException: [CANNOT_LOAD_FUNCTION_CLASS] Cannot
load class com.test.HelloUDF when registering the function
`spark_catalog`.`default`.`hello`, please make sure it is on the classpath.
SQLSTATE: 46103; line 1 pos 7 (state=46103,code=0)
Closing: 0: jdbc:hive2://localhost:10000
3. why set async=false, because pyhive client collect to hive in sync mode
was:
spark.sql.hive.thriftServer.async false
spark.jars file:///home/test/spark-4.1.1-bin-hadoop3-connect/custom/udf.jar
CREATE FUNCTION IF NOT EXISTS hello AS 'com.test.HelloUDF';
> CANNOT_LOAD_FUNCTION_CLASS in a new session
> -------------------------------------------
>
> Key: SPARK-56764
> URL: https://issues.apache.org/jira/browse/SPARK-56764
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.1.1
> Reporter: pin_zhang
> Priority: Major
> Attachments: udf.jar
>
>
> 1. ./sbin/start-thriftserver.sh with async=false and create a UDF
> spark.sql.hive.thriftServer.async false
> spark.jars
> file:///home/test/spark-4.1.1-bin-hadoop3-connect/custom/udf.jar
> CREATE FUNCTION IF NOT EXISTS hello AS 'com.test.HelloUDF';
> 2. restart thriftserver
> ./bin/beeline -u jdbc:hive2://localhost:10000 -e 'select 1;'
> ./bin/beeline -u jdbc:hive2://localhost:10000 -e 'select hello("a");'
> Connecting to jdbc:hive2://localhost:10000
> Connected to: Spark SQL (version 4.1.1)
> Driver: Hive JDBC (version 2.3.10)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Error: Error running query: [CANNOT_LOAD_FUNCTION_CLASS]
> org.apache.spark.sql.AnalysisException: [CANNOT_LOAD_FUNCTION_CLASS]
> Cannot load class com.test.HelloUDF when registering the function
> `spark_catalog`.`default`.`hello`, please make sure it is on the classpath.
> SQLSTATE: 46103; line 1 pos 7 (state=46103,code=0)
> Closing: 0: jdbc:hive2://localhost:10000
> 3. why set async=false, because pyhive client collect to hive in sync mode
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
