This is an automated email from the ASF dual-hosted git repository. yao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 7baf9dad196 [SPARK-44719][SQL] Fix NoClassDefFoundError when using Hive UDF 7baf9dad196 is described below commit 7baf9dad1962e7e44138c0b66422cca6e347f800 Author: Yuming Wang <yumw...@ebay.com> AuthorDate: Sat Aug 12 11:41:22 2023 +0800 [SPARK-44719][SQL] Fix NoClassDefFoundError when using Hive UDF ### What changes were proposed in this pull request? This PR changes jackson-mapper-asl's scope from `test` to `${hive.deps.scope}`. ### Why are the changes needed? Fix `NoClassDefFoundError` when using Hive UDF: ``` spark-sql (default)> add jar /Users/yumwang/Downloads/HiveUDFs-1.0-SNAPSHOT.jar; Time taken: 0.413 seconds spark-sql (default)> CREATE TEMPORARY FUNCTION long_to_ip as 'net.petrabarus.hiveudfs.LongToIP'; Time taken: 0.038 seconds spark-sql (default)> SELECT long_to_ip(2130706433L) FROM range(10); 23/08/08 20:17:58 ERROR SparkSQLDriver: Failed in [SELECT long_to_ip(2130706433L) FROM range(10)] java.lang.NoClassDefFoundError: org/codehaus/jackson/map/type/TypeFactory at org.apache.hadoop.hive.ql.udf.UDFJson.<clinit>(UDFJson.java:64) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) ... ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? manual test. Closes #42446 from wangyum/SPARK-44719. Authored-by: Yuming Wang <yumw...@ebay.com> Signed-off-by: Kent Yao <y...@apache.org> --- dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 ++ pom.xml | 8 +++++++- sql/hive-thriftserver/pom.xml | 9 --------- 3 files changed, 9 insertions(+), 10 deletions(-) diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3 index 2a5a6741bdd..3e8131b2734 100644 --- a/dev/deps/spark-deps-hadoop-3-hive-2.3 +++ b/dev/deps/spark-deps-hadoop-3-hive-2.3 @@ -99,11 +99,13 @@ ini4j/0.5.4//ini4j-0.5.4.jar istack-commons-runtime/3.0.8//istack-commons-runtime-3.0.8.jar ivy/2.5.1//ivy-2.5.1.jar jackson-annotations/2.15.2//jackson-annotations-2.15.2.jar +jackson-core-asl/1.9.13//jackson-core-asl-1.9.13.jar jackson-core/2.15.2//jackson-core-2.15.2.jar jackson-databind/2.15.2//jackson-databind-2.15.2.jar jackson-dataformat-cbor/2.15.2//jackson-dataformat-cbor-2.15.2.jar jackson-dataformat-yaml/2.15.2//jackson-dataformat-yaml-2.15.2.jar jackson-datatype-jsr310/2.15.2//jackson-datatype-jsr310-2.15.2.jar +jackson-mapper-asl/1.9.13//jackson-mapper-asl-1.9.13.jar jackson-module-scala_2.12/2.15.2//jackson-module-scala_2.12-2.15.2.jar jakarta.annotation-api/1.3.5//jakarta.annotation-api-1.3.5.jar jakarta.inject/2.6.1//jakarta.inject-2.6.1.jar diff --git a/pom.xml b/pom.xml index 30357f1b27f..f25fb6b5f62 100644 --- a/pom.xml +++ b/pom.xml @@ -1834,11 +1834,17 @@ </exclusions> </dependency> <!-- Hive 2.3 need this to init Hive's FunctionRegistry --> + <dependency> + <groupId>org.codehaus.jackson</groupId> + <artifactId>jackson-core-asl</artifactId> + <version>${codehaus.jackson.version}</version> + <scope>${hive.deps.scope}</scope> + </dependency> <dependency> <groupId>org.codehaus.jackson</groupId> <artifactId>jackson-mapper-asl</artifactId> <version>${codehaus.jackson.version}</version> - <scope>test</scope> + <scope>${hive.deps.scope}</scope> </dependency> <dependency> <groupId>${hive.group}</groupId> diff --git a/sql/hive-thriftserver/pom.xml b/sql/hive-thriftserver/pom.xml index 3d9f6e3b2e9..76a1037f1cb 100644 --- a/sql/hive-thriftserver/pom.xml +++ b/sql/hive-thriftserver/pom.xml @@ -147,15 +147,6 @@ <groupId>org.apache.httpcomponents</groupId> <artifactId>httpcore</artifactId> </dependency> - <!-- - SPARK-44601: Add this test dependency to ensure that `hive-thriftserver` module - can be tested using Maven - --> - <dependency> - <groupId>org.codehaus.jackson</groupId> - <artifactId>jackson-mapper-asl</artifactId> - <scope>test</scope> - </dependency> </dependencies> <build> <outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org