[
https://issues.apache.org/jira/browse/IMPALA-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-11342:
------------------------------------
Affects Version/s: Impala 4.1.0
> Hive UDFs are unable to use classes from the same Jar in catalogd
> -----------------------------------------------------------------
>
> Key: IMPALA-11342
> URL: https://issues.apache.org/jira/browse/IMPALA-11342
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 4.1.0
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Major
> Fix For: Impala 4.2.0
>
>
> While it is a known limitations that Hive UDFs in Impala cannot load
> dependencies from classpath, they should be able to load classes from the
> same jar (and thus creating an shaded jar with the dependencies should be a
> valid workaround). Meanwhile I got an error when the parent class of the UDF
> was a class defined in another package in the jar.
> The exception occurred when catalogd tried to instantiate the UDF class:
> {code}
> I0608 14:50:21.520296 22464 jni-util.cc:286] java.lang.NoClassDefFoundError:
> com/esri/core/geometry/Point
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.getConstructor(Class.java:1825)
> at
> org.apache.impala.hive.executor.HiveLegacyJavaFunction.instantiateUDFInstance(HiveLegacyJavaFunction.java:115)
> at
> org.apache.impala.hive.executor.HiveLegacyJavaFunction.<init>(HiveLegacyJavaFunction.java:74)
> at
> org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:50)
> at
> org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:67)
> at
> org.apache.impala.service.CatalogOpExecutor.createFunction(CatalogOpExecutor.java:2117)
> at
> org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:453)
> at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:245)
> Caused by: java.lang.ClassNotFoundException: com.esri.core.geometry.Point
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 11 more
> {code}
> The issue seems to be cause by closing the UrlClassLoader too early, after
> loading the class but before doing the actual instantiation:
> https://github.com/apache/impala/blob/13bbff4e4e5fc5d459cc6f7a5512f84ceba897cd/fe/src/main/java/org/apache/impala/hive/executor/HiveUdfLoader.java#L71
> Note that this is an issue only in the catalogd, not the executors, as the
> class loader is kept open longer in that case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
