Hi folks,

I’m running into a strange ClassNotFound error while trying to use one of my 
Hive UDFs in Impala.
I’ve defined the UDF:
> create function device_type(string, string, string) returns string location 
> 'my/path/to/my-udf-SNAPSHOT.jar' 
> symbol='com.criteo.hadoop.hive.udf.UDFUserAgentToDeviceType';

I try to use it in simple queries and it works fine:
> select device_type("a", "b", "c");
Query: select device_type("a", "b", "c")
…
+----------------------------------------+
| device_type('a', 'b', 'c') |
+----------------------------------------+
| Mobile - Other                         |
+----------------------------------------+
Fetched 1 row(s) in 0.01s
> select ua_device_family, ua_browser_family, ua_os_family, 
> device_type(ua_device_family, ua_browser_family, ua_os_family) from 
> bi_arbitrage_full limit 1;
Query: select ua_device_family, ua_browser_family, ua_os_family, 
device_type(ua_device_family, ua_browser_family, ua_os_family) from 
bi_arbitrage_full limit 1
…
+------------------+-------------------+--------------+----------------------------------------------------------------------------+
| ua_device_family | ua_browser_family | ua_os_family | 
device_type(ua_device_family, ua_browser_family, ua_os_family) |
+------------------+-------------------+--------------+----------------------------------------------------------------------------+
| iPhone           | mobile safari     | iOS          | iPhone                  
                                                   |
+------------------+-------------------+--------------+----------------------------------------------------------------------------+
Fetched 1 row(s) in 0.73s

Now when I try to run a more complex query (same database)
I get an error:
WARNINGS: ImpalaRuntimeException: Unable to find class.
CAUSED BY: ClassNotFoundException: 
com.criteo.hadoop.hive.udf.UDFUserAgentToDeviceType

I turned up the log level on the coordinator to debug. I see the calls being 
made to load the UDF and the use of the UDF in the query but no details on the 
ClassNotFound.

Has anyone run into a similar issue? We’re running Impala 2.11.0-cdh5.14.0 
RELEASE (build d68206561bce6b26762d62c01a78e6cd27aa7690) so I think we should 
hopefully be clear of this really old bug - 
https://issues.apache.org/jira/browse/IMPALA-695.
Thanks,

-- Piyush

Reply via email to