An issue that we're working on is described here: https://issues.apache.org/jira/browse/IMPALA-6215.
For your scenario, does the ClassNotFoundException come up deterministically for the complex query or just once in a while? When you test this query, is it the only query running on the system? If you don't mind sharing the query so that I can develop a repro, that would be appreciated. On Wed, Mar 7, 2018 at 2:57 AM, Piyush Narang <[email protected]> wrote: > Hi folks, > > > > I’m running into a strange ClassNotFound error while trying to use one of > my Hive UDFs in Impala. > > I’ve defined the UDF: > > > create function device_type(string, string, string) returns string > location 'my/path/to/my-udf-SNAPSHOT.jar' symbol='com.criteo.hadoop. > hive.udf.UDFUserAgentToDeviceType'; > > > > I try to use it in simple queries and it works fine: > > > select device_type("a", "b", "c"); > > Query: select device_type("a", "b", "c") > > … > > +----------------------------------------+ > > | device_type('a', 'b', 'c') | > > +----------------------------------------+ > > | Mobile - Other | > > +----------------------------------------+ > > Fetched 1 row(s) in 0.01s > > > select ua_device_family, ua_browser_family, ua_os_family, > device_type(ua_device_family, ua_browser_family, ua_os_family) from > bi_arbitrage_full limit 1; > > Query: select ua_device_family, ua_browser_family, ua_os_family, > device_type(ua_device_family, ua_browser_family, ua_os_family) from > bi_arbitrage_full limit 1 > > … > > +------------------+-------------------+--------------+----- > -----------------------------------------------------------------------+ > > | ua_device_family | ua_browser_family | ua_os_family | > device_type(ua_device_family, ua_browser_family, ua_os_family) | > > +------------------+-------------------+--------------+----- > -----------------------------------------------------------------------+ > > | iPhone | mobile safari | iOS | > iPhone > | > > +------------------+-------------------+--------------+----- > -----------------------------------------------------------------------+ > > Fetched 1 row(s) in 0.73s > > > > Now when I try to run a more complex query (same database) > > I get an error: > > WARNINGS: ImpalaRuntimeException: Unable to find class. > > CAUSED BY: ClassNotFoundException: com.criteo.hadoop.hive.udf. > UDFUserAgentToDeviceType > > > > I turned up the log level on the coordinator to debug. I see the calls > being made to load the UDF and the use of the UDF in the query but no > details on the ClassNotFound. > > > > Has anyone run into a similar issue? We’re running Impala 2.11.0-cdh5.14.0 > RELEASE (build d68206561bce6b26762d62c01a78e6cd27aa7690) so I think we > should hopefully be clear of this really old bug - > https://issues.apache.org/jira/browse/IMPALA-695. > > Thanks, > > > > -- Piyush > > >
