An issue that we're working on is described here:
https://issues.apache.org/jira/browse/IMPALA-6215.

For your scenario, does the ClassNotFoundException come up
deterministically for the complex query or just
once in a while? When you test this query, is it the only query running on
the system?

If you don't mind sharing the query so that I can develop a repro, that
would be appreciated.

On Wed, Mar 7, 2018 at 2:57 AM, Piyush Narang <p.nar...@criteo.com> wrote:

> Hi folks,
>
>
>
> I’m running into a strange ClassNotFound error while trying to use one of
> my Hive UDFs in Impala.
>
> I’ve defined the UDF:
>
> > create function device_type(string, string, string) returns string
> location 'my/path/to/my-udf-SNAPSHOT.jar' symbol='com.criteo.hadoop.
> hive.udf.UDFUserAgentToDeviceType';
>
>
>
> I try to use it in simple queries and it works fine:
>
> > select device_type("a", "b", "c");
>
> Query: select device_type("a", "b", "c")
>
> …
>
> +----------------------------------------+
>
> | device_type('a', 'b', 'c') |
>
> +----------------------------------------+
>
> | Mobile - Other                         |
>
> +----------------------------------------+
>
> Fetched 1 row(s) in 0.01s
>
> > select ua_device_family, ua_browser_family, ua_os_family,
> device_type(ua_device_family, ua_browser_family, ua_os_family) from
> bi_arbitrage_full limit 1;
>
> Query: select ua_device_family, ua_browser_family, ua_os_family,
> device_type(ua_device_family, ua_browser_family, ua_os_family) from
> bi_arbitrage_full limit 1
>
> …
>
> +------------------+-------------------+--------------+-----
> -----------------------------------------------------------------------+
>
> | ua_device_family | ua_browser_family | ua_os_family |
> device_type(ua_device_family, ua_browser_family, ua_os_family) |
>
> +------------------+-------------------+--------------+-----
> -----------------------------------------------------------------------+
>
> | iPhone           | mobile safari     | iOS          |
> iPhone
> |
>
> +------------------+-------------------+--------------+-----
> -----------------------------------------------------------------------+
>
> Fetched 1 row(s) in 0.73s
>
>
>
> Now when I try to run a more complex query (same database)
>
> I get an error:
>
> WARNINGS: ImpalaRuntimeException: Unable to find class.
>
> CAUSED BY: ClassNotFoundException: com.criteo.hadoop.hive.udf.
> UDFUserAgentToDeviceType
>
>
>
> I turned up the log level on the coordinator to debug. I see the calls
> being made to load the UDF and the use of the UDF in the query but no
> details on the ClassNotFound.
>
>
>
> Has anyone run into a similar issue? We’re running Impala 2.11.0-cdh5.14.0
> RELEASE (build d68206561bce6b26762d62c01a78e6cd27aa7690) so I think we
> should hopefully be clear of this really old bug -
> https://issues.apache.org/jira/browse/IMPALA-695.
>
> Thanks,
>
>
>
> -- Piyush
>
>
>

Reply via email to