Hans Zeller created TRAFODION-2222:
--------------------------------------
Summary: YarnConfiguration class not found with Trafodion on
HortonWorks HDP
Key: TRAFODION-2222
URL: https://issues.apache.org/jira/browse/TRAFODION-2222
Project: Apache Trafodion
Issue Type: Bug
Components: sql-general
Affects Versions: 2.0-incubating, 2.1-incubating
Environment: We saw this with Trafodion 2.0.1 on HortonWorks HDP 2.3.6.
Reporter: Hans Zeller
Assignee: Hans Zeller
Fix For: 2.1-incubating
There are a few places in Trafodion where we use a JNI call to execute a HiveQL
statement. On HDP 2.3.6, such operations now hang with the following error
occurring repeatedly (recorded in $MY_SQROOT/logs/trafodion.hdfs.log):
2016-09-13 22:45:15,439 WARN cli.CLIService: OperationHandle
[opType=EXECUTE_STATEMENT,
getHandleIdentifier()=1348cb58-6f70-4bfe-9a81-b887ae4862cc]: The background
operation was aborted
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError:
org/apache/hadoop/yarn/conf/YarnConfiguration
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at
org.apache.hive.service.cli.CLIService.getOperationStatus(CLIService.java:387)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.GetOperationStatus(ThriftCLIService.java:610)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hive.jdbc.HiveConnection$SynchronizedHandler.invoke(HiveConnection.java:1372)
at com.sun.proxy.$Proxy17.GetOperationStatus(Unknown Source)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:269)
at org.trafodion.sql.HiveClient.executeHiveSQL(HiveClient.java:295)
Caused by: java.lang.NoClassDefFoundError:
org/apache/hadoop/yarn/conf/YarnConfiguration
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:60)
at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1368)
at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1352)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1412)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1113)
at
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:154)
at
org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71)
at
org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at
org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.conf.YarnConfiguration
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 21 more
My good friend Google tells me that this is likely a CLASSPATH issue. The
Trafodion CLASSPATH currently does not have the jar files in
/usr/hdp/current/hadoop-yarn-client/ in it. Adding those manually to the
CLASSPATH (in $MY_SQROOT/etc/ms.env) fixed the issue in our server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
