onedigit edited a comment on pull request #170: URL: https://github.com/apache/incubator-livy/pull/170#issuecomment-703122226
This problem persists in the current master. ``` at org.apache.livy.utils.SparkYarnApp$.yarnClient(SparkYarnApp.scala:52) at org.apache.livy.utils.SparkYarnApp$$anon$1.run(SparkYarnApp.scala:78) Caused by: java.lang.ClassNotFoundException: javax.ws.rs.ext.MessageBodyReader at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 54 more java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.yarn.util.timeline.TimelineUtils at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:200) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.livy.utils.SparkYarnApp$.yarnClient$lzycompute(SparkYarnApp.scala:54) ``` And then I see the exact same issue that @akitanaka describes, i.e. the Livy session remains in `starting` state. Even if, `jersey-core-1.19.jar` is added, the problem won't be solved, since that jar no longer includes the missing class. That is ``` jar tvf jersey-core-1.19.jar | grep javax.ws.rs.ext.MessageBodyReader 1763 Thu Nov 21 07:17:18 UTC 2013 META-INF/services/javax.ws.rs.ext.MessageBodyReader ``` However, If we look inside the `jersey-core-1.9.jar`, we see that the missing class is there: ``` jar tvf jersey-core-1.9.jar | grep javax.ws.rs.ext.MessageBodyReader 1763 Fri Sep 02 11:16:04 UTC 2011 META-INF/services/javax.ws.rs.ext.MessageBodyReader 950 Fri Sep 02 11:16:40 UTC 2011 javax/ws/rs/ext/MessageBodyReader.class ``` In order to keep `jersey-core-1.19.jar` that's required by the thriftserver and get Livy server working, we need to add the right version of `jsr311-api` jar. For example, Hadoop 3.3.0 now includes `jsr311-api-1.1.1.jar`. This is the jar that now contains the required class: ``` jar tvf jsr311-api-1.1.1.jar | grep javax.ws.rs.ext.MessageBodyReader 950 Mon Nov 09 13:45:50 UTC 2009 javax/ws/rs/ext/MessageBodyReader.class ``` If I manually add this jar to the class path of the Livy server, then it works as expected. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org