[ 
https://issues.apache.org/jira/browse/HIVE-14029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15518591#comment-15518591
 ] 

Ferdinand Xu commented on HIVE-14029:
-------------------------------------

Removing the dependency will fail QTest with following error.
{code}
2016-09-23T23:48:02,527  INFO [Driver] util.log: Logging initialized @3685ms
Exception in thread "Driver" java.lang.NoClassDefFoundError: 
org/glassfish/jersey/servlet/ServletContainer
        at 
org.apache.spark.status.api.v1.ApiRootResource$.getServletHandler(ApiRootResource.scala:193)
        at org.apache.spark.ui.SparkUI.initialize(SparkUI.scala:75)
        at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:81)
        at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:215)
        at org.apache.spark.ui.SparkUI$.createLiveUI(SparkUI.scala:157)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:443)
        at 
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58)
        at 
org.apache.hive.spark.client.RemoteDriver.<init>(RemoteDriver.java:157)
        at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:516)
        at 
org.apache.hive.spark.client.SparkClientImpl$2.run(SparkClientImpl.java:228)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: 
org.glassfish.jersey.servlet.ServletContainer
        at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 11 more
{code}

> Update Spark version to 2.0.0
> -----------------------------
>
>                 Key: HIVE-14029
>                 URL: https://issues.apache.org/jira/browse/HIVE-14029
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Ferdinand Xu
>            Assignee: Ferdinand Xu
>         Attachments: HIVE-14029.1.patch, HIVE-14029.2.patch, 
> HIVE-14029.3.patch, HIVE-14029.4.patch, HIVE-14029.5.patch, 
> HIVE-14029.6.patch, HIVE-14029.patch
>
>
> There are quite some new optimizations in Spark 2.0.0. We need to bump up 
> Spark to 2.0.0 to benefit those performance improvements.
> To update Spark version to 2.0.0, the following changes are required:
> * Spark API updates:
> ** SparkShuffler#call return Iterator instead of Iterable
> ** SparkListener -> JavaSparkListener
> ** InputMetrics constructor doesn’t accept readMethod
> ** Method remoteBlocksFetched and localBlocksFetched in ShuffleReadMetrics 
> return long type instead of integer
> * Dependency upgrade:
> ** Jackson: 2.4.2 -> 2.6.5
> ** Netty version: 4.0.23.Final -> 4.0.29.Final
> ** Scala binary version: 2.10 -> 2.11
> ** Scala version: 2.10.4 -> 2.11.8



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to