[
https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893074#comment-15893074
]
Andrew Otto edited comment on SPARK-1693 at 3/2/17 9:42 PM:
------------------------------------------------------------
We just upgraded to CDH 5.10, which has Spark 1.6.0, Hadoop 2.6.0, Hive 1.1.0,
and Oozie 4.1.0.
We are having trouble running Spark jobs that use HiveContext from Oozie. They
run perfectly fine from the CLI with spark-submit, just not in Oozie. We
aren't certain that HiveContext is related, but we can reproduce regularly with
a job that uses HiveContext.
Anyway, I post this here, because the error we are getting is the same that
started this issue:
{code}class "javax.servlet.FilterRegistration"'s signer information does not
match signer information of other classes in the same package{code}
I've noticed that the Oozie sharelib includes
javax.servlet-3.0.0.v201112011016.jar. I also see that spark-assembly.jar
includes a javax.servlet.FilterRegistration class, although its hard for me to
tell which version. The jetty pom.xml files in spark-assembly.jar seem to say
{{javax.servlet.\*;version="2.6.0"}}, but I'm a little green on how all these
dependencies get resolved. I don't see any javax.servlet .jars in any of
/usr/lib/hadoop* (where CDH installs hadoop jars).
Help! :) If this is not related to this issue, I'll open a new one.
was (Author: ottomata):
We just upgraded to CDH 5.10, which has Spark 1.6.0, Hadoop 2.6.0, Hive 1.1.0,
and Oozie 4.1.0.
We are having trouble running Spark jobs that use HiveContext from Oozie. They
run perfectly fine from the CLI with spark-submit, just not in Oozie. We
aren't certain that HiveContext is related, but we can reproduce regularly with
a job that uses HiveContext.
Anyway, I post this here, because the error we are getting is the same that
started this issue:
{code}class "javax.servlet.FilterRegistration"'s signer information does not
match signer information of other classes in the same package{code}
I've noticed that the Oozie sharelib includes
javax.servlet-3.0.0.v201112011016.jar. I also see that spark-assembly.jar
includes a javax.servlet.FilterRegistration class, although its hard for me to
tell which version. The jetty pom.xml files in spark-assembly.jar seem to say
{{javax.servlet.*;version="2.6.0"}}, but I'm a little green on how all these
dependencies get resolved. I don't see any javax.servlet .jars in any of
/usr/lib/hadoop* (where CDH installs hadoop jars).
Help! :) If this is not related to this issue, I'll open a new one.
> Dependent on multiple versions of servlet-api jars lead to throw an
> SecurityException when Spark built for hadoop 2.3.0 , 2.4.0
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-1693
> URL: https://issues.apache.org/jira/browse/SPARK-1693
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Guoqiang Li
> Assignee: Guoqiang Li
> Priority: Blocker
> Fix For: 1.0.0
>
> Attachments: log.txt
>
>
> {code}mvn test -Pyarn -Dhadoop.version=2.4.0 -Dyarn.version=2.4.0 >
> log.txt{code}
> The log:
> {code}
> UnpersistSuite:
> - unpersist RDD *** FAILED ***
> java.lang.SecurityException: class "javax.servlet.FilterRegistration"'s
> signer information does not match signer information of other classes in the
> same package
> at java.lang.ClassLoader.checkCerts(ClassLoader.java:952)
> at java.lang.ClassLoader.preDefineClass(ClassLoader.java:666)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:794)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
