[
https://issues.apache.org/jira/browse/AMBARI-17537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiqing Yang updated AMBARI-17537:
----------------------------------
Description:
After deploy Spark history server is stopped.
16/06/27 19:55:45 INFO HistoryServer: History provider class:
org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider
Exception in thread "main" java.lang.ClassNotFoundException:
org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:224)
at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala).
Why this bug happened?
In the jira, the spark version is 1.5.1. which doesnot has the class
org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider. Instead, it
has "org.apache.spark.deploy.yarn.history.YarnHistoryProvider" and
"org.apache.spark.deploy.history.yarn.YarnHistoryProvider".
How to fix?
Actually, in spark internal logic,
"org.apache.spark.deploy.yarn.history.YarnHistoryProvider" is a wrapper of
"org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider" and
"org.apache.spark.deploy.history.yarn.YarnHistoryProvider".
The property "spark.history.provider" appears twice in the stack definition,
having two values: one is
"org.apache.spark.deploy.yarn.history.YarnHistoryProvider", and the other one
is "org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider". In this
patch, remove the one whose value is
"org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider".
This jira was originally reported by Denis Tarasyuk.
was:
After deploy Spark history server is stopped.
16/06/27 19:55:45 INFO HistoryServer: History provider class:
org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider
Exception in thread "main" java.lang.ClassNotFoundException:
org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:224)
at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala).
Why this error happened?
In the jira, the spark version is 1.5.1. which doesnot has the class
org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider. Instead, it
has "org.apache.spark.deploy.yarn.history.YarnHistoryProvider" and
"org.apache.spark.deploy.history.yarn.YarnHistoryProvider".
How to fix?
Actually, in spark internal logic,
"org.apache.spark.deploy.yarn.history.YarnHistoryProvider" is a wrapper of
"org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider" and
"org.apache.spark.deploy.history.yarn.YarnHistoryProvider".
The property "spark.history.provider" appears twice in the stack definition,
having two values: one is
"org.apache.spark.deploy.yarn.history.YarnHistoryProvider", and the other one
is "org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider". In this
patch, remove the one whose value is
"org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider".
This jira was originally reported by Denis Tarasyuk.
> Spark history server stopped after deploy
> -----------------------------------------
>
> Key: AMBARI-17537
> URL: https://issues.apache.org/jira/browse/AMBARI-17537
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.4.0
> Environment: ambari-server version: ii ambari-agent
> 2.4.0.0-694 amd64 Ambari Agent
> ii ambari-metrics-assembly 2.4.0.0-694
> amd64 Ambari Metrics Assembly
> ii ambari-server 2.4.0.0-694
> amd64 Ambari Server
> ambari-server --hash: 678877dcd8342d6a6c8edfdacbf215d02c89f580
> HDP Stack: 2.3
> HDP Version: 2.3.2.0-2950
> Ambari DB: :Oracle
> Oozie/Hive DB: Oracle/Oracle
> Security:yes
> Security Type:MIT/MIT
> Blueprints: false
> Umask:
> JDK: OracleJDK8
> HA: yes
> Repo URL:
> http://dev.hortonworks.com.s3.amazonaws.com/ambari/debian7/2.x/updates/2.4.0.0/ambariqe.list
> OS: Debian 7
> Ambari User: root
> Agents User: root
> MOTD enabled: true
> Customized service users: true
> Is tmp exec: false
> Reporter: Weiqing Yang
> Assignee: Weiqing Yang
> Fix For: trunk, 2.4.0
>
> Attachments: AMBARI-17537_v0.patch, AMBARI-17537_v1.patch
>
>
> After deploy Spark history server is stopped.
> 16/06/27 19:55:45 INFO HistoryServer: History provider class:
> org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:224)
> at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala).
> Why this bug happened?
> In the jira, the spark version is 1.5.1. which doesnot has the class
> org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider. Instead, it
> has "org.apache.spark.deploy.yarn.history.YarnHistoryProvider" and
> "org.apache.spark.deploy.history.yarn.YarnHistoryProvider".
> How to fix?
> Actually, in spark internal logic,
> "org.apache.spark.deploy.yarn.history.YarnHistoryProvider" is a wrapper of
> "org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider" and
> "org.apache.spark.deploy.history.yarn.YarnHistoryProvider".
> The property "spark.history.provider" appears twice in the stack definition,
> having two values: one is
> "org.apache.spark.deploy.yarn.history.YarnHistoryProvider", and the other one
> is "org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider". In this
> patch, remove the one whose value is
> "org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider".
> This jira was originally reported by Denis Tarasyuk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
