Hi all,
I am trying to start solr with a custom plugin which uses spark library. I
am trying to initialize sparkcontext in local mode. I have made a fat jar
for this plugin using maven shade and put it in the lib directory. *While
starting solr it is not able to initialize sparkcontext.* It says class not
found exception for AkkaRpcEnvFactory. Can anyone please help.
*It gives the following error:*
3870 [coreLoadExecutor-4-thread-1] ERROR org.apache.spark.SparkContext
– Error initializing SparkContext.
java.lang.ClassNotFoundException:org.apache.spark.rpc.akka.AkkaRpcEnvFactory
*Here is the detailed error*
java -jar start.jar0 [main] INFO org.eclipse.jetty.server.Server
– jetty-8.1.10.v2013031227 [main] INFO
org.eclipse.jetty.deploy.providers.ScanningAppProvider – Deployment
monitor /home/rahul/solr-4.7.2/example/contexts at interval 040
[main] INFO org.eclipse.jetty.deploy.DeploymentManager – Deployable
added: /home/rahul/solr-4.7.2/example/contexts/solr-jetty-context.xml1095
[main] INFO org.eclipse.jetty.webapp.StandardDescriptorProcessor –
NO JSP Support for /solr, did not find
org.apache.jasper.servlet.JspServlet1155 [main] INFO
org.apache.solr.servlet.SolrDispatchFilter –
SolrDispatchFilter.init()1189 [main] INFO
org.apache.solr.core.SolrResourceLoader – JNDI not configured for
solr (NoInitialContextEx)1190 [main] INFO
org.apache.solr.core.SolrResourceLoader – solr home defaulted to
'solr/' (could not find system property or JNDI)1190 [main] INFO
org.apache.solr.core.SolrResourceLoader – new SolrResourceLoader for
directory: 'solr/'1280 [main] INFO org.apache.solr.core.ConfigSolr –
Loading container configuration from
/home/rahul/solr-4.7.2/example/solr/solr.xml1458 [main] INFO
org.apache.solr.core.CoresLocator – Config-defined core root
directory: /home/rahul/solr-4.7.2/example/solr1465 [main] INFO
org.apache.solr.core.CoreContainer – New CoreContainer
602710225...............
3870 [coreLoadExecutor-4-thread-1] ERROR org.apache.spark.SparkContext
– Error initializing SparkContext.
java.lang.ClassNotFoundException: org.apache.spark.rpc.akka.AkkaRpcEnvFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at
org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:430)
at
org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:383)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at org.apache.spark.util.Utils$.classForName(Utils.scala:173)
at org.apache.spark.rpc.RpcEnv$.getRpcEnvFactory(RpcEnv.scala:42)
at org.apache.spark.rpc.RpcEnv$.create(RpcEnv.scala:53)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:252)
at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:193)
at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:276)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:441)
at
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:61)
at
com.snapdeal.search.spark.SparkLoadModel.loadModel(SparkLoadModel.java:11)
at
com.snapdeal.search.valuesource.parser.RankingModelValueSourceParser.init(RankingModelValueSourceParser.java:29)
at org.apache.solr.core.SolrCore.createInitInstance(SolrCore.java:591)
at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:2191)
at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:2185)
at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:2218)
at org.apache.solr.core.SolrCore.initValueSourceParsers(SolrCore.java:2130)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:765)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:630)
at
org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:562)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:597)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:258)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:250)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)3880
[coreLoadExecutor-4-thread-1] INFO org.apache.spark.SparkContext –
Successfully stopped SparkContext
Rahul Kumar
*Software Engineer- I (Search Snapdeal)*
*M*: +91 9023542950 *EXT: *14226
362-363, ASF CENTRE , UDYOG VIHAR , PHASE - IV , GURGAON 122 016 , INDIA
