Pravin Dahare created SPARK-24482:
-------------------------------------
Summary: Not able to run simple SOLR application
Key: SPARK-24482
URL: https://issues.apache.org/jira/browse/SPARK-24482
Project: Spark
Issue Type: Brainstorming
Components: Spark Submit, YARN
Affects Versions: 1.6.0
Reporter: Pravin Dahare
I am not able to run solr applciation may be due to "org.apache.httpcomponents"
jars.
I am using "org.apache.httpcomponents" 4.3.1 jars and my cluster has old
jars:4.2.5.
Event though I am excluding 4.2.5 jars , issue still exists. Adding exception
at bottom. Any help is appreciated.
=================================================
spark2-submit \
--master yarn \
--deploy-mode client \
--class com.citi.solrsearch.SolrSearch \
--conf
spark.executor.extraJavaOptions=-Djava.security.auth.login.config=/home/icttdnee/Sprinter/solrCitiscreening/jaas-client.conf
\
--conf
spark.jars.excludes=/opt/cloudera/parcels/CDH-5.9.1-1.cdh5.9.1.p2964.3211/jars/httpclient-4.2.5.jar,/opt/cloudera/parcels/CDH-5.9.1-1.cdh5.9.1.p2964.3211/jars/httpcore-4.2.5.jar
\
--conf spark.driver.userClassPathFirst=false \
--conf spark.executor.userClassPathFirst=false \
--conf
spark.driver.extraClassPath=/home/icttdnee/citiscreen_solr/httpclient-4.3.1.jar,/home/icttdnee/citiscreen_solr/httpcore-4.3.1.jar,/home/icttdnee/citiscreen_solr/httpclient-4.3.1.jar,/home/icttdnee/citiscreen_solr/httpcore-4.3.1.jar
\
--jars
/home/icttdnee/citiscreen_solr/spark-solr-2.1.0.jar,/home/pd94468/scala-logging_2.11-3.4.0.jar,/home/icttdnee/citiscreen_solr/spark-core_2.11-1.6.0.jar,/home/icttdnee/citiscreen_solr/httpclient-4.3.1.jar,/home/icttdnee/citiscreen_solr/httpcore-4.3.1.jar
\
/home/icttdnee/citiscreen_solr/spark-solr-1.0-SNAPSHOT-jar-with-dependencies.jar
\
https://bdgtr012x07h2.nam.nsroot.net:8985/solr/icttdnee_ttsd_collection \
&q=alert_id:[16000%20TO%20*]
=================================================
Maven Snippet:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>2.0.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.lucidworks.spark/spark-solr -->
<dependency>
<groupId>com.lucidworks.spark</groupId>
<artifactId>spark-solr</artifactId>
<version>3.5.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.solr/solr-core -->
<dependency>
<groupId>org.apache.solr</groupId>
<artifactId>solr-core</artifactId>
<version>4.10.3</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.3.1</version>
</dependency>
==============================================================
18/06/07 05:08:00 INFO spark.SparkContext: Registered listener
com.pepperdata.spark.metrics.PepperdataSparkListener
Exception in thread "main" com.google.common.util.concurrent.ExecutionError:
java.lang.NoSuchFieldError: INSTANCE
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2232)
at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
at
com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
at
com.lucidworks.spark.util.SolrSupport$.getCachedCloudClient(SolrSupport.scala:245)
at
com.lucidworks.spark.util.SolrQuerySupport$.getUniqueKey(SolrQuerySupport.scala:91)
at com.lucidworks.spark.rdd.SolrRDD.<init>(SolrRDD.scala:40)
at com.lucidworks.spark.rdd.SelectSolrRDD.<init>(SelectSolrRDD.scala:30)
at com.lucidworks.spark.rdd.SelectSolrRDD$.apply(SelectSolrRDD.scala:168)
at com.lucidworks.spark.rdd.SolrJavaRDD.get(SolrJavaRDD.java:48)
at com.citi.solrsearch.SolrSearch.main(SolrSearch.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.NoSuchFieldError: INSTANCE
at
org.apache.http.impl.conn.HttpClientConnectionOperator.<init>(HttpClientConnectionOperator.java:75)
at
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:149)
at
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:136)
at
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:112)
at
org.apache.solr.client.solrj.impl.HttpClientUtil.createPoolingConnectionManager(HttpClientUtil.java:228)
at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:223)
at
com.lucidworks.spark.util.SolrSupport$.getSolrCloudClient(SolrSupport.scala:218)
at
com.lucidworks.spark.util.SolrSupport$.getNewSolrCloudClient(SolrSupport.scala:241)
at
com.lucidworks.spark.util.CacheCloudSolrClient$$anon$1.load(SolrSupport.scala:39)
at
com.lucidworks.spark.util.CacheCloudSolrClient$$anon$1.load(SolrSupport.scala:37)
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
