Hi, I'm getting the following error:
java.lang.NoSuchMethodError: org.apache.http.impl.conn.DefaultClientConnectionOperator.<init>(Lorg/apache/http/conn/scheme/SchemeRegistry;Lorg/apache/http/conn/DnsResolver;)V at org.apache.http.impl.conn.PoolingClientConnectionManager.createConnectionOperator(PoolingClientConnectionManager.java:140) at org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:114) at org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:99) at org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:85) at org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:93) at com.amazonaws.http.ConnectionManagerFactory.createPoolingClientConnManager(ConnectionManagerFactory.java:26) at com.amazonaws.http.HttpClientFactory.createHttpClient(HttpClientFactory.java:96) at com.amazonaws.http.AmazonHttpClient.<init>(AmazonHttpClient.java:155) at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:119) at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:103) at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:334) at com.celtra.analyzer.TrackingLogRDD.createClient(TrackingLogRDD.scala:131) at com.celtra.analyzer.TrackingLogRDD.compute(TrackingLogRDD.scala:117) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:237) at org.apache.spark.rdd.RDD.iterator(RDD.scala:226) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:107) at org.apache.spark.scheduler.Task.run(Task.scala:53) at org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:215) at org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:50) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:182) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) My app uses the AWS SDK, which requires org.apache.httpcomponents:httpclient:4.2. I believe the error is caused by the fact that an older version of the package is already present on the classpath: Spark -> Akka -> sjson -> org.apache.httpcomponents:httpclient:4.1 Spark -> jets3t -> commons-httpclient:commons-httpclient:3.1 What are my options if I need to use a newer version of the library in my app? Thanks, Jaka
