Hi Steve, I was indeed using spark 2.1. I was getting this error while calling spark via Zeppelin. Zeppelin comes with older version of httpclient apparently. I copied the httpclient 4.5.2 and httpclient 4.2.2 into zeppelin/interpreter/spark and this problem went away.
Thank you for your help. On Thu, Mar 30, 2017 at 9:55 AM, Steve Loughran <ste...@hortonworks.com> wrote: > > On 29 Mar 2017, at 14:42, Arvind Kandaswamy <aravind.ka...@gmail.com> > wrote: > > Hello, > > I am getting the following error. I get this error when trying to use AWS > S3. This appears to be a conflict with httpclient. AWS S3 comes with > httplient-4.5.2.jar. I am not sure how to force spark to use this version. > I have tried spark.driver.userClassPathFirst = true, spark.executor. > userClassPathFirst=true. Did not help. I am using Zeppelin to call the > spark engine in case if that is an issue. > > > Spark 2.x ships with httpclient 4.5.2; there is no conflict there. if you > are on 1.6, you could actually try bumping the httplient and httpcomponents > to be consistent > > <!-- org.apache.httpcomponents/httpclient--> > <commons.httpclient.version>4.5.2</commons.httpclient.version> > <commons.httpcore.version>4.4.4</commons.httpcore.version> > > its important to have org.apache.httpcomponents / httpcore in sync with > httpclient; it's probably there where your problem is arising > > Is there anything else that I can try? > > java.lang.NoSuchMethodError: org.apache.http.conn.ssl. > SSLConnectionSocketFactory.<init>(Ljavax/net/ssl/ > SSLContext;Ljavax/net/ssl/HostnameVerifier;)V > at com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.<init>( > SdkTLSSocketFactory.java:56) > at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory > .getPreferredSocketFactory(ApacheConnectionManagerFactory.java:92) > at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory > .create(ApacheConnectionManagerFactory.java:65) > at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory > .create(ApacheConnectionManagerFactory.java:58) > at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create( > ApacheHttpClientFactory.java:51) > at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create( > ApacheHttpClientFactory.java:39) > at com.amazonaws.http.AmazonHttpClient.<init>(AmazonHttpClient.java:314) > at com.amazonaws.http.AmazonHttpClient.<init>(AmazonHttpClient.java:298) > at com.amazonaws.AmazonWebServiceClient.<init>( > AmazonWebServiceClient.java:165) > at com.amazonaws.services.s3.AmazonS3Client.<init>( > AmazonS3Client.java:583) > at com.amazonaws.services.s3.AmazonS3Client.<init>( > AmazonS3Client.java:563) > at com.amazonaws.services.s3.AmazonS3Client.<init>( > AmazonS3Client.java:541) > at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize( > S3AFileSystem.java:235) > > >
