Hi, Thanks for your reply. I tried with the jar you pointed but It complains about missing HttpPatch that appears on httpclient 4.2
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/http/client/methods/HttpPatch at com.amazonaws.http.AmazonHttpClient.<clinit>(AmazonHttpClient.java:129) at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:120) at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:104) at com.amazonaws.services.cloudwatch.AmazonCloudWatchClient.<init>(AmazonCloudWatchClient.java:171) at com.amazonaws.services.cloudwatch.AmazonCloudWatchClient.<init>(AmazonCloudWatchClient.java:152) at com.scmspain.synapse.SynapseMonitor$.sendMetrics(SynapseMonitor.scala:48) at com.scmspain.synapse.SynapseMonitor$.main(SynapseMonitor.scala:28) at com.scmspain.synapse.SynapseMonitor.main(SynapseMonitor.scala) ... I do not understand who is compiled against such an old version of httpclient, I see in the project dependencies that amazonaws 1.9.10 depends on httclient 4.3... It is spark who is compiled against an old version of amazonaws? Thanks. -- Albert Manyà alber...@eml.cc On Fri, Dec 12, 2014, at 09:27 AM, Akhil Das wrote: > Its a jar conflict (http-client[1] jar), You could download the > appropriate version of that jar and put that in the classpath before > your assembly jar and hopefully it will avoid the conflict. > > Thanks Best Regards > > On Thu, Dec 11, 2014 at 10:13 PM, Albert Manyà > <alber...@eml.cc> wrote: >> Hi, >> >> I've made a simple script in scala that after doing a spark sql query it >> sends the result to AWS's cloudwatch. >> >> I've tested both parts individually (the spark sql one and the >> cloudwatch one) and they worked fine. The trouble comes when I execute >> the script through spark-submit that gives me the following exception: >> >> Exception in thread "main" java.lang.NoSuchMethodError: >> org.apache.http.params.HttpConnectionParams.setSoKeepalive(Lorg/apache/http/params/HttpParams;Z)V >> at >> com.amazonaws.http.HttpClientFactory.createHttpClient(HttpClientFactory.java:95) >> at >> com.amazonaws.http.AmazonHttpClient.<init>(AmazonHttpClient.java:193) >> at >> com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:120) >> at >> com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:104) >> at >> com.amazonaws.services.cloudwatch.AmazonCloudWatchClient.<init>(AmazonCloudWatchClient.java:171) >> at >> com.amazonaws.services.cloudwatch.AmazonCloudWatchClient.<init>(AmazonCloudWatchClient.java:152) >> at >> com.scmspain.synapse.SynapseMonitor$.sendMetrics(SynapseMonitor.scala:50) >> at >> com.scmspain.synapse.SynapseMonitor$.main(SynapseMonitor.scala:45) >> at >> com.scmspain.synapse.SynapseMonitor.main(SynapseMonitor.scala) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:483) >> at >> org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328) >> at >> org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) >> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) >> >> my class is com.scmspain.synapse.SynapseMonitor >> >> I've build my script with sbt assembly, having the following >> dependencies: >> >> libraryDependencies ++= Seq( >> "org.apache.spark" %% "spark-core" % "1.1.0" % "provided", >> "org.apache.spark" %% "spark-sql" % "1.1.0" % "provided", >> "com.amazonaws" % "aws-java-sdk-cloudwatch" % "1.9.10" >> ) >> >> I've unzipped the generated jar assembly and searched for the >> HttpConnectionParams.class and I've found it out under >> org/apache/http/params and having the following signature for >> setSoKeepalive: >> >> public static void setSoKeepalive(HttpParams params, boolean >> enableKeepalive) >> >> At this point I'm stuck and didn't know where to keep looking... some >> help would be greatly appreciated :) >> >> Thank you very much! >> >> -- >> Albert Manyà >> alber...@eml.cc >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org Links: 1. http://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient/4.0-alpha4
