Did you use spark.files.userClassPathFirst = true? it's exactly for
this kind of problem.
On Fri, Jan 23, 2015 at 4:42 AM, William-Smith
<williamsmith.m...@gmail.com> wrote:
> I have had the same issue while using HttpClient from AWS EMR Spark Streaming
> to post to a nodejs server.
>
> I have found ... using
> Classloder.getResource('org/apache/http/client/HttpClient") .... that the
> class
> Is being loaded front the spark-assembly-1.1.0-hadoop2.4.0.jar.
>
> That in itself is not the issue because the version is 4.2.5 .... the same
> version I am using on my local machine with success .... using Hadoop cdh 5.
>
>
>
> The issue is that HttpClient relies on Httpcore .... and there is an old
> commons-httpcore-1.3.jar as well as httpcore-4.5.2 in the spark-assembly
> jar.
>
> It looks like the old one is getting loaded first.
>
> So the fix might be to build the Spark jar myself without the httpcore-1.3
> and replace it on bootstrap.
> I will keep you posted on the outcome.
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-developers-list.1001551.n3.nabble.com/spark-1-1-0-w-hadoop-2-4-vs-aws-java-sdk-1-7-2-tp8481p10250.html
> Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
> For additional commands, e-mail: dev-h...@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org
