Re: Error in running JavaALSExample example from spark examples

[VG]

Great. thanks a ton for helping out on this Sean.
I somehow messed this up (and was running in loops for last 2 hours )

thanks again

-VG

On Fri, Jul 22, 2016 at 11:28 PM, Sean Owen  wrote:

> You mark these provided, which is correct. If the version of Scala
> provided at runtime differs, you'll have a problem.
>
> In fact you can also see you mixed Scala versions in your dependencies
> here. MLlib is on 2.10.
>
> On Fri, Jul 22, 2016 at 6:49 PM, VG  wrote:
> > Sean,
> >
> > I am only using the maven dependencies for spark in my pom file.
> > I don't have anything else. I guess maven dependency should resolve to
> the
> > correct scala version .. isn;t it ? Any ideas.
> >
> > 
> > org.apache.spark
> > spark-core_2.11
> > 2.0.0-preview
> > provided
> > 
> > 
> > 
> > org.apache.spark
> > spark-sql_2.11
> > 2.0.0-preview
> > provided
> > 
> > 
> > 
> > org.apache.spark
> > spark-streaming_2.11
> > 2.0.0-preview
> > provided
> > 
> > 
> > 
> > org.apache.spark
> > spark-mllib_2.10
> > 2.0.0-preview
> > provided
> > 
> >
> >
> >
> > On Fri, Jul 22, 2016 at 11:16 PM, Sean Owen  wrote:
> >>
> >> -dev
> >> Looks like you are mismatching the version of Spark you deploy on at
> >> runtime then. Sounds like it was built for Scala 2.10
> >>
> >> On Fri, Jul 22, 2016 at 6:43 PM, VG  wrote:
> >> > Using 2.0.0-preview using maven
> >> > So all dependencies should be correct I guess
> >> >
> >> > 
> >> > org.apache.spark
> >> > spark-core_2.11
> >> > 2.0.0-preview
> >> > provided
> >> > 
> >> >
> >> > I see in maven dependencies that this brings in
> >> > scala-reflect-2.11.4
> >> > scala-compiler-2.11.0
> >> >
> >> > and so on
> >> >
> >> >
> >> >
> >> > On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici  >
> >> > wrote:
> >> >>
> >> >> What version of Spark/Scala are you running?
> >> >>
> >> >>
> >> >>
> >> >> -Aaron
> >> >
> >> >
> >
> >
>

Re: Error in running JavaALSExample example from spark examples

[Aaron Ilovici]

Your error stems from spark.ml, and in your pom mllib is the only dependency 
that is 2.10. Is there a reason for this? IE, you tell maven mllib 2.10 is 
provided at runtime. Is 2.10 on the machine, or is 2.11?

-Aaron

From: VG <vlin...@gmail.com>
Date: Friday, July 22, 2016 at 1:49 PM
To: Sean Owen <so...@cloudera.com>
Cc: User <user@spark.apache.org>
Subject: Re: Error in running JavaALSExample example from spark examples

Sean,

I am only using the maven dependencies for spark in my pom file.
I don't have anything else. I guess maven dependency should resolve to the 
correct scala version .. isn;t it ? Any ideas.


org.apache.spark
spark-core_2.11
2.0.0-preview
provided



org.apache.spark
spark-sql_2.11
2.0.0-preview
provided



org.apache.spark
spark-streaming_2.11
2.0.0-preview
provided



org.apache.spark
spark-mllib_2.10
2.0.0-preview
provided




On Fri, Jul 22, 2016 at 11:16 PM, Sean Owen 
<so...@cloudera.com<mailto:so...@cloudera.com>> wrote:
-dev
Looks like you are mismatching the version of Spark you deploy on at
runtime then. Sounds like it was built for Scala 2.10

On Fri, Jul 22, 2016 at 6:43 PM, VG 
<vlin...@gmail.com<mailto:vlin...@gmail.com>> wrote:
> Using 2.0.0-preview using maven
> So all dependencies should be correct I guess
>
> 
> org.apache.spark
> spark-core_2.11
> 2.0.0-preview
> provided
> 
>
> I see in maven dependencies that this brings in
> scala-reflect-2.11.4
> scala-compiler-2.11.0
>
> and so on
>
>
>
> On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici 
> <ailov...@wayfair.com<mailto:ailov...@wayfair.com>>
> wrote:
>>
>> What version of Spark/Scala are you running?
>>
>>
>>
>> -Aaron
>
>

Re: Error in running JavaALSExample example from spark examples

[Sean Owen]

You mark these provided, which is correct. If the version of Scala
provided at runtime differs, you'll have a problem.

In fact you can also see you mixed Scala versions in your dependencies
here. MLlib is on 2.10.

On Fri, Jul 22, 2016 at 6:49 PM, VG  wrote:
> Sean,
>
> I am only using the maven dependencies for spark in my pom file.
> I don't have anything else. I guess maven dependency should resolve to the
> correct scala version .. isn;t it ? Any ideas.
>
> 
> org.apache.spark
> spark-core_2.11
> 2.0.0-preview
> provided
> 
> 
> 
> org.apache.spark
> spark-sql_2.11
> 2.0.0-preview
> provided
> 
> 
> 
> org.apache.spark
> spark-streaming_2.11
> 2.0.0-preview
> provided
> 
> 
> 
> org.apache.spark
> spark-mllib_2.10
> 2.0.0-preview
> provided
> 
>
>
>
> On Fri, Jul 22, 2016 at 11:16 PM, Sean Owen  wrote:
>>
>> -dev
>> Looks like you are mismatching the version of Spark you deploy on at
>> runtime then. Sounds like it was built for Scala 2.10
>>
>> On Fri, Jul 22, 2016 at 6:43 PM, VG  wrote:
>> > Using 2.0.0-preview using maven
>> > So all dependencies should be correct I guess
>> >
>> > 
>> > org.apache.spark
>> > spark-core_2.11
>> > 2.0.0-preview
>> > provided
>> > 
>> >
>> > I see in maven dependencies that this brings in
>> > scala-reflect-2.11.4
>> > scala-compiler-2.11.0
>> >
>> > and so on
>> >
>> >
>> >
>> > On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici 
>> > wrote:
>> >>
>> >> What version of Spark/Scala are you running?
>> >>
>> >>
>> >>
>> >> -Aaron
>> >
>> >
>
>

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Error in running JavaALSExample example from spark examples

[VG]

Sean,

I am only using the maven dependencies for spark in my pom file.
I don't have anything else. I guess maven dependency should resolve to the
correct scala version .. isn;t it ? Any ideas.


org.apache.spark
spark-core_2.11
2.0.0-preview
provided



org.apache.spark
spark-sql_2.11
2.0.0-preview
provided



org.apache.spark
spark-streaming_2.11
2.0.0-preview
provided



org.apache.spark
spark-mllib_2.10
2.0.0-preview
provided




On Fri, Jul 22, 2016 at 11:16 PM, Sean Owen  wrote:

> -dev
> Looks like you are mismatching the version of Spark you deploy on at
> runtime then. Sounds like it was built for Scala 2.10
>
> On Fri, Jul 22, 2016 at 6:43 PM, VG  wrote:
> > Using 2.0.0-preview using maven
> > So all dependencies should be correct I guess
> >
> > 
> > org.apache.spark
> > spark-core_2.11
> > 2.0.0-preview
> > provided
> > 
> >
> > I see in maven dependencies that this brings in
> > scala-reflect-2.11.4
> > scala-compiler-2.11.0
> >
> > and so on
> >
> >
> >
> > On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici 
> > wrote:
> >>
> >> What version of Spark/Scala are you running?
> >>
> >>
> >>
> >> -Aaron
> >
> >
>

Re: Error in running JavaALSExample example from spark examples

[Sean Owen]

-dev
Looks like you are mismatching the version of Spark you deploy on at
runtime then. Sounds like it was built for Scala 2.10

On Fri, Jul 22, 2016 at 6:43 PM, VG  wrote:
> Using 2.0.0-preview using maven
> So all dependencies should be correct I guess
>
> 
> org.apache.spark
> spark-core_2.11
> 2.0.0-preview
> provided
> 
>
> I see in maven dependencies that this brings in
> scala-reflect-2.11.4
> scala-compiler-2.11.0
>
> and so on
>
>
>
> On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici 
> wrote:
>>
>> What version of Spark/Scala are you running?
>>
>>
>>
>> -Aaron
>
>

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Error in running JavaALSExample example from spark examples

[Inam Ur Rehman]

Hello guys..i know its irrelevant to this topic but i've been looking
desperately for the solution. I am facing en exception
http://apache-spark-user-list.1001560.n3.nabble.com/how-to-resolve-you-must-build-spark-with-hive-exception-td27390.html

plz help me.. I couldn't find any solution..

On Fri, Jul 22, 2016 at 10:43 PM, VG  wrote:

> Using 2.0.0-preview using maven
> So all dependencies should be correct I guess
>
> 
> org.apache.spark
> spark-core_2.11
> 2.0.0-preview
> provided
> 
>
> I see in maven dependencies that this brings in
> scala-reflect-2.11.4
> scala-compiler-2.11.0
>
> and so on
>
>
>
> On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici 
> wrote:
>
>> What version of Spark/Scala are you running?
>>
>>
>>
>> -Aaron
>>
>
>

Re: Error in running JavaALSExample example from spark examples

[VG]

Using 2.0.0-preview using maven
So all dependencies should be correct I guess


org.apache.spark
spark-core_2.11
2.0.0-preview
provided


I see in maven dependencies that this brings in
scala-reflect-2.11.4
scala-compiler-2.11.0

and so on



On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici 
wrote:

> What version of Spark/Scala are you running?
>
>
>
> -Aaron
>

Re: Error in running JavaALSExample example from spark examples

[Aaron Ilovici]

What version of Spark/Scala are you running?

-Aaron

Error in running JavaALSExample example from spark examples

[VG]

I am getting the following error

Exception in thread "main" java.lang.NoSuchMethodError:
scala.reflect.api.JavaUniverse.runtimeMirror(Ljava/lang/ClassLoader;)Lscala/reflect/api/JavaMirrors$JavaMirror;
at org.apache.spark.ml.recommendation.ALS.fit(ALS.scala:452)

Any suggestions to resolve this

VG