Re: Not albe to run FP-growth Example
You should add spark-mllib_2.10 as a dependency instead of declaring
it as the artifactId. And always use the same version for spark-core
and spark-mllib. I saw you used 1.3.0 for spark-core but 1.4.0 for
spark-mllib, which is not guaranteed to work. If you set the scope to
"provided", mllib jar won't be included in the run time dependency. It
means that you need to use spark-summit from Spark to launch your
application. Please read the user guide:
http://spark.apache.org/docs/latest/submitting-applications.html.
-Xiangrui
On Sun, Jun 14, 2015 at 11:39 PM, masoom alam wrote:
> even if the following POM is also not working:
>
> http://maven.apache.org/POM/4.0.0";
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> http://maven.apache.org/maven-v4_0_0.xsd";>
>
> spark-parent_2.10
> org.apache.spark
> 1.4.0
>
> 4.0.0
> org.apache.spark
> spark-mllib_2.10
> Spark Project ML Library
> http://spark.apache.org/
>
>
> target/scala-${scala.binary.version}/classes
>
> target/scala-${scala.binary.version}/test-classes
>
>
>
> netlib-lgpl
>
>
> com.github.fommil.netlib
> all
> ${netlib.java.version}
> pom
>
>
>
>
>
>
> org.apache.spark
> spark-core_2.10
> 1.4.0
> compile
>
>
> org.apache.spark
> spark-streaming_2.10
> 1.4.0
> compile
>
>
> org.apache.spark
> spark-sql_2.10
> 1.4.0
> compile
>
>
> org.apache.spark
> spark-graphx_2.10
> 1.4.0
> compile
>
>
> org.jblas
> jblas
> 1.2.4
> test
>
>
> org.scalanlp
> breeze_2.10
> 0.11.2
> compile
>
>
> junit
> junit
>
>
> commons-math3
> org.apache.commons
>
>
>
>
> org.apache.commons
> commons-math3
> 3.4.1
> compile
>
>
> org.scalacheck
> scalacheck_2.10
> 1.11.3
> test
>
>
> test-interface
> org.scala-sbt
>
>
>
>
> junit
> junit
> 4.10
> test
>
>
> hamcrest-core
> org.hamcrest
>
>
>
>
> com.novocode
> junit-interface
> 0.10
> test
>
>
> junit-dep
> junit
>
>
> test-interface
> org.scala-tools.testing
>
>
>
>
> org.mockito
> mockito-all
> 1.9.5
> test
>
>
> org.apache.spark
> spark-streaming_2.10
> 1.4.0
> test-jar
> test
>
>
> org.jpmml
> pmml-model
> 1.1.15
> compile
>
>
> FastInfoset
> com.sun.xml.fastinfoset
>
>
> istack-commons-runtime
> com.sun.istack
>
>
>
>
> org.codehaus.groovy
> groovy-all
> 2.3.7
> provided
>
>
> org.scalatest
> scalatest_2.10
> 2.2.1
> test
>
>
>
> mllib
>
>
>
> Any clues?
>
> On Sun, Jun 14, 2015 at 8:20 PM, masoom alam
> wrote:
>>
>> Getting the following error:
>>
>> [INFO]
>> [INFO]
>>
>> [INFO] Building example 0.0.1
>> [INFO]
>>
>> Downloading:
>> http://repo.maven.apache.org/maven2/org/apache/spark/spark-mllib_2.10/1.4.0/spark-mllib_2.10-1.4.0.pom
>> [INFO]
>>
>> [INFO] BUILD FAILURE
>> [INFO]
>>
>> [INFO] Total time: 41.561s
>>
Re: Not albe to run FP-growth Example
even if the following POM is also not working:
http://maven.apache.org/POM/4.0.0"; xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="
http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd";>
spark-parent_2.10
org.apache.spark
1.4.0
4.0.0
org.apache.spark
spark-mllib_2.10
Spark Project ML Library
http://spark.apache.org/
target/scala-${scala.binary.version}/classes
target/scala-${scala.binary.version}/test-classes
netlib-lgpl
com.github.fommil.netlib
all
${netlib.java.version}
pom
org.apache.spark
spark-core_2.10
1.4.0
compile
org.apache.spark
spark-streaming_2.10
1.4.0
compile
org.apache.spark
spark-sql_2.10
1.4.0
compile
org.apache.spark
spark-graphx_2.10
1.4.0
compile
org.jblas
jblas
1.2.4
test
org.scalanlp
breeze_2.10
0.11.2
compile
junit
junit
commons-math3
org.apache.commons
org.apache.commons
commons-math3
3.4.1
compile
org.scalacheck
scalacheck_2.10
1.11.3
test
test-interface
org.scala-sbt
junit
junit
4.10
test
hamcrest-core
org.hamcrest
com.novocode
junit-interface
0.10
test
junit-dep
junit
test-interface
org.scala-tools.testing
org.mockito
mockito-all
1.9.5
test
org.apache.spark
spark-streaming_2.10
1.4.0
test-jar
test
org.jpmml
pmml-model
1.1.15
compile
FastInfoset
com.sun.xml.fastinfoset
istack-commons-runtime
com.sun.istack
org.codehaus.groovy
groovy-all
2.3.7
provided
org.scalatest
scalatest_2.10
2.2.1
test
mllib
Any clues?
On Sun, Jun 14, 2015 at 8:20 PM, masoom alam
wrote:
> *Getting the following error:*
>
> [INFO]
>
> [INFO]
>
> [INFO] Building example 0.0.1
> [INFO]
>
> Downloading:
> http://repo.maven.apache.org/maven2/org/apache/spark/spark-mllib_2.10/1.4.0/spark-mllib_2.10-1.4.0.pom
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 41.561s
> [INFO] Finished at: Mon Jun 15 08:17:43 PKT 2015
> [INFO] Final Memory: 6M/16M
> [INFO]
>
> [ERROR] Failed to execute goal on project learning-spark-mini-example:
> Could not resolve dependencies for project
> com.oreilly.learningsparkexamples.mini:learning-spark-mini-example:jar:0.0.1:
> Failed to collect dependencies for
> [org.apache.spark:spark-core_2.10:jar:1.3.0 (provided),
> org.apache.spark:spark-mllib_2.10:jar:1.4.0 (compile)]: Failed to read
> artifact descriptor for org.apache.spark:spark-mllib_2.10:jar:1.4.0: Could
> not transfer artifact org.apache.spark:spark-mllib_2.10:pom:1.4.0 from/to
> central (http://repo.maven.apache.org/maven2): repo.maven.apache.org:
> Unknown host repo.maven.apache.org -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the
> -e switch.
> [ERROR] Re-run Maven using the -X switch to en
Re: Not albe to run FP-growth Example
*Getting the following error:*
[INFO]
[INFO]
[INFO] Building example 0.0.1
[INFO]
Downloading:
http://repo.maven.apache.org/maven2/org/apache/spark/spark-mllib_2.10/1.4.0/spark-mllib_2.10-1.4.0.pom
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 41.561s
[INFO] Finished at: Mon Jun 15 08:17:43 PKT 2015
[INFO] Final Memory: 6M/16M
[INFO]
[ERROR] Failed to execute goal on project learning-spark-mini-example:
Could not resolve dependencies for project
com.oreilly.learningsparkexamples.mini:learning-spark-mini-example:jar:0.0.1:
Failed to collect dependencies for
[org.apache.spark:spark-core_2.10:jar:1.3.0 (provided),
org.apache.spark:spark-mllib_2.10:jar:1.4.0 (compile)]: Failed to read
artifact descriptor for org.apache.spark:spark-mllib_2.10:jar:1.4.0: Could
not transfer artifact org.apache.spark:spark-mllib_2.10:pom:1.4.0 from/to
central (http://repo.maven.apache.org/maven2): repo.maven.apache.org:
Unknown host repo.maven.apache.org -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
_
*My POM file is as follows:-*
com.oreilly.learningsparkexamples.mini
learning-spark-mini-example
4.0.0
example
jar
0.0.1
org.apache.spark
spark-core_2.10
1.3.0
provided
org.apache.spark
spark-mllib_2.10
1.4.0
provided
1.7
org.apache.maven.plugins
maven-compiler-plugin
3.1
${java.version}
${java.version}
___
*I have noticed that it tries to download the following file*:
http://repo.maven.apache.org/maven2/org/apache/spark/spark-mllib_2.10/1.4.0/spark-mllib_2.10-1.4.0.pom
*which
is available*
Any pointers?
Thanks for the help.
On Sun, Jun 14, 2015 at 5:24 AM, masoom alam
wrote:
> Thanks a lot. Will try in a while n update
>
> Thanks again
> On Jun 14, 2015 5:13 PM, "Sonal Goyal" wrote:
>
>> Try with spark-mllib_2.10 as the artifactid
>> On Jun 14, 2015 12:02 AM, "masoom alam"
>> wrote:
>>
>>> This is not working:
>>>
>>>
>>> org.apache.spark.mlib
>>> spark-mlib
>>>
>>> provided
>>>
>>>
>>>
>>>
>>> On Sat, Jun 13, 2015 at 11:56 PM, masoom alam >> > wrote:
>>>
These two imports are missing and thus FP-growth is not compiling...
import org.apache.spark.*mllib.fpm.FPGrowth*;
import org.apache.spark.*mllib.fpm.FPGrowthModel*;
How to include the dependency in the POM file?
On Sat, Jun 13, 2015 at 4:26 AM, masoom alam >>> > wrote:
> Thanks for the answer. Any example?
> On Jun 13, 2015 2:13 PM, "Sonal Goyal" wrote:
>
>> I think you need to add dependency to spark mllib too.
>> On Jun 13, 2015 11:10 AM, "masoom alam"
>> wrote:
>>
>>> Hi every one,
>>>
>>> I am trying to run the FP growth example. I have tried to compile
>>> the following POM file:
>>>
>>>
>>> com.oreilly.learningsparkexamples.mini
>>> learning-spark-mini-example
>>> 4.0.0
>>> example
>>> jar
>>> 0.0.1
>>>
>>>
>>> org.apache.spark
>>> spark-core_2.10
>>> 1.3.0
>>> provided
>>>
>>>
>>>
>>> 1.7
>>>
>>>
>>>
>>>
>>> org.apache.maven.plugins
>>> maven-compiler-plugin
>>> 3.1
>>>
>>> ${java.version}
>>> ${java.version}
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> It successfully builds the project, but IDE is complaining
>>> that: Error:(29, 34) java: package org.apache.spark.mlli
Re: Not albe to run FP-growth Example
This is not working:
org.apache.spark.mlib
spark-mlib
provided
On Sat, Jun 13, 2015 at 11:56 PM, masoom alam
wrote:
> These two imports are missing and thus FP-growth is not compiling...
>
> import org.apache.spark.*mllib.fpm.FPGrowth*;
> import org.apache.spark.*mllib.fpm.FPGrowthModel*;
>
> How to include the dependency in the POM file?
>
> On Sat, Jun 13, 2015 at 4:26 AM, masoom alam
> wrote:
>
>> Thanks for the answer. Any example?
>> On Jun 13, 2015 2:13 PM, "Sonal Goyal" wrote:
>>
>>> I think you need to add dependency to spark mllib too.
>>> On Jun 13, 2015 11:10 AM, "masoom alam"
>>> wrote:
>>>
Hi every one,
I am trying to run the FP growth example. I have tried to compile the
following POM file:
com.oreilly.learningsparkexamples.mini
learning-spark-mini-example
4.0.0
example
jar
0.0.1
org.apache.spark
spark-core_2.10
1.3.0
provided
1.7
org.apache.maven.plugins
maven-compiler-plugin
3.1
${java.version}
${java.version}
It successfully builds the project, but IDE is complaining
that: Error:(29, 34) java: package org.apache.spark.mllib.fpm does not
exist
Just as a side note, I downloaded Version 1.3 of Spark so FP-growth
algorithm should be part of it?
Thanks.
>>>
>
Re: Not albe to run FP-growth Example
These two imports are missing and thus FP-growth is not compiling...
import org.apache.spark.*mllib.fpm.FPGrowth*;
import org.apache.spark.*mllib.fpm.FPGrowthModel*;
How to include the dependency in the POM file?
On Sat, Jun 13, 2015 at 4:26 AM, masoom alam
wrote:
> Thanks for the answer. Any example?
> On Jun 13, 2015 2:13 PM, "Sonal Goyal" wrote:
>
>> I think you need to add dependency to spark mllib too.
>> On Jun 13, 2015 11:10 AM, "masoom alam"
>> wrote:
>>
>>> Hi every one,
>>>
>>> I am trying to run the FP growth example. I have tried to compile the
>>> following POM file:
>>>
>>>
>>> com.oreilly.learningsparkexamples.mini
>>> learning-spark-mini-example
>>> 4.0.0
>>> example
>>> jar
>>> 0.0.1
>>>
>>>
>>> org.apache.spark
>>> spark-core_2.10
>>> 1.3.0
>>> provided
>>>
>>>
>>>
>>> 1.7
>>>
>>>
>>>
>>>
>>> org.apache.maven.plugins
>>> maven-compiler-plugin
>>> 3.1
>>>
>>> ${java.version}
>>> ${java.version}
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> It successfully builds the project, but IDE is complaining
>>> that: Error:(29, 34) java: package org.apache.spark.mllib.fpm does not exist
>>>
>>> Just as a side note, I downloaded Version 1.3 of Spark so FP-growth
>>> algorithm should be part of it?
>>>
>>> Thanks.
>>>
>>
Re: Not albe to run FP-growth Example
Thanks for the answer. Any example?
On Jun 13, 2015 2:13 PM, "Sonal Goyal" wrote:
> I think you need to add dependency to spark mllib too.
> On Jun 13, 2015 11:10 AM, "masoom alam" wrote:
>
>> Hi every one,
>>
>> I am trying to run the FP growth example. I have tried to compile the
>> following POM file:
>>
>>
>> com.oreilly.learningsparkexamples.mini
>> learning-spark-mini-example
>> 4.0.0
>> example
>> jar
>> 0.0.1
>>
>>
>> org.apache.spark
>> spark-core_2.10
>> 1.3.0
>> provided
>>
>>
>>
>> 1.7
>>
>>
>>
>>
>> org.apache.maven.plugins
>> maven-compiler-plugin
>> 3.1
>>
>> ${java.version}
>> ${java.version}
>>
>>
>>
>>
>>
>>
>>
>> It successfully builds the project, but IDE is complaining
>> that: Error:(29, 34) java: package org.apache.spark.mllib.fpm does not exist
>>
>> Just as a side note, I downloaded Version 1.3 of Spark so FP-growth
>> algorithm should be part of it?
>>
>> Thanks.
>>
>
Not albe to run FP-growth Example
Hi every one,
I am trying to run the FP growth example. I have tried to compile the
following POM file:
com.oreilly.learningsparkexamples.mini
learning-spark-mini-example
4.0.0
example
jar
0.0.1
org.apache.spark
spark-core_2.10
1.3.0
provided
1.7
org.apache.maven.plugins
maven-compiler-plugin
3.1
${java.version}
${java.version}
It successfully builds the project, but IDE is complaining that: Error:(29,
34) java: package org.apache.spark.mllib.fpm does not exist
Just as a side note, I downloaded Version 1.3 of Spark so FP-growth
algorithm should be part of it?
Thanks.
