[
https://issues.apache.org/jira/browse/IGNITE-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15739213#comment-15739213
]
Denis Magda commented on IGNITE-3710:
-------------------------------------
[~vkulichenko], please review the changes which are in branch IGNITE-3710.
[~avinogradov], I failed to check my changes on TeamCity [1] because get the
following exception all the time
{code}
[09:05:42]W: [Step 5/6] [ERROR] error: missing or invalid dependency
detected while loading class file 'SparkContext.class'.
{code}
The exception is obviously caused by this maven related issue
{code}
[09:05:38]W: [Step 5/6] [WARNING] Invalid POM for
org.apache.spark:spark-sql_2.11:jar:2.0.2, transitive dependencies (if any)
will not be available, enable debug logging for more details: Some problems
were encountered while processing the POMs:
[09:05:38]W: [Step 5/6] [ERROR]
'dependencyManagement.dependencies.dependency.groupId' for
${jline.groupid}:jline:jar with value '${jline.groupid}' does not match a valid
id pattern. @ org.apache.spark:spark-parent_2.11:2.0.2,
/home/teamcity/.m2/repository/org/apache/spark/spark-parent_2.11/2.0.2/spark-parent_2.11-2.0.2.pom,
line 1838, column 18
{code}
I didn't manage to fix this maven related issue on TeamCity. However,
everything works perfectly fine and the test suite passes on my local machine
(even if I clean .m2 completely).
Could you take a look and suggest how to get through this? I'm attaching the
build logs in the debug mode.
[1]
http://ci.ignite.apache.org/viewLog.html?buildId=387310&tab=buildResultsDiv&buildTypeId=IgniteTests_IgniteRdd
> Upgrade ignite-spark module to Spark 2.0
> ----------------------------------------
>
> Key: IGNITE-3710
> URL: https://issues.apache.org/jira/browse/IGNITE-3710
> Project: Ignite
> Issue Type: Improvement
> Components: Ignite RDD
> Affects Versions: 1.7
> Reporter: Valentin Kulichenko
> Assignee: Denis Magda
>
> Currently {{ignite-spark}} depends on Spark 1.5.2 and fails with 2.0 with
> this exception:
> {noformat}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/spark/Logging
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.ignite.spark.JavaIgniteContext.<init>(JavaIgniteContext.scala:42)
> at client.SparkIgniteClient.main(SparkIgniteClient.java:75)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
> Caused by: java.lang.ClassNotFoundException: org.apache.spark.Logging
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> {noformat}
> Need to investigate if we can upgrade without breaking compatibility with old
> versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
