Re: Spark examples on TC
I can help with JDK8 enforcement — just give me the links to target suites when agreed. > On 25 Sep 2019, at 01:53, Denis Magda wrote: > > Nikolay, > > Thanks for the details. How about enforcing JDK 8 for the Spark integration > in general and run its tests on the VM only? We'll do this until Spark > community fully support JDK 11+. > > As for Spark 2.4.0 module [1], how about releasing Ignite 2.7.7 with Spark > related changes only? As long as Ignite modularization is a long process, > we can follow your advice and create two more modules ignite_spark_24 and > ignite_spark_25 . Plus Alexey is working on extra Spark improvements. > > [1] https://issues.apache.org/jira/browse/IGNITE-12054 > - > Denis > > > On Thu, Sep 19, 2019 at 11:58 PM Nikolay Izhikov > wrote: > >> Hello, Denis. >> >> Spark uses Scala 2.11 [1] >> Scala 2.11 compatible with the jdk version described on [2]. >> >> Few notes from the link: >> >>> We recommend using Java 8 for compiling Scala code >>> JDK 11 compatibility notes >>> As of Scala 2.13.0, 2.12.8 and 2.11.12, JDK 11 support is incomplete >> >> I also got several issues with the Spark itself when trying to run it on >> the Java 11. >> Please, take a look at [3], [4] >> >> All in all, with the spark 2.3 we should stay on java8 to get things >> worked. >> >> [1] http://spark.apache.org/docs/2.4.0/ >> [2] https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html >> [3] >> >> https://stackoverflow.com/questions/49961991/java-lang-illegalargumentexception-at-org-apache-xbean-asm5-classreader-initu?noredirect=1=1 >> [4] >> >> https://stackoverflow.com/questions/51352591/spark-java-illegalargumentexception-at-org-apache-xbean-asm5-classreader >> >> >> чт, 19 сент. 2019 г. в 21:39, Denis Magda : >> >>> Nikolay, >>> >>> What needs to be done to ensure Java 9++ works with Spark Examples? Do we >>> need to create another Spark module or drop some scala version? >>> >>> - >>> Denis >>> >>> >>> On Thu, Sep 19, 2019 at 9:23 AM Nikolay Izhikov >>> wrote: >>> Hello, Igniters. Examples suite on TC doesn't work correctly under java9+. [1], [2] The reason is the Spark examples. Spark 2.3(our current version) work with scala 2.11 and scala 2.11 >>> support of java9+ is "incomplete". Can we stick example suite to jdk8 only? [1] >>> >> https://ci.ignite.apache.org/viewLog.html?buildId=4610507=IgniteTests24Java8_Examples [2] >>> >> https://ci.ignite.apache.org/viewLog.html?buildId=4609013=IgniteTests24Java8_Examples >>> >>
Re: Spark examples on TC
Nikolay, Thanks for the details. How about enforcing JDK 8 for the Spark integration in general and run its tests on the VM only? We'll do this until Spark community fully support JDK 11+. As for Spark 2.4.0 module [1], how about releasing Ignite 2.7.7 with Spark related changes only? As long as Ignite modularization is a long process, we can follow your advice and create two more modules ignite_spark_24 and ignite_spark_25 . Plus Alexey is working on extra Spark improvements. [1] https://issues.apache.org/jira/browse/IGNITE-12054 - Denis On Thu, Sep 19, 2019 at 11:58 PM Nikolay Izhikov wrote: > Hello, Denis. > > Spark uses Scala 2.11 [1] > Scala 2.11 compatible with the jdk version described on [2]. > > Few notes from the link: > > > We recommend using Java 8 for compiling Scala code > > JDK 11 compatibility notes > > As of Scala 2.13.0, 2.12.8 and 2.11.12, JDK 11 support is incomplete > > I also got several issues with the Spark itself when trying to run it on > the Java 11. > Please, take a look at [3], [4] > > All in all, with the spark 2.3 we should stay on java8 to get things > worked. > > [1] http://spark.apache.org/docs/2.4.0/ > [2] https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html > [3] > > https://stackoverflow.com/questions/49961991/java-lang-illegalargumentexception-at-org-apache-xbean-asm5-classreader-initu?noredirect=1=1 > [4] > > https://stackoverflow.com/questions/51352591/spark-java-illegalargumentexception-at-org-apache-xbean-asm5-classreader > > > чт, 19 сент. 2019 г. в 21:39, Denis Magda : > > > Nikolay, > > > > What needs to be done to ensure Java 9++ works with Spark Examples? Do we > > need to create another Spark module or drop some scala version? > > > > - > > Denis > > > > > > On Thu, Sep 19, 2019 at 9:23 AM Nikolay Izhikov > > wrote: > > > > > Hello, Igniters. > > > > > > Examples suite on TC doesn't work correctly under java9+. [1], [2] > > > > > > The reason is the Spark examples. > > > > > > Spark 2.3(our current version) work with scala 2.11 and scala 2.11 > > support > > > of java9+ is "incomplete". > > > > > > Can we stick example suite to jdk8 only? > > > > > > [1] > > > > > > https://ci.ignite.apache.org/viewLog.html?buildId=4610507=IgniteTests24Java8_Examples > > > [2] > > > > > > https://ci.ignite.apache.org/viewLog.html?buildId=4609013=IgniteTests24Java8_Examples > > > > > >
Re: Spark examples on TC
Hello, Denis. Spark uses Scala 2.11 [1] Scala 2.11 compatible with the jdk version described on [2]. Few notes from the link: > We recommend using Java 8 for compiling Scala code > JDK 11 compatibility notes > As of Scala 2.13.0, 2.12.8 and 2.11.12, JDK 11 support is incomplete I also got several issues with the Spark itself when trying to run it on the Java 11. Please, take a look at [3], [4] All in all, with the spark 2.3 we should stay on java8 to get things worked. [1] http://spark.apache.org/docs/2.4.0/ [2] https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html [3] https://stackoverflow.com/questions/49961991/java-lang-illegalargumentexception-at-org-apache-xbean-asm5-classreader-initu?noredirect=1=1 [4] https://stackoverflow.com/questions/51352591/spark-java-illegalargumentexception-at-org-apache-xbean-asm5-classreader чт, 19 сент. 2019 г. в 21:39, Denis Magda : > Nikolay, > > What needs to be done to ensure Java 9++ works with Spark Examples? Do we > need to create another Spark module or drop some scala version? > > - > Denis > > > On Thu, Sep 19, 2019 at 9:23 AM Nikolay Izhikov > wrote: > > > Hello, Igniters. > > > > Examples suite on TC doesn't work correctly under java9+. [1], [2] > > > > The reason is the Spark examples. > > > > Spark 2.3(our current version) work with scala 2.11 and scala 2.11 > support > > of java9+ is "incomplete". > > > > Can we stick example suite to jdk8 only? > > > > [1] > > > https://ci.ignite.apache.org/viewLog.html?buildId=4610507=IgniteTests24Java8_Examples > > [2] > > > https://ci.ignite.apache.org/viewLog.html?buildId=4609013=IgniteTests24Java8_Examples > > >
Re: Spark examples on TC
Nikolay, What needs to be done to ensure Java 9++ works with Spark Examples? Do we need to create another Spark module or drop some scala version? - Denis On Thu, Sep 19, 2019 at 9:23 AM Nikolay Izhikov wrote: > Hello, Igniters. > > Examples suite on TC doesn't work correctly under java9+. [1], [2] > > The reason is the Spark examples. > > Spark 2.3(our current version) work with scala 2.11 and scala 2.11 support > of java9+ is "incomplete". > > Can we stick example suite to jdk8 only? > > [1] > https://ci.ignite.apache.org/viewLog.html?buildId=4610507=IgniteTests24Java8_Examples > [2] > https://ci.ignite.apache.org/viewLog.html?buildId=4609013=IgniteTests24Java8_Examples >
Re: Spark examples on TC
I support this solution, I think there is no issue to run with java 8 next 1-2 years all examples for TC purposes чт, 19 сент. 2019 г. в 19:23, Nikolay Izhikov : > Hello, Igniters. > > Examples suite on TC doesn't work correctly under java9+. [1], [2] > > The reason is the Spark examples. > > Spark 2.3(our current version) work with scala 2.11 and scala 2.11 support > of java9+ is "incomplete". > > Can we stick example suite to jdk8 only? > > [1] > https://ci.ignite.apache.org/viewLog.html?buildId=4610507=IgniteTests24Java8_Examples > [2] > https://ci.ignite.apache.org/viewLog.html?buildId=4609013=IgniteTests24Java8_Examples >
Spark examples on TC
Hello, Igniters. Examples suite on TC doesn't work correctly under java9+. [1], [2] The reason is the Spark examples. Spark 2.3(our current version) work with scala 2.11 and scala 2.11 support of java9+ is "incomplete". Can we stick example suite to jdk8 only? [1] https://ci.ignite.apache.org/viewLog.html?buildId=4610507=IgniteTests24Java8_Examples [2] https://ci.ignite.apache.org/viewLog.html?buildId=4609013=IgniteTests24Java8_Examples signature.asc Description: This is a digitally signed message part
