[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16187823#comment-16187823 ] ASF GitHub Bot commented on FLINK-5005: --- Github user joan38 commented on the issue: https://github.com/apache/flink/pull/3703 So I guess this PR is abandoned? > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156747#comment-16156747 ] ASF GitHub Bot commented on FLINK-5005: --- Github user joan38 commented on the issue: https://github.com/apache/flink/pull/3703 Is there any news on this? > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088559#comment-16088559 ] ASF GitHub Bot commented on FLINK-5005: --- Github user ariskk commented on the issue: https://github.com/apache/flink/pull/3703 We are really looking forward to this > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087288#comment-16087288 ] ASF GitHub Bot commented on FLINK-5005: --- Github user joan38 commented on the issue: https://github.com/apache/flink/pull/3703 @greghogan That's a pretty good news. Thanks for your answer. > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087274#comment-16087274 ] ASF GitHub Bot commented on FLINK-5005: --- Github user greghogan commented on the issue: https://github.com/apache/flink/pull/3703 @joan38 there has been a discussion on the mailing list about dropping Java 7 support (no one has objected) which will make it simpler to support Scala 2.12 in the upcoming release. > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087075#comment-16087075 ] ASF GitHub Bot commented on FLINK-5005: --- Github user joan38 commented on the issue: https://github.com/apache/flink/pull/3703 Any news on this? 2.13.0-M1 is out https://github.com/scala/scala/releases/tag/v2.13.0-M1 I'm wondering if we will still be on 2.11 when 2,13,0 is out. > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16018622#comment-16018622 ] ASF GitHub Bot commented on FLINK-5005: --- Github user fhueske commented on the issue: https://github.com/apache/flink/pull/3703 @frankbohman just watch the JIRA issue: https://issues.apache.org/jira/browse/FLINK-5005 > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017956#comment-16017956 ] ASF GitHub Bot commented on FLINK-5005: --- Github user frankbohman commented on the issue: https://github.com/apache/flink/pull/3703 where can i watch the status page-thing.. that tells us when we can get off of old 2.11 ? > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005260#comment-16005260 ] ASF GitHub Bot commented on FLINK-5005: --- Github user joan38 commented on the issue: https://github.com/apache/flink/pull/3703 @StephanEwen Nice! That looks promising. > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005256#comment-16005256 ] ASF GitHub Bot commented on FLINK-5005: --- Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3703 @joan38 I have some WIP for a flag that allows you to use vanilla akka when running on Java 8, Scala 2.11 Here is the branch: https://github.com/StephanEwen/incubator-flink/commits/vanilla_akka You can try to build it via: `mvn clean package -Dscala-2.11 -Pjdk8,vanilla-akka` > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005103#comment-16005103 ] ASF GitHub Bot commented on FLINK-5005: --- Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3703 There is about 10% Flink users on Java 7 (we did a poll recently). Big clusters change slowly... > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005026#comment-16005026 ] ASF GitHub Bot commented on FLINK-5005: --- Github user joan38 commented on the issue: https://github.com/apache/flink/pull/3703 > Java 7 Reaches End of Life. Oracle ceased public availability of security fixes and upgrades for Java 7 as of April 2015 We are in 2017 @StephanEwen but I understand if there is people still using Java 7 > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004767#comment-16004767 ] ASF GitHub Bot commented on FLINK-5005: --- Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3703 @joan - I actually agree with you. We needed to use "flakka" to be able to support Java 7 and bind to a wildcard address (across interfaces). Would be great to be able to do that differently and not have a custom akka build (at least for Java 8 / Scala 2.11 / 2.12) > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002791#comment-16002791 ] ASF GitHub Bot commented on FLINK-5005: --- Github user joan38 commented on the issue: https://github.com/apache/flink/pull/3703 Going into the direction of dropping flakka would be a good win for users of Flink like us that are dealing everyday with SBT/Maven hacking to be able to use vanilla Akka and Flink. > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15975055#comment-15975055 ] ASF GitHub Bot commented on FLINK-5005: --- Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3703 We cannot add any other dependencies to the pom files. Adding "akka" back will create a conflict with the "flakka" files. What we can do is wither of the following two options: - Release "flakka" for Scala 2.12 and then we need to change nothing in Flink. The flakka code is at https://github.com/mxm/flakka - we can do the release, you could help up by checking out what needs to be done to use flakka with Scala 2.12 (if it is at all possible) - See if we can pull out the dependency as a property ans use "flakka" int the Scala 2.10 and 2.11 case and use vanilla akka 2.4 in the java8/scala2.12 case. That would be a lot of Maven hacking, though - if possible, I would prefer the first variant (less complexity in Flink). We can also not add more Travis build profiles (builds take too long already). We need to keep that number as it is and simply select one of these profiles to use Scala 2.12 rather than for example 2.10. > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15975032#comment-15975032 ] ASF GitHub Bot commented on FLINK-5005: --- Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3703 One thing you can try and do is to run `TypeExtractionUtils.checkAndExtractLambda` to see if it is a generated serializable Lambda. In the case of a Lambda, you could switch to a different code path (possibly not clean anything in the first version). @twalthr may have some thoughts on that as well... > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966606#comment-15966606 ] ASF GitHub Bot commented on FLINK-5005: --- Github user DieBauer commented on the issue: https://github.com/apache/flink/pull/3703 Ok, started looking into the issue a bit more. It seems like it has to do with the new lambda generation in scala 2.12 and not with the asm library. From the scala 2.12-M3 release notes (https://github.com/scala/scala/releases/tag/v2.12.0-M3): > Java 8 style closure classes > > Scala 2.12 emits closures in the same style as Java 8. > > For each lambda the compiler generates a method containing the lambda body. > At runtime, this method is passed as an argument to the LambdaMetaFactory provided by the JDK, which creates a closure object. > > Compared to Scala 2.11, the new scheme has the advantage that the compiler does not generate an anonymous class for each lambda anymore. > This leads to significantly smaller JAR files. Our ClosureCleaner uses the class name for instantiating the ClassReader, which is used later on. However, since scala2.12 doesn't generate anonymous classes, the file isn't found (null), therefore we get class not found exception, which make sense now. We have to look into how to circumvent/implement this new generation of 'lambdas'. A small technical example, the testclass which throwed an exception `AcceptPFTestBase`. And then the line containing: `protected val groupedTuples = tuples.groupBy(_._1)` Since tuples is a `Dataset` the function that we have to check is `_._1` (an anonymous function). when compiling/executing with scala 2.11 we get `class org.apache.flink.api.scala.extensions.base.AcceptPFTestBase$$anonfun$1` as `cls` in the `ClosureCleaner.getClassReader` method. And this is indeed a file generated by the scala compiler and can be resolved by `ls.getResourceAsStream(className)`. However when using scala 2.12 we get `class org.apache.flink.api.scala.extensions.base.AcceptPFTestBase$$Lambda$11/1489743810` which is not an existing file, and cannot be resolved by `ls.getResourceAsStream(className)`. Concluding, according to me, with the new scala 2.12 style with lambdas, the current closurecleaner doesn't suffice. There is also a Spark issue (https://issues.apache.org/jira/browse/SPARK-14540) regarding closures in scala 2.12/java8. Any thoughts how to proceed? > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962087#comment-15962087 ] ASF GitHub Bot commented on FLINK-5005: --- Github user DieBauer commented on the issue: https://github.com/apache/flink/pull/3703 I'm running into an issue with asm in for example the flink-scala module, when compiling with 2.12. ``` java.io.IOException: Class not found at org.objectweb.asm.ClassReader.a(Unknown Source) at org.objectweb.asm.ClassReader.(Unknown Source) at org.apache.flink.api.scala.ClosureCleaner$.getClassReader(ClosureCleaner.scala:44) at org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:92) at org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115) at org.apache.flink.api.scala.DataSet.clean(DataSet.scala:125) at org.apache.flink.api.scala.DataSet$$anon$12.(DataSet.scala:910) ``` since the closurecleaner was initially copied from spark, I've looked there and found an issues (https://github.com/apache/spark/pull/9512) regarding asm5 and java8. However, flink is already using asm5 in the closurecleaner. Their dependency is ``` org.apache.xbean xbean-asm5-shaded ``` and ours is from org.ow2.asm, asm. There are things going on in the shaded plugin in the parent pom with regard to relocating dependencies of asm, but I'm not sure how that all works out. So for now, I'm a bit puzzled why we get this error. @greghogan you're right, the profile jdk8 is only enabling the module with examples in java8. But since they are also compiled in the scala-2.11 case, I thought we want to have them? We can drop it of course. > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15961831#comment-15961831 ] ASF GitHub Bot commented on FLINK-5005: --- Github user greghogan commented on the issue: https://github.com/apache/flink/pull/3703 @DieBauer thanks for taking this on! I haven't been using Flink with Scala but I think this will be important to have for the May release. The required changes for type inferences are interesting. I'm puzzled why this would regress. Also, if developers are writing against 2.10 then these issues will not manifest until integration tests are run (the same problem you are experiencing). One other thought: since Scala 2.12 requires Java 8, is it still necessary to specify `jdk8` when executing the `scala-2.12` profile? Flink Forward starts Monday so developer activity will be low this week. @StephanEwen thoughts when you have the chance? > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[
https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15961781#comment-15961781
]
ASF GitHub Bot commented on FLINK-5005:
---
GitHub user DieBauer opened a pull request:
https://github.com/apache/flink/pull/3703
[FLINK-5005] WIP: publish scala 2.12 artifacts
- [ ] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira
title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the
JIRA id)
- [ ] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [ ] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis
build has passed
This is an initial approach to make flink scala 2.12 ready.
I've introduced profiles to switch between 2.12, 2.11 and 2.10. All three
profiles now compile.
`mvn clean install -D$version` where $version is `scala-2.12`, `scala-2.11`
or `scala-2.10`.
To overcome the `flakka` artifacts (akka2.3-custom) for scala 2.12, I've
replaced them with the latest typesafe-akka artifacts when using the 2.12
profile.
TravisCI profiles are added and I've changed the initial release script to
accomodate for 2.12, but this is by no means finished.
I encountered a lot of compilation errors, because types could not be
inferred. Therefore I've added types to problematic expressions.
The kafka 0.10 dependency is bumped to 0.10.1.1 since that's the first
released version for 2.12.
There is some trickery in the connector-parent-pom because only kafka-0.10
is released for 2.12, kafka-0.9 and kafka-0.8 aren't compiled for 2.12. I've to
look into that a little more.
More updated dependencies:
javassist was bumped because of java.lang.IllegalStateException: Failed to
transform class with name scala.concurrent.duration.Duration. Reason:
javassist.bytecode.InterfaceMethodrefInfo cannot be cast to
javassist.bytecode.MethodrefInfo. which led me to:
http://stackoverflow.com/questions/31189086/powermock-and-java-8-issue-interfacemethodrefinfo-cannot-be-cast-to-methodrefin#37217871
twitter-chill was bumped to 0.7.7 version for cross-compiled versions.
grizzled slf4j was bumped for scala 2.12 to version 1.3.0.
scalatest was bumped for scala 2.12 to version 3.0.1
Right now I'm trying to make the travis build succeed.
Any other suggestions are welcome!
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/DieBauer/flink feature/scala-2.12
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3703.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3703
commit 5ac12faddda03456b5c70fabb0bae30a82104a2e
Author: Jens Kat
Date: 2017-01-03T21:32:58Z
initial commit.
commit 1b345a1cb511660e39004ac44685573234a6dca0
Author: Jens Kat
Date: 2017-04-07T12:59:17Z
Merge branch 'master' of https://github.com/apache/flink into
feature/scala-2.12
# Conflicts:
# pom.xml
commit 7b8c8872f3597986df63c097053abebcf276f861
Author: Jens Kat
Date: 2017-04-07T13:05:55Z
Change shell scripts
commit 61d38b8ee3573df7d09734b6d086c1501363d339
Author: Jens Kat
Date: 2017-04-07T16:22:36Z
part1
commit 7f32cad571fb86618f11ba3cc9f5dd06dbd63f52
Author: Jens Kat
Date: 2017-04-07T18:34:06Z
flink-tests
commit e858db282d6546e7e4afc016b18a28aad231ab04
Author: Jens Kat
Date: 2017-04-07T18:51:13Z
fix compile error
[ERROR]
/Users/jens/Development/flink/flink-tests/src/test/java/org/apache/flink/test/javaApiOperators/GroupCombineITCase.java:54:
error: not found: type TestExecutionMode
[INFO] public GroupCombineITCase(TestExecutionMode mode) {
maybe related to https://github.com/scala/bug/issues/10207 ?
commit 260a8356c180c23484f73b5ab13e5f7cbd5a997f
Author: Jens Kat
Date: 2017-04-07T19:02:05Z
since scala 2.12 use java.util.concurrent.forkjoinpool directly
commit 23f513dbf92121e451ff6929269ee76623b3a931
Author: Jens Kat
Date: 2017-04-07T19:42:48Z
fix ambiguous methods
commit b7d48ecc3c2dd14dd916ee95749d9aab286a88b8
Author: Jens Kat
Date: 2017-04-07T20:14:49Z
take into account that only kafka 0.10 is for scala 2.12
commit
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951139#comment-15951139 ] Joan Goyeau commented on FLINK-5005: I was thinking more about 1.3. How hard is the upgrade? > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951071#comment-15951071 ] Greg Hogan commented on FLINK-5005: --- [~j...@goyeau.com] [~michael.meyers] [~jenskat] are you looking for 2.12 artifacts for the 1.2 release or waiting for 1.3 in mid-May? Does adding support for 2.12 require removing support for 2.10 (are the upgraded dependency versions available for 2.10?)? If you are able to contribute a PR and/or validation and testing this is much more likely to happen. > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950998#comment-15950998 ] Joan Goyeau commented on FLINK-5005: Hi guys, Is there any update on this? We are still blocked on 2.11 with only Flink retaining us. Cheers > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856327#comment-15856327 ] Robert Metzger commented on FLINK-5005: --- I can look into releasing flakka for Scala 2.12 next week. > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856310#comment-15856310 ] Greg Hogan commented on FLINK-5005: --- While waiting for 2.12 support, Flink provides release artifacts against 2.11 in addition to 2.10. > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856278#comment-15856278 ] Michael Meyers commented on FLINK-5005: --- Need this too...there's a huge difference between 2.10 and 2.12. Would love to see this! > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855870#comment-15855870 ] Joan Goyeau commented on FLINK-5005: Ned > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement > Components: Scala API >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[
https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15830124#comment-15830124
]
Jens Kat commented on FLINK-5005:
-
I didn't see the flakka project in dataArtisans, thanks for that.
I see the org.scalanlp.breeze_${scala.binary.version} is compatible with scala
2.12 from 0.13 onwards (currently we use 0.12 in the flink-ml module).
So dependency-wise, only the dataartisans are 'blocker' :)
> Publish Scala 2.12 artifacts
>
>
> Key: FLINK-5005
> URL: https://issues.apache.org/jira/browse/FLINK-5005
> Project: Flink
> Issue Type: Improvement
> Components: Scala API
>Reporter: Andrew Roberts
>
> Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and
> offers many compile-time and runtime speed improvements. It would be great to
> get artifacts up on maven central to allow Flink users to migrate to Scala
> 2.12.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[
https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829993#comment-15829993
]
Robert Metzger commented on FLINK-5005:
---
For the {{flakka-*}} artifacts, I can probably build and deploy 2.12 versions.
Flakka is a akka fork with a feature backport from newer akka versions.
> Publish Scala 2.12 artifacts
>
>
> Key: FLINK-5005
> URL: https://issues.apache.org/jira/browse/FLINK-5005
> Project: Flink
> Issue Type: Improvement
> Components: Scala API
>Reporter: Andrew Roberts
>
> Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and
> offers many compile-time and runtime speed improvements. It would be great to
> get artifacts up on maven central to allow Flink users to migrate to Scala
> 2.12.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[
https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15798029#comment-15798029
]
Jens Kat commented on FLINK-5005:
-
I did some initial research, besides all changes that are needed for coding it
seems the following dependencies are a blocker (because they don't exist yet
for 2.12):
- com.data-artisans
- flakka-actor_${scala.binary.version}
- flakka-remote_${scala.binary.version}
- flakka-slf4j_${scala.binary.version}
- flakka-testkit_${scala.binary.version}
- flakka-camel_${scala.binary.version}
- org.scalanlp.breeze_${scala.binary.version}
- org.scalamacros.quasiquotes_${scala.binary.version} (but this one is only
used in the scala-2.10 profile, so not an issue)
The following dependencies exist for 2.12 but only for a higher version number:
- com.twitter.chill_${scala.binary.version}: 0.8.1 => 0.9.0
- org.scalatest.scalatest_${scala.binary.version}: 2.2.2 => 3.0.1
- com.github.scopt.scopt_${scala.binary.version}: 3.2.0 => 3.5.0
- org.clapper.grizzled-slf4j_${scala.binary.version}: 1.1.1 => 1.3.0
- org.apache.kafka.kafka_${scala.binary.version}: 0.8.2.2, 0.9.0.1 => 0.10.1.1
> Publish Scala 2.12 artifacts
>
>
> Key: FLINK-5005
> URL: https://issues.apache.org/jira/browse/FLINK-5005
> Project: Flink
> Issue Type: Improvement
>Reporter: Andrew Roberts
>
> Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and
> offers many compile-time and runtime speed improvements. It would be great to
> get artifacts up on maven central to allow Flink users to migrate to Scala
> 2.12.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795241#comment-15795241 ] Jens Kat commented on FLINK-5005: - Hi, I couldn't find the mailinglist discussion and was wondering what the status of this issue is? I'm willing to put some effort in this to make flink-scala compile with 2.12. > Publish Scala 2.12 artifacts > > > Key: FLINK-5005 > URL: https://issues.apache.org/jira/browse/FLINK-5005 > Project: Flink > Issue Type: Improvement >Reporter: Andrew Roberts > > Scala 2.12 was [released|http://www.scala-lang.org/news/2.12.0] today, and > offers many compile-time and runtime speed improvements. It would be great to > get artifacts up on maven central to allow Flink users to migrate to Scala > 2.12.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
