I also agree with Robert and Kostas that it has to be a community decision. I understand the problems with Eclipse and the Scala IDE which is a pain in the ass. But eventually these things will be fixed. Maybe we could also talk to the typesafe guy and tell him that this problem bothers us a lot.
I also believe that the project is not about a specific programming language but a problem we want to tackle with Flink. From time to time it might be necessary to adapt the tools in order to reach the goal. In fact, I don't believe that Scala parts would drive people away from the project. Instead, Scala enthusiasts would be motivated to join us. Actually I stumbled across a quote of Leibniz which put's my point of view quite accurately in a nutshell: In symbols one observes an advantage in discovery which is greatest when they express the exact nature of a thing briefly and, as it were, picture it; then indeed the labor of thought is wonderfully diminished -- Gottfried Wilhelm Leibniz On Thu, Aug 28, 2014 at 12:57 PM, Kostas Tzoumas <[email protected]> wrote: > On Thu, Aug 28, 2014 at 11:49 AM, Robert Metzger <[email protected]> > wrote: > > > > > Changing the programming language of a very important system component is > > something we should carefully discuss. > > > > Definitely agree, I think the community should discuss this very carefully. > > > > I understand that Akka is written in Scala and that it will be much more > > natural to implement the actor based system using Scala. > > I see the following issues that we should consider: > > Until now, Flink is clearly a project implemented only in Java. The Scala > > API basically sits on top of the Java-based runtime. We do not really > > depend on Scala (we could easily remove the Scala API if we want to). > > Having code written in Scala in the main system will add a hard > dependency > > on a scala version. > > Being a pure Java project has some advantages: I think its a fact that > > there are more Java programmers than Scala programmers. So our chances of > > attracting new contributors are higher when being a Java project. > > On the other hand, we could maybe attract Scala developers to our > project. > > But that has not happened (for contributors, not users!) so far for our > > Scala API, so I don't see any reason for that to happen. > > > > > This is definitely an issue to consider. We need to carefully weight how > important this issue is. If we want to break things, incubation is the > right time to do it. Below are some arguments in favor of breaking things, > but do keep in mind that I am undecided, and I would really like to see the > community weighing in. > > First, I would dare say that the primary reason for someone to contribute > to Flink so far has not been that the code is written in Java, but more the > content and nature of the project. Most contributors are Big Data > enthusiasts in some way or another. > > Second, Scala projects have attracted contributors in the past. > > Third, it should not be too hard for someone that does not know Scala to > contribute to a different component if the interfaces are clear. > > > > Another issue is tooling: There are a lot of problems with Scala and > > Eclipse: I've recently switched to Eclipse Luna. It seems to be > impossible > > to compile Scala code with Luna because ScalaIDE does not properly cope > > with it. > > Even with Eclipse versions that are supported by ScalaIDE, you have to > > manually install 3 plugins, some of them are not available in the Eclipse > > Marketplace. So with a JobManager written in Scala, users can not just > > import our project as a Maven project into Eclipse and start developing. > > The support for Maven is probably also limited. For example, I don't know > > if there is a checkstyle plugin for Scala. > > > > I'm looking forward to hearing other opinions on this issue. As I said in > > the beginning, we should exchange arguments on this and think about it > for > > some time before we decide on this. > > > Best, > > Robert > > > > > > > > On Thu, Aug 28, 2014 at 1:08 AM, Till Rohrmann <[email protected]> > > wrote: > > > > > Hi guys, > > > > > > I currently working on replacing the old rpc infrastructure with an > akka > > > based actor system. In the wake of this change I will reimplement the > > > JobManager and TaskManager which will then be actors. Akka offers a > Java > > > API but the implementation turns out to be very verbose and laborious, > > > because Java 6 and 7 do not support lambdas and pattern matching. Using > > > Scala instead, would allow a far more succinct and clear implementation > > of > > > the JobManager and TaskManager. Instead of a lot of if statements using > > > instanceof to figure out the message type, we could simply use pattern > > > matching. Furthermore, the callback functions could simply be Scala's > > > anonymous functions. Therefore I would propose to use Scala for these > two > > > systems. > > > > > > The Akka system uses the slf4j library as logging interface. Therefore > I > > > would also propose to replace the jcl logging system with the slf4j > > logging > > > system. Since we want to use Akka in many parts of the runtime system > and > > > it recommends using logback as logging backend, I would also like to > > > replace log4j with logback. But this change should inflict only few > > changes > > > once we established the slf4j logging interface everywhere. > > > > > > What do you guys think of that idea? > > > > > > Best regards, > > > > > > Till > > > > > >
