I'm fine with a vote, but I'll be voting to keep Java as the single language for the (non-test) code. I share the same concerns as many of the other folks as far as accepting other languages, it's mainly the "slippery slope" argument that I don't want to turn into a JVM-language flame war. If Scala, why not Groovy? Certainly the syntax is closer to Java, and the community has accepted it as a valid language for writing unit tests, although we stopped short for allowing it for the deployable NiFi codebase, for the same reasons IIRC. If Scala and/or Groovy, why not Kotlin? The same argument (albeit more tenuous) goes for Clojure and just about every other JVM language (although I don't expect a call for LuaJ processors lol).
Whether we decide to support various languages ad-hoc or not, I would strenuously object to multiple/hybrid build systems for the deployed artifacts. If I could switch NiFi completely to Gradle I would, but I realize there are good reasons for not doing so (yet?) in the Apache NiFi community, and I would never want any hybrid Maven/Gradle build for the deployable code, likewise for SBT, Leiningen, etc. With a custom Mojo for Maven NAR builds, and the complexity for hybrid builds in general, I think this would create a maintenance nightmare. The language thing is a tough decision though, it's not awesome that specifying a single language can be a barrier to a more diverse community, certainly Scala-based bundles would be more than welcome in the overall NiFi ecosystem, I just think the cons outweigh the pros for the baseline code. I've written Groovy processors/NARs using Gradle as the build system, and I'm good with keeping them in my own repo, especially when the Extension Registry becomes a thing. I can see the Extension Registry perhaps making this a moot point, but clearly we need to have this discussion in the meantime. Regards, Matt On Sat, Feb 10, 2018 at 5:23 PM, Andrew Grande <apere...@gmail.com> wrote: > Wasn't there a warning trigger about the NiFi distro size from Apache > recently? IMO, before talking alternative languages, solve the modularity > and NAR distribution problem. I think the implementation of a module won't > matter much then, the point being not everything has to go in the core, > base distribution, but can still be easily sourced from a known repo, for > example. > > I have a feeling NiFi 1.6+ can be approaching 2GB distro size soon :) > > Andrew > > On Sat, Feb 10, 2018, 5:12 PM Joey Frazee <joey.fra...@icloud.com> wrote: > >> This probably necessitates a vote, yeah? >> >> Frankly, I’m usually happier writing Scala, and I’ve not encountered any >> problems using processors written in Scala, but I think it’ll be important >> to tread lightly. >> >> There’s a few things that pop into my head: >> >> - Maintainability and reviewability. A very very good Java developer need >> not, by definition, either know how to write or identify good Scala or spot >> problems and bugs. >> - Every Scala processor would either end up with a 5MB scala-lang.jar >> packaged into the .nar or we’d have to start including it in the core >> somewhere, if it’s not. It’s possible it might have already gotten pulled >> up from other dependencies. >> - Style. There’s a tremendous amount of variation in Scala style because >> of its type system, implicits, macros, and functional nature. There are >> very good people out there that can write good Scala that isn’t readable by >> the 99%. >> - Binary compatibility. Scala tends to be a little more brazen about >> breaking binary compatibility in major releases and those happen a bit more >> often than with Java. That’s not a problem for any potential source code in >> the project, but it could present some dependency issues someday. >> - Testing. There’s N > 1 test frameworks and testing styles within those, >> so there’s a lot of options for introducing more variability into the tests. >> - NiFi uses a lot of statics in setting up properties and relationships >> and the like, and idiomatic Scala approaches that stuff a bit differently, >> so it’ll be necessary to impose some style guidelines so there isn’t too >> much variation. >> >> That said, there are some things that won’t be problematic: >> >> - As mentioned, processors written in Scala do just work. >> - The scala-maven-plugin works just fine allowing mixed Java-Scala >> projects (btw, it’d probably be not super great to do mixed Java-Scala and >> mixed Maven-SBT though). >> - A lot of the above concerns could be addressed by having clear style >> guidelines. >> >> Another thing: most of the projects I see deliver separate jars for Scala >> components are delivering idiomatic APIs wrapping Java (or vice versa). I >> think publishing a separate set of jars/nars for stuff written in Scala >> would be odd since here it’d mostly be processors with new functionality >> and not functionality for using Scala. I could imagine a lib of implicits, >> traits, classes that could make the Scala development more enjoyable. That >> probably would make sense to deliver that way. >> >> -joey >> >> On Feb 10, 2018, 10:33 AM -0600, Bryan Bende <bbe...@gmail.com>, wrote: >> > I agree more with Andy about sticking with Java. The more varying >> languages >> > used, the more challenging it is to maintain. Once the code is part of >> the >> > Apache NiFi git repo, it is now the responsibility of the committers and >> > PMC members to maintain it. >> > >> > I’d even say I am somewhat against the groovy/Spock test code that Andy >> > mentioned. I have frequently spent hours trying to fix a Spock test that >> > broke from something I was working on. Every committer is familiar with >> > JUnit, but only a couple know Spock. Just using this as an example that >> > every committer knows Java, but only a couple probably know Scala, >> Clojure, >> > etc. >> > >> > On Sat, Feb 10, 2018 at 10:25 AM Jeff <jtsw...@gmail.com> wrote: >> > >> > > +1 to Joe's response. If you can develop a component in Groovy or Scala >> > > (or Clojure!) more quickly/comfortably, or if allowing components >> written >> > > in other languages would encourage people to contribute more, I'm all >> for >> > > it. >> > > >> > > On Sat, Feb 10, 2018 at 7:42 AM Joe Witt <joe.w...@gmail.com> wrote: >> > > >> > > > i personally would be ok with it for an extension/processor provided >> it >> > > > integrates well with the build. >> > > > >> > > > i would agree with andys view for core framework stuff but for >> > > extensions i >> > > > think we can do it like mikethomsen suggested. >> > > > >> > > > others? >> > > > >> > > > thanks >> > > > joe >> > > > >> > > > On Feb 10, 2018 7:30 AM, "Mike Thomsen" <mikerthom...@gmail.com> >> wrote: >> > > > >> > > > > I'm just a community contributor, so take that FWIW, but a >> compromise >> > > > might >> > > > > be to publish the Scala code as separate maven modules to maven >> central >> > > > and >> > > > > then submit a thoroughly tested processor written in Java. As long >> as >> > > you >> > > > > have enough unit and integration tests to give strong coverage, I >> > > > wouldn't >> > > > > imagine anyone here would have issues reviewing it. If the tests >> fail >> > > > > because of code issues in the external dependencies, the obvious >> answer >> > > > is >> > > > > to just hold the PR until the tests pass. >> > > > > >> > > > > On Fri, Feb 9, 2018 at 9:00 AM, Weiss, Adam < >> > > adam.we...@perkinelmer.com >> > > > > wrote: >> > > > > >> > > > > > Devs, >> > > > > > >> > > > > > I have some interns starting with my team and we use Scala >> internally >> > > > for >> > > > > > our work. >> > > > > > If I wanted to have them work to contribute some new processors, >> > > would >> > > > > > they have to be Java to be included with the base distribution or >> > > could >> > > > > > they use Scala as long as it fit within your current build and >> test >> > > > > process? >> > > > > > >> > > > > > Thanks, >> > > > > > -Adam >> > > > > > >> > > > > >> > > > >> > > >> > -- >> > Sent from Gmail Mobile >>
