It is like Matt read my mind. On Sat, Feb 10, 2018 at 6:26 PM, Matt Burgess <mattyb...@apache.org> wrote:
> I'm fine with a vote, but I'll be voting to keep Java as the single > language for the (non-test) code. I share the same concerns as many of > the other folks as far as accepting other languages, it's mainly the > "slippery slope" argument that I don't want to turn into a > JVM-language flame war. If Scala, why not Groovy? Certainly the > syntax is closer to Java, and the community has accepted it as a valid > language for writing unit tests, although we stopped short for > allowing it for the deployable NiFi codebase, for the same reasons > IIRC. If Scala and/or Groovy, why not Kotlin? The same argument > (albeit more tenuous) goes for Clojure and just about every other JVM > language (although I don't expect a call for LuaJ processors lol). > > Whether we decide to support various languages ad-hoc or not, I would > strenuously object to multiple/hybrid build systems for the deployed > artifacts. If I could switch NiFi completely to Gradle I would, but I > realize there are good reasons for not doing so (yet?) in the Apache > NiFi community, and I would never want any hybrid Maven/Gradle build > for the deployable code, likewise for SBT, Leiningen, etc. With a > custom Mojo for Maven NAR builds, and the complexity for hybrid builds > in general, I think this would create a maintenance nightmare. > > The language thing is a tough decision though, it's not awesome that > specifying a single language can be a barrier to a more diverse > community, certainly Scala-based bundles would be more than welcome in > the overall NiFi ecosystem, I just think the cons outweigh the pros > for the baseline code. I've written Groovy processors/NARs using > Gradle as the build system, and I'm good with keeping them in my own > repo, especially when the Extension Registry becomes a thing. I can > see the Extension Registry perhaps making this a moot point, but > clearly we need to have this discussion in the meantime. > > Regards, > Matt > > > On Sat, Feb 10, 2018 at 5:23 PM, Andrew Grande <apere...@gmail.com> wrote: > > Wasn't there a warning trigger about the NiFi distro size from Apache > > recently? IMO, before talking alternative languages, solve the modularity > > and NAR distribution problem. I think the implementation of a module > won't > > matter much then, the point being not everything has to go in the core, > > base distribution, but can still be easily sourced from a known repo, for > > example. > > > > I have a feeling NiFi 1.6+ can be approaching 2GB distro size soon :) > > > > Andrew > > > > On Sat, Feb 10, 2018, 5:12 PM Joey Frazee <joey.fra...@icloud.com> > wrote: > > > >> This probably necessitates a vote, yeah? > >> > >> Frankly, I’m usually happier writing Scala, and I’ve not encountered any > >> problems using processors written in Scala, but I think it’ll be > important > >> to tread lightly. > >> > >> There’s a few things that pop into my head: > >> > >> - Maintainability and reviewability. A very very good Java developer > need > >> not, by definition, either know how to write or identify good Scala or > spot > >> problems and bugs. > >> - Every Scala processor would either end up with a 5MB scala-lang.jar > >> packaged into the .nar or we’d have to start including it in the core > >> somewhere, if it’s not. It’s possible it might have already gotten > pulled > >> up from other dependencies. > >> - Style. There’s a tremendous amount of variation in Scala style because > >> of its type system, implicits, macros, and functional nature. There are > >> very good people out there that can write good Scala that isn’t > readable by > >> the 99%. > >> - Binary compatibility. Scala tends to be a little more brazen about > >> breaking binary compatibility in major releases and those happen a bit > more > >> often than with Java. That’s not a problem for any potential source > code in > >> the project, but it could present some dependency issues someday. > >> - Testing. There’s N > 1 test frameworks and testing styles within > those, > >> so there’s a lot of options for introducing more variability into the > tests. > >> - NiFi uses a lot of statics in setting up properties and relationships > >> and the like, and idiomatic Scala approaches that stuff a bit > differently, > >> so it’ll be necessary to impose some style guidelines so there isn’t too > >> much variation. > >> > >> That said, there are some things that won’t be problematic: > >> > >> - As mentioned, processors written in Scala do just work. > >> - The scala-maven-plugin works just fine allowing mixed Java-Scala > >> projects (btw, it’d probably be not super great to do mixed Java-Scala > and > >> mixed Maven-SBT though). > >> - A lot of the above concerns could be addressed by having clear style > >> guidelines. > >> > >> Another thing: most of the projects I see deliver separate jars for > Scala > >> components are delivering idiomatic APIs wrapping Java (or vice versa). > I > >> think publishing a separate set of jars/nars for stuff written in > Scala > >> would be odd since here it’d mostly be processors with new functionality > >> and not functionality for using Scala. I could imagine a lib of > implicits, > >> traits, classes that could make the Scala development more enjoyable. > That > >> probably would make sense to deliver that way. > >> > >> -joey > >> > >> On Feb 10, 2018, 10:33 AM -0600, Bryan Bende <bbe...@gmail.com>, wrote: > >> > I agree more with Andy about sticking with Java. The more varying > >> languages > >> > used, the more challenging it is to maintain. Once the code is part of > >> the > >> > Apache NiFi git repo, it is now the responsibility of the committers > and > >> > PMC members to maintain it. > >> > > >> > I’d even say I am somewhat against the groovy/Spock test code that > Andy > >> > mentioned. I have frequently spent hours trying to fix a Spock test > that > >> > broke from something I was working on. Every committer is familiar > with > >> > JUnit, but only a couple know Spock. Just using this as an example > that > >> > every committer knows Java, but only a couple probably know Scala, > >> Clojure, > >> > etc. > >> > > >> > On Sat, Feb 10, 2018 at 10:25 AM Jeff <jtsw...@gmail.com> wrote: > >> > > >> > > +1 to Joe's response. If you can develop a component in Groovy or > Scala > >> > > (or Clojure!) more quickly/comfortably, or if allowing components > >> written > >> > > in other languages would encourage people to contribute more, I'm > all > >> for > >> > > it. > >> > > > >> > > On Sat, Feb 10, 2018 at 7:42 AM Joe Witt <joe.w...@gmail.com> > wrote: > >> > > > >> > > > i personally would be ok with it for an extension/processor > provided > >> it > >> > > > integrates well with the build. > >> > > > > >> > > > i would agree with andys view for core framework stuff but for > >> > > extensions i > >> > > > think we can do it like mikethomsen suggested. > >> > > > > >> > > > others? > >> > > > > >> > > > thanks > >> > > > joe > >> > > > > >> > > > On Feb 10, 2018 7:30 AM, "Mike Thomsen" <mikerthom...@gmail.com> > >> wrote: > >> > > > > >> > > > > I'm just a community contributor, so take that FWIW, but a > >> compromise > >> > > > might > >> > > > > be to publish the Scala code as separate maven modules to maven > >> central > >> > > > and > >> > > > > then submit a thoroughly tested processor written in Java. As > long > >> as > >> > > you > >> > > > > have enough unit and integration tests to give strong coverage, > I > >> > > > wouldn't > >> > > > > imagine anyone here would have issues reviewing it. If the tests > >> fail > >> > > > > because of code issues in the external dependencies, the obvious > >> answer > >> > > > is > >> > > > > to just hold the PR until the tests pass. > >> > > > > > >> > > > > On Fri, Feb 9, 2018 at 9:00 AM, Weiss, Adam < > >> > > adam.we...@perkinelmer.com > >> > > > > wrote: > >> > > > > > >> > > > > > Devs, > >> > > > > > > >> > > > > > I have some interns starting with my team and we use Scala > >> internally > >> > > > for > >> > > > > > our work. > >> > > > > > If I wanted to have them work to contribute some new > processors, > >> > > would > >> > > > > > they have to be Java to be included with the base > distribution or > >> > > could > >> > > > > > they use Scala as long as it fit within your current build and > >> test > >> > > > > process? > >> > > > > > > >> > > > > > Thanks, > >> > > > > > -Adam > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > -- > >> > Sent from Gmail Mobile > >> >
