I think it makes the most sense to me for us to publish a separate repo with a 
module and nar build for now and post when it's available in the users group.

Thanks for the discussion everyone, hopefully we can start making some helpful 
contributions soon.

-Adam


On 2018/02/10 23:43:31, Tony Kurc <t...@gmail.com<mailto:t...@gmail.com>> wrote:
> It is like Matt read my mind.>
>
> On Sat, Feb 10, 2018 at 6:26 PM, Matt Burgess 
> <ma...@apache.org<mailto:ma...@apache.org>> wrote:>
>
> > I'm fine with a vote, but I'll be voting to keep Java as the single>
> > language for the (non-test) code. I share the same concerns as many of>
> > the other folks as far as accepting other languages, it's mainly the>
> > "slippery slope" argument that I don't want to turn into a>
> > JVM-language flame war.  If Scala, why not Groovy? Certainly the>
> > syntax is closer to Java, and the community has accepted it as a valid>
> > language for writing unit tests, although we stopped short for>
> > allowing it for the deployable NiFi codebase, for the same reasons>
> > IIRC.  If Scala and/or Groovy, why not Kotlin?  The same argument>
> > (albeit more tenuous) goes for Clojure and just about every other JVM>
> > language (although I don't expect a call for LuaJ processors lol).>
> >>
> > Whether we decide to support various languages ad-hoc or not, I would>
> > strenuously object to multiple/hybrid build systems for the deployed>
> > artifacts. If I could switch NiFi completely to Gradle I would, but I>
> > realize there are good reasons for not doing so (yet?) in the Apache>
> > NiFi community, and I would never want any hybrid Maven/Gradle build>
> > for the deployable code, likewise for SBT, Leiningen, etc. With a>
> > custom Mojo for Maven NAR builds, and the complexity for hybrid builds>
> > in general, I think this would create a maintenance nightmare.>
> >>
> > The language thing is a tough decision though, it's not awesome that>
> > specifying a single language can be a barrier to a more diverse>
> > community, certainly Scala-based bundles would be more than welcome in>
> > the overall NiFi ecosystem, I just think the cons outweigh the pros>
> > for the baseline code. I've written Groovy processors/NARs using>
> > Gradle as the build system, and I'm good with keeping them in my own>
> > repo, especially when the Extension Registry becomes a thing. I can>
> > see the Extension Registry perhaps making this a moot point, but>
> > clearly we need to have this discussion in the meantime.>
> >>
> > Regards,>
> > Matt>
> >>
> >>
> > On Sat, Feb 10, 2018 at 5:23 PM, Andrew Grande 
> > <ap...@gmail.com<mailto:ap...@gmail.com>> wrote:>
> > > Wasn't there a warning trigger about the NiFi distro size from Apache>
> > > recently? IMO, before talking alternative languages, solve the modularity>
> > > and NAR distribution problem. I think the implementation of a module>
> > won't>
> > > matter much then, the point being not everything has to go in the core,>
> > > base distribution, but can still be easily sourced from a known repo, for>
> > > example.>
> > >>
> > > I have a feeling NiFi 1.6+ can be approaching 2GB distro size soon :)>
> > >>
> > > Andrew>
> > >>
> > > On Sat, Feb 10, 2018, 5:12 PM Joey Frazee 
> > > <jo...@icloud.com<mailto:jo...@icloud.com>>>
> > wrote:>
> > >>
> > >> This probably necessitates a vote, yeah?>
> > >>>
> > >> Frankly, I’m usually happier writing Scala, and I’ve not encountered any>
> > >> problems using processors written in Scala, but I think it’ll be>
> > important>
> > >> to tread lightly.>
> > >>>
> > >> There’s a few things that pop into my head:>
> > >>>
> > >> - Maintainability and reviewability. A very very good Java developer>
> > need>
> > >> not, by definition, either know how to write or identify good Scala or>
> > spot>
> > >> problems and bugs.>
> > >> - Every Scala processor would either end up with a 5MB scala-lang.jar>
> > >> packaged into the .nar or we’d have to start including it in the core>
> > >> somewhere, if it’s not. It’s possible it might have already gotten>
> > pulled>
> > >> up from other dependencies.>
> > >> - Style. There’s a tremendous amount of variation in Scala style because>
> > >> of its type system, implicits, macros, and functional nature. There are>
> > >> very good people out there that can write good Scala that isn’t>
> > readable by>
> > >> the 99%.>
> > >> - Binary compatibility. Scala tends to be a little more brazen about>
> > >> breaking binary compatibility in major releases and those happen a bit>
> > more>
> > >> often than with Java. That’s not a problem for any potential source>
> > code in>
> > >> the project, but it could present some dependency issues someday.>
> > >> - Testing. There’s N > 1 test frameworks and testing styles within>
> > those,>
> > >> so there’s a lot of options for introducing more variability into the>
> > tests.>
> > >> - NiFi uses a lot of statics in setting up properties and relationships>
> > >> and the like, and idiomatic Scala approaches that stuff a bit>
> > differently,>
> > >> so it’ll be necessary to impose some style guidelines so there isn’t too>
> > >> much variation.>
> > >>>
> > >> That said, there are some things that won’t be problematic:>
> > >>>
> > >> - As mentioned, processors written in Scala do just work.>
> > >> - The scala-maven-plugin works just fine allowing mixed Java-Scala>
> > >> projects (btw, it’d probably be not super great to do mixed Java-Scala>
> > and>
> > >> mixed Maven-SBT though).>
> > >> - A lot of the above concerns could be addressed by having clear style>
> > >> guidelines.>
> > >>>
> > >> Another thing: most of the projects I see deliver separate jars for>
> > Scala>
> > >> components are delivering idiomatic APIs wrapping Java (or vice versa).>
> > I>
> > >> think publishing   a separate set of jars/nars for stuff written in>
> > Scala>
> > >> would be odd since here it’d mostly be processors with new functionality>
> > >> and not functionality for using Scala. I could imagine a lib of>
> > implicits,>
> > >> traits, classes that could make the Scala development more enjoyable.>
> > That>
> > >> probably would make sense to deliver that way.>
> > >>>
> > >> -joey>
> > >>>
> > >> On Feb 10, 2018, 10:33 AM -0600, Bryan Bende 
> > >> <bb...@gmail.com<mailto:bb...@gmail.com>>, wrote:>
> > >> > I agree more with Andy about sticking with Java. The more varying>
> > >> languages>
> > >> > used, the more challenging it is to maintain. Once the code is part of>
> > >> the>
> > >> > Apache NiFi git repo, it is now the responsibility of the committers>
> > and>
> > >> > PMC members to maintain it.>
> > >> >>
> > >> > I’d even say I am somewhat against the groovy/Spock test code that>
> > Andy>
> > >> > mentioned. I have frequently spent hours trying to fix a Spock test>
> > that>
> > >> > broke from something I was working on. Every committer is familiar>
> > with>
> > >> > JUnit, but only a couple know Spock. Just using this as an example>
> > that>
> > >> > every committer knows Java, but only a couple probably know Scala,>
> > >> Clojure,>
> > >> > etc.>
> > >> >>
> > >> > On Sat, Feb 10, 2018 at 10:25 AM Jeff 
> > >> > <jt...@gmail.com<mailto:jt...@gmail.com>> wrote:>
> > >> >>
> > >> > > +1 to Joe's response. If you can develop a component in Groovy or>
> > Scala>
> > >> > > (or Clojure!) more quickly/comfortably, or if allowing components>
> > >> written>
> > >> > > in other languages would encourage people to contribute more, I'm>
> > all>
> > >> for>
> > >> > > it.>
> > >> > >>
> > >> > > On Sat, Feb 10, 2018 at 7:42 AM Joe Witt 
> > >> > > <jo...@gmail.com<mailto:jo...@gmail.com>>>
> > wrote:>
> > >> > >>
> > >> > > > i personally would be ok with it for an extension/processor>
> > provided>
> > >> it>
> > >> > > > integrates well with the build.>
> > >> > > >>
> > >> > > > i would agree with andys view for core framework stuff but for>
> > >> > > extensions i>
> > >> > > > think we can do it like mikethomsen suggested.>
> > >> > > >>
> > >> > > > others?>
> > >> > > >>
> > >> > > > thanks>
> > >> > > > joe>
> > >> > > >>
> > >> > > > On Feb 10, 2018 7:30 AM, "Mike Thomsen" 
> > >> > > > <mi...@gmail.com<mailto:mi...@gmail.com>>>
> > >> wrote:>
> > >> > > >>
> > >> > > > > I'm just a community contributor, so take that FWIW, but a>
> > >> compromise>
> > >> > > > might>
> > >> > > > > be to publish the Scala code as separate maven modules to maven>
> > >> central>
> > >> > > > and>
> > >> > > > > then submit a thoroughly tested processor written in Java. As>
> > long>
> > >> as>
> > >> > > you>
> > >> > > > > have enough unit and integration tests to give strong coverage,>
> > I>
> > >> > > > wouldn't>
> > >> > > > > imagine anyone here would have issues reviewing it. If the tests>
> > >> fail>
> > >> > > > > because of code issues in the external dependencies, the obvious>
> > >> answer>
> > >> > > > is>
> > >> > > > > to just hold the PR until the tests pass.>
> > >> > > > >>
> > >> > > > > On Fri, Feb 9, 2018 at 9:00 AM, Weiss, Adam <>
> > >> > > adam.we...@perkinelmer.com<mailto:adam.we...@perkinelmer.com>>
> > >> > > > > wrote:>
> > >> > > > >>
> > >> > > > > > Devs,>
> > >> > > > > >>
> > >> > > > > > I have some interns starting with my team and we use Scala>
> > >> internally>
> > >> > > > for>
> > >> > > > > > our work.>
> > >> > > > > > If I wanted to have them work to contribute some new>
> > processors,>
> > >> > > would>
> > >> > > > > > they have to be Java to be included with the base>
> > distribution or>
> > >> > > could>
> > >> > > > > > they use Scala as long as it fit within your current build and>
> > >> test>
> > >> > > > > process?>
> > >> > > > > >>
> > >> > > > > > Thanks,>
> > >> > > > > > -Adam>
> > >> > > > > >>
> > >> > > > >>
> > >> > > >>
> > >> > >>
> > >> > -->
> > >> > Sent from Gmail Mobile>
> > >>>
> >>
>

Adam Weiss | Senior Manager, Digital Solutions R&D
PerkinElmer | Innovating for a Healthier World
adam.we...@perkinelmer.com<mailto:adam.we...@perkinelmer.com>
Mobile: +1 716.429.7324
77 Goodell St. Suite 470, Buffalo, NY, 14203
www.perkinelmer.com

Reply via email to