I think we should not add blindly any language but should be open to add
couple of language like Scala.
In Bigdata world Scala/Python/Java are widely accepted.
Thanks,
Milan Das
Interset
On 2/13/18, 10:20 AM, "Weiss, Adam" <[email protected]> wrote:
I think it makes the most sense to me for us to publish a separate
repo with a module and nar build for now and post when it's available in
the users group.
Thanks for the discussion everyone, hopefully we can start making some
helpful contributions soon.
-Adam
On 2018/02/10 23:43:31, Tony Kurc <[email protected]<mailto:t...@
gmail.com>> wrote:
> It is like Matt read my mind.>
>
> On Sat, Feb 10, 2018 at 6:26 PM, Matt Burgess <[email protected]
<mailto:[email protected]>> wrote:>
>
> > I'm fine with a vote, but I'll be voting to keep Java as the
single>
> > language for the (non-test) code. I share the same concerns as
many of>
> > the other folks as far as accepting other languages, it's mainly
the>
> > "slippery slope" argument that I don't want to turn into a>
> > JVM-language flame war. If Scala, why not Groovy? Certainly the>
> > syntax is closer to Java, and the community has accepted it as a
valid>
> > language for writing unit tests, although we stopped short for>
> > allowing it for the deployable NiFi codebase, for the same reasons>
> > IIRC. If Scala and/or Groovy, why not Kotlin? The same argument>
> > (albeit more tenuous) goes for Clojure and just about every other
JVM>
> > language (although I don't expect a call for LuaJ processors lol).>
> >>
> > Whether we decide to support various languages ad-hoc or not, I
would>
> > strenuously object to multiple/hybrid build systems for the
deployed>
> > artifacts. If I could switch NiFi completely to Gradle I would,
but I>
> > realize there are good reasons for not doing so (yet?) in the
Apache>
> > NiFi community, and I would never want any hybrid Maven/Gradle
build>
> > for the deployable code, likewise for SBT, Leiningen, etc. With a>
> > custom Mojo for Maven NAR builds, and the complexity for hybrid
builds>
> > in general, I think this would create a maintenance nightmare.>
> >>
> > The language thing is a tough decision though, it's not awesome
that>
> > specifying a single language can be a barrier to a more diverse>
> > community, certainly Scala-based bundles would be more than
welcome in>
> > the overall NiFi ecosystem, I just think the cons outweigh the
pros>
> > for the baseline code. I've written Groovy processors/NARs using>
> > Gradle as the build system, and I'm good with keeping them in my
own>
> > repo, especially when the Extension Registry becomes a thing. I
can>
> > see the Extension Registry perhaps making this a moot point, but>
> > clearly we need to have this discussion in the meantime.>
> >>
> > Regards,>
> > Matt>
> >>
> >>
> > On Sat, Feb 10, 2018 at 5:23 PM, Andrew Grande <[email protected]
<mailto:[email protected]>> wrote:>
> > > Wasn't there a warning trigger about the NiFi distro size from
Apache>
> > > recently? IMO, before talking alternative languages, solve the
modularity>
> > > and NAR distribution problem. I think the implementation of a
module>
> > won't>
> > > matter much then, the point being not everything has to go in
the core,>
> > > base distribution, but can still be easily sourced from a known
repo, for>
> > > example.>
> > >>
> > > I have a feeling NiFi 1.6+ can be approaching 2GB distro size
soon :)>
> > >>
> > > Andrew>
> > >>
> > > On Sat, Feb 10, 2018, 5:12 PM Joey Frazee <[email protected]
<mailto:[email protected]>>>
> > wrote:>
> > >>
> > >> This probably necessitates a vote, yeah?>
> > >>>
> > >> Frankly, I’m usually happier writing Scala, and I’ve not
encountered any>
> > >> problems using processors written in Scala, but I think it’ll
be>
> > important>
> > >> to tread lightly.>
> > >>>
> > >> There’s a few things that pop into my head:>
> > >>>
> > >> - Maintainability and reviewability. A very very good Java
developer>
> > need>
> > >> not, by definition, either know how to write or identify good
Scala or>
> > spot>
> > >> problems and bugs.>
> > >> - Every Scala processor would either end up with a 5MB
scala-lang.jar>
> > >> packaged into the .nar or we’d have to start including it in
the core>
> > >> somewhere, if it’s not. It’s possible it might have already
gotten>
> > pulled>
> > >> up from other dependencies.>
> > >> - Style. There’s a tremendous amount of variation in Scala
style because>
> > >> of its type system, implicits, macros, and functional nature.
There are>
> > >> very good people out there that can write good Scala that isn’t>
> > readable by>
> > >> the 99%.>
> > >> - Binary compatibility. Scala tends to be a little more brazen
about>
> > >> breaking binary compatibility in major releases and those
happen a bit>
> > more>
> > >> often than with Java. That’s not a problem for any potential
source>
> > code in>
> > >> the project, but it could present some dependency issues
someday.>
> > >> - Testing. There’s N > 1 test frameworks and testing styles
within>
> > those,>
> > >> so there’s a lot of options for introducing more variability
into the>
> > tests.>
> > >> - NiFi uses a lot of statics in setting up properties and
relationships>
> > >> and the like, and idiomatic Scala approaches that stuff a bit>
> > differently,>
> > >> so it’ll be necessary to impose some style guidelines so there
isn’t too>
> > >> much variation.>
> > >>>
> > >> That said, there are some things that won’t be problematic:>
> > >>>
> > >> - As mentioned, processors written in Scala do just work.>
> > >> - The scala-maven-plugin works just fine allowing mixed
Java-Scala>
> > >> projects (btw, it’d probably be not super great to do mixed
Java-Scala>
> > and>
> > >> mixed Maven-SBT though).>
> > >> - A lot of the above concerns could be addressed by having
clear style>
> > >> guidelines.>
> > >>>
> > >> Another thing: most of the projects I see deliver separate jars
for>
> > Scala>
> > >> components are delivering idiomatic APIs wrapping Java (or vice
versa).>
> > I>
> > >> think publishing a separate set of jars/nars for stuff
written in>
> > Scala>
> > >> would be odd since here it’d mostly be processors with new
functionality>
> > >> and not functionality for using Scala. I could imagine a lib of>
> > implicits,>
> > >> traits, classes that could make the Scala development more
enjoyable.>
> > That>
> > >> probably would make sense to deliver that way.>
> > >>>
> > >> -joey>
> > >>>
> > >> On Feb 10, 2018, 10:33 AM -0600, Bryan Bende <[email protected]
<mailto:[email protected]>>, wrote:>
> > >> > I agree more with Andy about sticking with Java. The more
varying>
> > >> languages>
> > >> > used, the more challenging it is to maintain. Once the code
is part of>
> > >> the>
> > >> > Apache NiFi git repo, it is now the responsibility of the
committers>
> > and>
> > >> > PMC members to maintain it.>
> > >> >>
> > >> > I’d even say I am somewhat against the groovy/Spock test code
that>
> > Andy>
> > >> > mentioned. I have frequently spent hours trying to fix a
Spock test>
> > that>
> > >> > broke from something I was working on. Every committer is
familiar>
> > with>
> > >> > JUnit, but only a couple know Spock. Just using this as an
example>
> > that>
> > >> > every committer knows Java, but only a couple probably know
Scala,>
> > >> Clojure,>
> > >> > etc.>
> > >> >>
> > >> > On Sat, Feb 10, 2018 at 10:25 AM Jeff <[email protected]
<mailto:[email protected]>> wrote:>
> > >> >>
> > >> > > +1 to Joe's response. If you can develop a component in
Groovy or>
> > Scala>
> > >> > > (or Clojure!) more quickly/comfortably, or if allowing
components>
> > >> written>
> > >> > > in other languages would encourage people to contribute
more, I'm>
> > all>
> > >> for>
> > >> > > it.>
> > >> > >>
> > >> > > On Sat, Feb 10, 2018 at 7:42 AM Joe Witt <[email protected]
<mailto:[email protected]>>>
> > wrote:>
> > >> > >>
> > >> > > > i personally would be ok with it for an
extension/processor>
> > provided>
> > >> it>
> > >> > > > integrates well with the build.>
> > >> > > >>
> > >> > > > i would agree with andys view for core framework stuff
but for>
> > >> > > extensions i>
> > >> > > > think we can do it like mikethomsen suggested.>
> > >> > > >>
> > >> > > > others?>
> > >> > > >>
> > >> > > > thanks>
> > >> > > > joe>
> > >> > > >>
> > >> > > > On Feb 10, 2018 7:30 AM, "Mike Thomsen" <[email protected]
<mailto:[email protected]>>>
> > >> wrote:>
> > >> > > >>
> > >> > > > > I'm just a community contributor, so take that FWIW,
but a>
> > >> compromise>
> > >> > > > might>
> > >> > > > > be to publish the Scala code as separate maven modules
to maven>
> > >> central>
> > >> > > > and>
> > >> > > > > then submit a thoroughly tested processor written in
Java. As>
> > long>
> > >> as>
> > >> > > you>
> > >> > > > > have enough unit and integration tests to give strong
coverage,>
> > I>
> > >> > > > wouldn't>
> > >> > > > > imagine anyone here would have issues reviewing it. If
the tests>
> > >> fail>
> > >> > > > > because of code issues in the external dependencies,
the obvious>
> > >> answer>
> > >> > > > is>
> > >> > > > > to just hold the PR until the tests pass.>
> > >> > > > >>
> > >> > > > > On Fri, Feb 9, 2018 at 9:00 AM, Weiss, Adam <>
> > >> > > [email protected]<mailto:Adam.Weiss@perkinelmer.
com>>
> > >> > > > > wrote:>
> > >> > > > >>
> > >> > > > > > Devs,>
> > >> > > > > >>
> > >> > > > > > I have some interns starting with my team and we use
Scala>
> > >> internally>
> > >> > > > for>
> > >> > > > > > our work.>
> > >> > > > > > If I wanted to have them work to contribute some new>
> > processors,>
> > >> > > would>
> > >> > > > > > they have to be Java to be included with the base>
> > distribution or>
> > >> > > could>
> > >> > > > > > they use Scala as long as it fit within your current
build and>
> > >> test>
> > >> > > > > process?>
> > >> > > > > >>
> > >> > > > > > Thanks,>
> > >> > > > > > -Adam>
> > >> > > > > >>
> > >> > > > >>
> > >> > > >>
> > >> > >>
> > >> > -->
> > >> > Sent from Gmail Mobile>
> > >>>
> >>
>
Adam Weiss | Senior Manager, Digital Solutions R&D
PerkinElmer | Innovating for a Healthier World
[email protected]<mailto:[email protected]>
Mobile: +1 716.429.7324
77 Goodell St. Suite 470, Buffalo, NY, 14203
www.perkinelmer.com
