Yeah for the three/commit James replied on in that link - i dont see how we can keep those. They import from something that I doubt very much is compatible with ALv2/ASF guidelines.
On Mon, May 4, 2020 at 12:01 PM Marc Parisi <[email protected]> wrote: > Afternoon Joe, > > "So in the case of minificpp we are certain there is no L&N concern > regarding either our source OR convenience binary?" > > I believe that to be the case; however, per > https://github.com/apache/nifi/pull/4242#issuecomment-622655276 I think > that maybe this is moot and the author can convert those processors in > MiNiFI to use that open source python lib. I'm not sure if he's on this > chain so I will follow up on the MiNiFi tickets to ensure that is the case > when I am able to log in again ( have to fix some account issue ). > > "Lastly, why are the JIRAs still open if this is merged? Can you please > close them and assign to the proper fix version if these are 'done'?" > > Sorry, when I am able to log in ( account issue ) and make the above > comment I will close the tx. I would prefer the fix be a follow on so this > is all tracked. The ticket fell off the radar ( as did the account issue, > to be frank ), so when that is resolved I will take these actions. You know > me Joe, I'm like a dog who sees a squirrel, but I've put a note on my > laptop to serve as a reminder. > > Thanks, > Marc > > On Mon, May 4, 2020 at 11:49 AM Joe Witt <[email protected]> wrote: > > > Marc > > > > I'm not opposed to it staying if there is zero license issue. I only > > mentioned the maintenance as a 'additional thing to consider' - that is > > more of a discussion item. L&N isn't discussion - it is hard rule. > > > > Ok so in the minificpp case you're saying "there is no source or binary > > dependency (that we'd package)" that isn't entirely ALv2/ASF compatible > and > > so if people want to use this they can do so by adding in the 'h2o' bits > at > > their own effort, right? In this line of thinking I still don't love it > > but it is arguably not a lot different that our components which talk to > > services you have to have a paid or otherwise agreement with like Cloud > > services offering by AWS, Azure, GCP, etc.. > > > > So in the case of minificpp we are certain there is no L&N concern > > regarding either our source OR convenience binary? > > > > Lastly, why are the JIRAs still open if this is merged? Can you please > > close them and assign to the proper fix version if these are 'done'? > > > > Thanks > > > > On Mon, May 4, 2020 at 11:46 AM Marc Parisi <[email protected]> wrote: > > > > > "We should find a way for vendors to offer extension points like this > > > without having to take on the burden of maintenance. How can we > possibly > > do > > > this well? We're learning this lesson the hard way in NiFi itself and > > this > > > is why the registry is being formulated." > > > > > > I think there is a fine line when walking vendor tie-in. Having the > > > registry allows for an arm's length agreement with the Apache > community. > > I > > > know others will disagree but my opinion is that extension points that > > > don't introduce licensing concerns from libraries should be fair game > but > > > the fact that a paid product requires a license for that code to be > > useful > > > should not be a non-starter -- if that makes sense. I believe I stated, > > "I > > > don't have a strong argument" as I don't really have a strong opinion ( > > and > > > certainly can be wrong too ) -- so I'm interested in hearing others in > > > regards to this. > > > > > > > > > > > > On Mon, May 4, 2020 at 11:38 AM Marc Parisi <[email protected]> > wrote: > > > > > > > Hi Joe, > > > > > > > > When merging I did not use the NiFi processors to test as I already > > have > > > > tooling around H20 driven ML in my home. While I don't admittedly use > > it > > > > very often, I thought the python processors were quite cool and could > > > > certainly be useful for others. My rationale is that the dependencies > > are > > > > not built into source or binary for MiNiFi C++. I would agree on the > > Java > > > > side that this would be unavoidable. While it would be preferential > to > > > have > > > > the NiFi analogue I do not think it would be required when > considering > > > the > > > > MiNiFi C++ processors; however, James should be the arbiter of that > > > > decision. > > > > > > > > While I disagree that the MiNiFi portion should be removed, I don't > > have > > > a > > > > strong argument to keep or remove the MiNiFI C++ code so I'd be happy > > to > > > > revert if the community feels the need to. > > > > > > > > Thanks, > > > > Marc > > > > > > > > On Mon, May 4, 2020 at 10:50 AM Joe Witt <[email protected]> wrote: > > > > > > > >> Team > > > >> > > > >> The below two commits raise serious concerns. > > > >> > > > >> I want to be clear here and point out that h2o is cool. Having such > > > >> integration is a neat concept and idea and one that certainly > warrants > > > >> determination on how best to do so. > > > >> > > > >> My issue is with these two commits as it relates to licensing and > > > >> maintenance. > > > >> > > > >> License: > > > >> We should support vendors like this wanting to bring their > > capabilities > > > >> into NiFi generally sure. But the licensing and mode of use is > > critical. > > > >> In talking about this with the contributor for NiFi as well it is > > clear > > > >> that at least some or important portions of this require the user to > > > have > > > >> a > > > >> 'driverless ai license' so they can include their jar or build > > pipelines > > > >> or > > > >> whatnot. Thus it isn't usable without that first. So it might be > the > > > case > > > >> that this stuff is source dependent only on ASF compatible licenses > in > > > >> terms of source - but it certainly doesn't seem to be true in terms > of > > > >> binary dependencies. Where in the PR or JIRA is there any discussion > > or > > > >> review of the licensing? I see plenty from James to believe this > > needs > > > to > > > >> be reverted. Any contrary info? > > > >> https://github.com/apache/nifi/pull/4242#issuecomment-622654527 > > > >> > > > >> > > > >> Maintenance: > > > >> We should find a way for vendors to offer extension points like this > > > >> without having to take on the burden of maintenance. How can we > > possibly > > > >> do > > > >> this well? We're learning this lesson the hard way in NiFi itself > and > > > >> this > > > >> is why the registry is being formulated. > > > >> > > > >> JIRA/hygiene: > > > >> https://issues.apache.org/jira/browse/MINIFICPP-1199 > > > >> and > > > >> https://issues.apache.org/jira/browse/MINIFICPP-1201 > > > >> Both are open. Yet we've merged commits that claim to be against > > each. > > > >> > > > >> I believe both of these commits need to be reverted as I do not > > believe > > > >> the > > > >> licensing considerations have been addressed. I'd like to see > > > discussion > > > >> on the above maintenance concern as well but that is less pressing. > > > >> > > > >> > > > >> Thanks > > > >> Joe > > > >> > > > >> > > > >> commit 7206c62240647520cf35649868d5d87903a256c2 > > > >> Author: James Medel <[email protected]> > > > >> Date: Wed Apr 29 12:38:04 2020 -0700 > > > >> > > > >> MINIFI-1201: Integrate H2O Driverless AI MSP in MiNFi (#766) > > > >> > > > >> MINIFI-1201: Integrate H2O Driverless AI MSP in MiNFi (#766) > > > >> > > > >> commit 6e5f96518764df7791519c0ee625a94a207ddc69 > > > >> Author: James Medel <[email protected]> > > > >> Date: Wed Apr 29 12:37:00 2020 -0700 > > > >> > > > >> MINIFI-1199: Integrate H2O Driverless AI PSP in MiNiFi (#763) > > > >> > > > >> * MINIFI-1199: Integrate H2O Driverless AI in MiNiFi > > > >> > > > >> MiNiFi C++ and H2O Driverless AI Integration via Custom Python > > > >> Processors: > > > >> Integrates MiNiFi C++ with H2O's Driverless AI by Using > Driverless > > > >> AI's > > > >> Python Scoring Pipeline and MiNiFi's Custom Python Processors. Uses > > the > > > >> Python Processors to execute the Python Scoring Pipeline scorer to > do > > > >> batch > > > >> scoring and real-time scoring for one or more predicted labels on > test > > > >> data > > > >> in the incoming flow file content. I would like to contribute my > > > >> processors > > > >> to MiNiFi C++ as a new feature. > > > >> > > > >> * Update H2oPspScoreBatches processor > > > >> > > > >> This update includes passing "index=False" to > > > >> "batch_scores_df.to_string(index=False)". By updating this line of > > code, > > > >> we > > > >> tell pandas to not include the DataFrame index when converting the > > > >> DataFrame to a string. The reason for this update is that we don't > > want > > > >> this extra column pandas tries to add to our predictions frame, we > > only > > > >> want the predictions. Thus, later when the predictions get saved to > a > > > csv > > > >> file, it will only include the predictions. > > > >> > > > >> * Moved ConvertDsToCsv to h2o base dir > > > >> > > > >> Since this ConvertDsToCsv python processor is used by the > > > >> Python Scoring Pipeline and MOJO Scoring Pipeline processors, > > > >> I moved ConvertDsToCsv to h2o base dir for easier access to it. > > > >> > > > > > > > > > >
