I think this situation is sort of analogous to some other scenarios we have run into, such as the integrations between NiFi and stream processing.
For example, the storm and spark integration lives in NiFi's code base, and NiFi is responsible for determining if a new version of those APIs is available, updating the dependency, and resolving any necessary code changes. The Flink and Apex integration lives in each of those respective code bases, so if Flink changes all of their APIs, they will be forced to update the NiFi Flink source/sink in order to make their build pass. I can see value in either approach. If we consider the JNI stuff to be a more generic capability, then I think I lean towards NiFi owning it. Even with that, MiNiFi will still be dependent on some version of whatever JNI artifacts NiFi produces, so MiNiFi would still have ownership over when to change that dependency and ensure that it still works properly in a given release of MiNiFi. On Wed, Feb 27, 2019 at 12:02 PM Arpad Boda <[email protected]> wrote: > > Hi, > > In my opinion, there is another question here to be answered: what version of > NiFi NARs do we expect to work in MiNiFi? > > I see two different approaches here: > -#1) MiNiFi is compatible with a given NiFi version, like 1.9.0, NARs in that > release work properly in MiNiFi. > In this case the tests belong to MiNiFi and NiFi there becomes a dependency > -> the tests are required to pass when upgrading the dependency. > -#2) MiNiFi is compatible with master. This means that whatever we introduce > in NiFi shouldn't break, so the tests more belong to NiFi. > > According to my experience preventing the merge of a breaking change is > easier than fixing later, which would make me vote for #2, however this > approach has a side effect, too: introducing a breaking change (that MiNiFi > needs to adapt) becomes a huge pain, it's much easier in #1. > > Regards, > Arpad > > On 27/02/2019, 17:44, "Marc Parisi" <[email protected]> wrote: > > Hi , > I've submitted a PR to Apache NiFi MiNiFi C++ that enables the project > to access and run NiFi processors. This requires that certain jars be > included to bootstrap that process long with a set of NARs from which we > will access the desired components. Effectively this means that a user can > run the current set of C++ processors alongside Java processors. > To facilitate this feature's use, I submitted a PR [1] to NiFi with > that > assembles the appropriate JARs and a basic set of useful/necessary NARs. > I'd like opinions on where this should live. The options have been to > place > it within Apache NiFi MiNIFi C++ ( specifically within the aforementioned > feature's build directory) as it only relates to that project and, perhaps > allowing us to more easily address problems...or as per the PR: placing it > in NiFi proper, hopefully building regression tests that notify us of > issues before a release. > > I've been wavering on the options and want community opinion. The end > goal in either case is that if/when a user builds with this functionality > we can deliver a package that runs out of the box. This can be done with > either approach. > > The net positive of placing it in NiFi as I see it: sense that it is > an > official feature and perhaps notification before NiFi release or in > travis. Side effects include: questioning if this belongs here, hair > loss, > and perhaps a sudden loss of appetite. > > The net positive of isolating this within MiNIFi C++: cleaner end > solution, it's produced in the same place it's consumed, ability to notify > of problems in our CI and release process ( this could be viewed as a > negative too ? ). > > TL;DR: Where feature Live? > > Any insight/opinions are appreciated. Thanks! > > [1] https://github.com/apache/nifi/pull/3330 > >
