+1 to all of this - I'm really looking forward to seeing more of this! Le lun. 16 nov. 2020 à 18:39, Kevin Doran <[email protected]> a écrit :
> Thanks for your attention and effort on this, Matt. I look forward to the > discuss threads, but at a high level, this sounds like a good and important > step for MiNiFi Java and C2. > > Cheers, > Kevin > > > On Nov 16, 2020, at 12:28, Matt Burgess <[email protected]> wrote: > > > > All, > > > > As you know, MiNiFi Java is a separate project and codebase than NiFi, > > it's my understanding that this was done because MiNiFi Java was a > > fledgling effort while NiFi was an established project, and trying to > > do rapid development and releases on MiNiFi would cause some > > interference with NiFi features, releases, etc. > > > > Since then, MiNiFi Java has advanced in its own right, but lately > > development and release cadence has slowed, and AFAICT much of the > > effort is to bring NiFi capabilities into MiNiFi, exposing previously > > ignored NiFi properties in MiNiFi's config.yml, etc. > > > > I've begun work on getting MiNiFi Java back into the NiFi codebase, > > under its own area with its own assembly, bootstrap, etc. This is > > being tracked in Jira under MINIFI-422 [1]. > > > > The idea is to be able to build a MiNiFi assembly from NiFi and MiNiFi > > components, thereby giving MiNiFi capabilities from NiFi without > > having to port them to a separate project. Kerberos support is a > > shining example of this, as are other features available via NiFi's > > various Context objects (where MiNiFi currently has them stubbed out). > > > > The first step was refactoring some NiFi NAR structure in order to > > allow for a "headless" NiFi [2][3], the idea being that we could reuse > > some existing framework components/NARs without being tied to the UI, > > associated webapps, and Jetty itself. > > > > I will create separate [DISCUSS] threads for each of the following, > > but here are some things I'm looking at going forward: > > > > 1) Bring MiNiFi bootstrap and external command-and-control (C2) code > > over to a MiNiFi area in the NiFi project structure. Also create a > > MiNiFi assembly in that area for the purposes of generating release > > artifacts when ready. > > > > 2) Bring over and update the C2 Server reference implementation, for > > testing and to give some way to externally update config/flows in > > MiNiFi. This may mean we don't need the change ingestors anymore, but > > I will send a separate DISCUSS thread to see who's using them, if > > they're important to keep, etc. > > > > 3) Discuss whether to bring over the config.yml mechanism for > > defining/updating configuration and flow. If we eventually want to > > share more between MiNiFi and NiFi, should we return to using > > nifi.properties and flow.xml.gz rather than maintaining (and updating) > > the code to expose nifi properties as human-readable YAML strings, to > > include all the schemas and converters? This too will be a separate > > [DISCUSS] thread. > > > > 4) Once all is determined and implemented, look at ways to share any > > components between NiFi and MiNiFi. This might include refactoring the > > C2 stuff into module(s) for use by NiFi [4], adding/enabling profiles > > and/or assemblies to build various products (headless NiFi with C2, > > MiNiFi with Elasticsearch NARs, e.g.) > > > > I plan on starting on 1 soon, but am glad to discuss any of these > > things or any other topics you'd like to bring up (for 2 and 3 let's > > have the discussion in the forthcoming DISCUSS threads) > > > > Regards, > > Matt > > > > [1] https://issues.apache.org/jira/browse/MINIFI-422 > > [2] https://issues.apache.org/jira/browse/NIFI-7592 > > [3] https://issues.apache.org/jira/browse/MINIFI-485 > > [4] https://issues.apache.org/jira/browse/NIFI-6813 > >
