Bryan, Good luck. Godspeed, my friend. As a hard-core NiFi user, I can’t wait for the illusive extension registry to be a reality.
Rick. -- Richard St. John, PhD Asymmetrik AGILITY | EXPERIENCE | RESULTS 308 Sentinel Drive, Suite 400 Annapolis Junction, MD 20701 On Nov 13, 2018, 12:37 PM -0500, Joe Witt <[email protected]>, wrote: > Bryan > > Very exciting to see this under way!!! We desperately have to get our > core nifi build size way down and make it far easier for people to > contribute new processors. We have a long line of extensions that > could be really useful/valuable and this will help unlock that while > improving the user experience tremendously. > > For the doc/concerns noted above we should also add/mention that the > relationship between nars (dependencies between nars for > apis/controller services/parent nars/etc..) we want to have a way to > manage/show that or a user experience for it. Maybe not a day-1 thing > but important to call out. > > Thanks! > Joe > On Tue, Nov 13, 2018 at 12:22 PM Bryan Bende <[email protected]> wrote: > > > > All, > > > > We've needed the elusive extension registry for quite some time now, > > and with NiFi Registry I think we are in a good place to make some > > progress in this area. > > > > I've started looking into adding "extension bundles" to NiFi Registry > > as the next type of versioned item, along side the existing versioned > > flows, and I wanted to take a minute to outline how that approach > > could work before getting too far into it. > > > > Also, I'd like to focus this discussion on the design and > > functionality of the extension registry, and not on how the community > > is going to use it. Topics like hosting a central registry, changing > > the build process, restructuring the git repo, releasing NARs, etc, > > are all important topics, but first we need an extension registry > > before we can do any of that :) > > > > Here is a high-level description of what needs to be done... > > > > NiFi Registry > > > > - Add a new type of item called an extension bundle, where each bundle > > can contain one ore extensions or APIs > > > > - Support bundles for traditional NiFi (aka NARs) and also bundles for > > MiNiFi CPP > > > > - Ability to upload the binary artifact for a bundle and extract the > > metadata about the bundle, and metadata about the extensions contained > > in the bundle (more on this later) > > > > - Provide a pluggable storage provider for saving the content of each > > extension bundle so that we can have different implementations like > > local fileysystem, S3, and other object stores > > > > - Provide a REST API for listing and retrieving available bundles, > > integrate this into the registry Java client and NiFi CLI > > > > NAR Maven Plugin > > > > - Generate a descriptor for each component in the NAR which will > > provide information like the description, tags, restricted or not, > > etc. > > > > - These descriptors will be parsed by NiFi Registry when a NAR is > > being uploaded so that NiFi Registry will know about the extensions > > contained with in the NAR > > > > NiFi > > > > - Provide some type of extension manager experience where users can > > search, browse, & install bundles that are available in any of the > > registry clients that have been defined > > > > - Introduce a new security policy to control which users are allowed > > to access the extension manager > > > > - Installing a bundle should load the NAR and make the extensions > > available leveraging the recent work done to auto-load new NARs > > > > - Importing versioned flows from registry should provide an easy way > > to install bundles that are required by the flow, but missing from the > > local NiFi instance > > > > > > If anyone has any thoughts or concerns about this approach, please let me > > know. > > > > Thanks, > > > > Bryan
