[
https://issues.apache.org/jira/browse/NIFI-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852137#comment-17852137
]
Bryan Bende commented on NIFI-13301:
------------------------------------
If NiFi Registry was configured with database flow persistence then multiple
instances behind a load balancer should work fine for HA, it does not need to
function as a cluster like NiFi with request replication.
That being said, I think we should be focusing more on building the right
integration points for NiFi, such as an API for Extension Registry Clients, and
implementations for common artifact repositories. After that anyone is welcome
to provide a public service for publishing/retrieving NARs with a corresponding
extension registry client.
For Apache, all released NARs are already published to Apache Nexus and
mirrored into Maven central, and Nexus would be one of the primary extension
registry clients, so that would take care of all Apache provided NARs. Ideally
there would be some way to discover community NARs as well, but just like the
code for those NARs lives outside Apache, I think those published NARs also
live somewhere outside Apache.
> Create ExtensionRegistryClient for External Extension Registry Interaction
> --------------------------------------------------------------------------
>
> Key: NIFI-13301
> URL: https://issues.apache.org/jira/browse/NIFI-13301
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions, NiFi Registry
> Environment: Linux: Ubuntu 20.04 or 22.04
> Reporter: James Guzman (Medel)
> Assignee: James Guzman (Medel)
> Priority: Minor
>
> *Objective:* Improve/enable sharing/reuse of at least two features of Apache
> NiFi, so the community can have an easier time contributing their flows
> and/or custom processors into a NiFi Marketplace:
> * {*}Pre-Designed Flows{*}: (Approach 1) stored in a NiFi Registry or
> accessible location. (Approach 2) NiFi Registry can also substituted by
> NiFi's git registry client having the location of the NiFi Marketplace.
> Thus, we just have a git location and NiFi uses that to find/store flows.
> * *Components/Processors* built in *Java/Python* that anyone is free to use
> * *End to End Full Stack Applications Powered By NiFi or MiNiFi CPP?* The
> frontend could be various ones from PyQT, ReactJS, H2O.ai Wave, 3D Slicer,
> OHIF Viewer, etc: (Maybe this one can be subscription based where users get
> access if they invest in the monthly or yearly subscription?)
> *Potential Solution:* Apache NiFi builds an extension point for interacting
> with Extension Registries, similar to how it currently has
> *{{FlowRegistryClient}}* with implementations for NiFi Registry and now
> GitHub, there would be *{{ExtensionRegistryClient}}* with implementations for
> stuff like Nexus, NiFi Registry, and any other vendor ones, for example maybe
> Datavolo provides one, but basically in apache we don't want to build another
> application, we already have NiFi Registry. (Briefly Discussed with [~bbende]
> )
> * I will start by looking into *{{FlowRegistryClient}}* and then base
> *{{ExtensionRegistryClient}}* toward that approach. If I am understanding
> correctly, we would contribute an *{{ExtensionRegistryClient}}* feature into
> Apache NiFi that enables NiFi to integrate with other vendors (Datavolo,
> H2Oai, etc) group of processors, data flow templates and so on. I see where
> you are coming from with for Apache the goal being not to build another
> application. That application could be by the another and NiFi's
> *{{ExtensionRegistryClient}}* hooks up to their {*}RegistryServer{*}. (Heres
> the path I will take toward that goal).
>
> {*}Project Ownership{*}: There will be a clear line that the NiFi Marketplace
> is not owned by the Apache NiFi PMC, rather it will be managed and owned by a
> vendor in close alignment with NiFi, which we will announce at a later time.
>
> {*}Motivation (NiFi){*}: Some people in the community have shown interest in
> having a NiFi Processor Marketplace where they can contribute their category
> of processors. In particular, I saw some people interested in contributing
> their NiFi python processors. This NiFi Processor marketplace could be also
> applied toward the community interested in contributing custom NiFi java
> processors.
> * {*}Extra MiNiFi C+{+}{{+}}{*}: We could even add a section for MiNiFi C+
> custom processors. There is a part of the MiNiFi build process that brings in
> NiFi through building the JNI extension. So, that is a way to integrate NiFi
> Registry there too.
> I have briefly talked with [~bbende] and [~joewitt] asking them questions on
> ways to bring custom python processors into production. One of the routes was
> through leveraging NiFi Registry in connection with Apache NiFi to streamline
> integration. For my use case, I would leverage the NiFi Processor Marketplace
> to start contributing my AI/DL "Medical Imaging" Pipeline of custom NiFi
> Python Processors where I focused on my master thesis AI/DL for stroke
> diagnosis.
> *High Level Details of NiFi Processor Marketplace App:* For a full stack NiFi
> Processor Marketplace App, there are frameworks we can leverage for the UI
> from ReactJS (this mainly is used for frontend), H2O.ai Wave (this can be
> used for full stack, but probably would mainly be used for the frontend
> portion) and other options. in the case that we want to make something more
> custom to display each category of processors. This full stack application
> could interact wth the NiFi Registry, which is the central location for
> storage and management of shared resources across one or more instances of
> NiFi and/or MiNiFi (C++). What are some hosting options for Apache projects?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
