ASF GitHub Bot commented on METRON-777:

Github user mmiklavc commented on the issue:

    Got it. Thanks @ottobackwards. 
    I want to repeat my appreciation for this contribution. I know it's taking 
us time to get through review, and there's a lot of work in merging ongoing 
changes, so thank you for being so patient and responsive!
    I have some comments around the documentation as it stands. I love the 
detail. This is probably more broadly and fully documented than many other 
pieces of Metron. The only issue I see right now is that in order to know which 
pages/READMEs are needed to add functionality requires the contributor to skim 
through the full doc TOC.
    I think it's still challenging to get a quick comprehension of the 
extension/bundle/assembly/parser architecture with this README doc layout. The 
subject separation is both useful and overwhelming at the same time. I want 
users and contributors alike to get a quick sense of how to add general purpose 
functionality, concrete OOTB core functionality, as well as their own custom 
functionality. In order to even understand or know that these levels of 
abstraction exist and then act on them, I found it necessary to read through 
the entire doc structure to get a sense of things. For example, this is the 
minimal set of docs a developer would need to read for managing 
parsers/bundles/extensions, and I collected this list by clicking through every 
link in the TOC. 
    - bundles-lib
    - bundles-maven-plugin
    - maven-archetypes
        - maven-parser-extension-archeytpe
    - extensions
        - deployment
        - packaging
        - terms
        - parser-extensions
            - adding system parsers
            - etc.
    - parsers
        - parser-testing
    I think we should have an overarching document that ties all of these 
components together cohesively from the perspective of extending Metron. The 
mindset I'm taking here is very use case driven. I'm thinking something like a 
document in the top-level of the project that covers "Extending Metron" and 
then provides very clear overview and path for a soup-to-nuts new parser 
stream. The outline might even look kind of like that list of projects outlined 
above (but organized in order of operation), with links to the relevant 
README's and an explanation of how it all fits together. A diagram like we have 
for the topologies would also be extremely useful (at some point).

> Create a plugin system for Metron based on 'NAR'
> ------------------------------------------------
>                 Key: METRON-777
>                 URL: https://issues.apache.org/jira/browse/METRON-777
>             Project: Metron
>          Issue Type: New Feature
>            Reporter: Otto Fowler
>            Assignee: Otto Fowler
> The success of the Metron project will be greatly dependent on community 
> participation, and with that the ability to adapt and extend Metron without 
> having to maintain a fork of the project.
> As organizations and individuals look to extend the Metron system with custom 
> parsers, enrichments, and stellar functions that may be proprietary in 
> nature, the ability to develop and deploy these extensions outside the Metron 
> code base is critically important.
> To that end, and after community discussion and proposal we create or 
> formalize the 'plugin' development story in Metron.  
> The proposal is to adapt the Apache Nifi NAR system for use in Metron.  This 
> will provide the system with:
> * archetype(s) for developer projects and independent development
> * defined packaging and metadata for 'plugin' products
> * loading and instantiation with classloader isolation capabilities
> * removing the necessity for shading plugin jars
> These capabilities will also enable other features, such as plugin lifecycle, 
> plugin configuration+redeployment, and other things.
> The plugin archetypes and their installation will be a followon

This message was sent by Atlassian JIRA

Reply via email to