[
https://issues.apache.org/jira/browse/METRON-777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120524#comment-16120524
]
ASF GitHub Bot commented on METRON-777:
---------------------------------------
Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/530
Got it. Thanks @ottobackwards.
I want to repeat my appreciation for this contribution. I know it's taking
us time to get through review, and there's a lot of work in merging ongoing
changes, so thank you for being so patient and responsive!
I have some comments around the documentation as it stands. I love the
detail. This is probably more broadly and fully documented than many other
pieces of Metron. The only issue I see right now is that in order to know which
pages/READMEs are needed to add functionality requires the contributor to skim
through the full doc TOC.
I think it's still challenging to get a quick comprehension of the
extension/bundle/assembly/parser architecture with this README doc layout. The
subject separation is both useful and overwhelming at the same time. I want
users and contributors alike to get a quick sense of how to add general purpose
functionality, concrete OOTB core functionality, as well as their own custom
functionality. In order to even understand or know that these levels of
abstraction exist and then act on them, I found it necessary to read through
the entire doc structure to get a sense of things. For example, this is the
minimal set of docs a developer would need to read for managing
parsers/bundles/extensions, and I collected this list by clicking through every
link in the TOC.
- bundles-lib
- bundles-maven-plugin
- maven-archetypes
- maven-parser-extension-archeytpe
- extensions
- deployment
- packaging
- terms
- parser-extensions
- adding system parsers
- etc.
- parsers
- parser-testing
I think we should have an overarching document that ties all of these
components together cohesively from the perspective of extending Metron. The
mindset I'm taking here is very use case driven. I'm thinking something like a
document in the top-level of the project that covers "Extending Metron" and
then provides very clear overview and path for a soup-to-nuts new parser
stream. The outline might even look kind of like that list of projects outlined
above (but organized in order of operation), with links to the relevant
README's and an explanation of how it all fits together. A diagram like we have
for the topologies would also be extremely useful (at some point).
> Create a plugin system for Metron based on 'NAR'
> ------------------------------------------------
>
> Key: METRON-777
> URL: https://issues.apache.org/jira/browse/METRON-777
> Project: Metron
> Issue Type: New Feature
> Reporter: Otto Fowler
> Assignee: Otto Fowler
>
> The success of the Metron project will be greatly dependent on community
> participation, and with that the ability to adapt and extend Metron without
> having to maintain a fork of the project.
> As organizations and individuals look to extend the Metron system with custom
> parsers, enrichments, and stellar functions that may be proprietary in
> nature, the ability to develop and deploy these extensions outside the Metron
> code base is critically important.
> To that end, and after community discussion and proposal we create or
> formalize the 'plugin' development story in Metron.
> The proposal is to adapt the Apache Nifi NAR system for use in Metron. This
> will provide the system with:
> * archetype(s) for developer projects and independent development
> * defined packaging and metadata for 'plugin' products
> * loading and instantiation with classloader isolation capabilities
> * removing the necessity for shading plugin jars
> These capabilities will also enable other features, such as plugin lifecycle,
> plugin configuration+redeployment, and other things.
> The plugin archetypes and their installation will be a followon
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
