[
https://issues.apache.org/jira/browse/METRON-777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16106777#comment-16106777
]
ASF GitHub Bot commented on METRON-777:
---------------------------------------
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/530
"Based on feedback from reviews and discussion with mattf, this commit…
… represent a major refactoring for the bundle functionality.
While trying to resolve the ugliness bought about by working around VFS's
read only support for HDFS, I have come up with a different approach.
While this approach makes bundles-lib significantly different than
nar-utils, it makes it much more simple, and may arguably be an approach worth
concidering.
The original bundles-lib maintained the nar system's method of unpacking
the bundle into a 'working' directory, and then using having the classloader
load from that directory structure. Trying to avoid too far a departure (
at the time ) from nifi, bundle-lib maintained the 'create folders and files as
nec'
scheme found in nifi. Since nifi is always working with local disk and
File objects, this is ok. But with VFS and HDFS this did not work. The work
around
was to create a poop static 'class cluster' type step with multiple
implemenations of FileUtils ( VFS and HDFS ).
The new approach, which is a more willing departure from the nifi approach,
is to avoid having to ever do writes ( file or folder creation ) from
bundles-lib at all.
This allows for a simplier, single implementation of the file utils and the
removal of the anti-pattern. It also removes the need for a 'working' area.
The new approach is to better utilize the capabilities of VFS such that we
never have to unpack the bundle at all. VFS can load JAR files as Filesystems.
I have created
a new Classloader ( based on the VFS classloader implemenation from the VFS
project ) that is bundle aware. It can load the bundle and it's dependencies
as resources
and make them available. This is what removes the requirement to unpack
the bundles into a working directory.
As such, the bundle-lib has been refactored to move from the 'working
directory' to only needing to work with the bundle in the lib. I have also
removed the doc directory support
since we are not using it.
We no longer need multiple file utils, the hdfs working directory, and
other things.
Thanks to mattf for the time and discussion."
> Create a plugin system for Metron based on 'NAR'
> ------------------------------------------------
>
> Key: METRON-777
> URL: https://issues.apache.org/jira/browse/METRON-777
> Project: Metron
> Issue Type: New Feature
> Reporter: Otto Fowler
> Assignee: Otto Fowler
>
> The success of the Metron project will be greatly dependent on community
> participation, and with that the ability to adapt and extend Metron without
> having to maintain a fork of the project.
> As organizations and individuals look to extend the Metron system with custom
> parsers, enrichments, and stellar functions that may be proprietary in
> nature, the ability to develop and deploy these extensions outside the Metron
> code base is critically important.
> To that end, and after community discussion and proposal we create or
> formalize the 'plugin' development story in Metron.
> The proposal is to adapt the Apache Nifi NAR system for use in Metron. This
> will provide the system with:
> * archetype(s) for developer projects and independent development
> * defined packaging and metadata for 'plugin' products
> * loading and instantiation with classloader isolation capabilities
> * removing the necessity for shading plugin jars
> These capabilities will also enable other features, such as plugin lifecycle,
> plugin configuration+redeployment, and other things.
> The plugin archetypes and their installation will be a followon
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
