Re: [DISCUSS] SIDELOADING PARSERS: Packaging and Loading and Extensions [oh.my]

[James Sirota]

Sounds like a great proposal. Im excited to see what you come up with.

james 

14.03.2017, 19:56, "Otto Fowler" :
> RE: questions -
>
> - The archetype will produce NARs so having it with the VFSClassloader
> doesn’t work, unless
> I do an archetype without nars, but that would be so short lived I wouldn’t
> bother.
> - My goal is to do it in such a way that I could propose / submit the
> changes made back to nifi ( worse case is work alike ).
> It is not out of the questions that between metron-nar ( or whatever I call
> it ) and nifi-nar, we come up with
> a 3rd thing that both switch to, that is generic and cross project. To
> that end, in the work I’m doing I am keeping dependencies
> to metron or nifi to zero.
>
> I am still doing some poc work. But the nar work ( that I will be putting
> into jira ) will be:
>
> * nar-utils ( the class loaders and expanders ) to a top level but not
> dependent metron library
> * refactor nar -utils to be independent of the nifi-properties and more
> generically configurable
> * replace the nifi file utils and file / uri class loaders with VFS
> FileSystemManager based implementation. ( this will work for them too, and
> I will be posting it back to them )
> * fork the nifi maven plugin and make it configurable as to output name
> extension and metadata names ( XXX-ID instead of Nar-ID ) - this too I
> will propose as a PR back
>
> I feel the right thing to do, when landing the parsers as separate, is not
> what I have done so far. Each parser should be a product of the archetype
> themselves.
> That is how I intend to proceed.
>
> On March 14, 2017 at 17:42:28, Kyle Richardson (kylerichards...@gmail.com)
> wrote:
>
> I like the direction of using NAR. The key benefits I see (there are
> others, but I picked by favorites):
>
> 1. Move away from the uber jars in Storm. Granted, this should be possible
> with modifications from the new Stellar classloader. My hope is that this
> would allow us to stop shading the parser project packages.
> 2. Allow parser development to be somewhat independent of the core code
> base. I do think we'll have to address versioning and backwards
> compatibility.
> 3. "Drop in" extensions to Metron. Future expansion beyond parsers.
>
> Questions:
> - Is there a way to do 1 using the VFS Classloader and land the archetype
> as an MVP? If so, we could avoid shading as part of the archetype and maybe
> iterate from there on it. Do you think this would still be too much change
> to the archetype after putting it out there?
> - How would we adapt NAR from Nifi without frankensteining it? I'm all for
> code reuse but, ideally, we'd like Nifi and Metron's versions of NAR to end
> up converging in the end.
>
> On Fri, Mar 10, 2017 at 3:43 PM, Casey Stella  wrote:
>
>>  I would definitely agree that moving forward we should consider something
>>  like Nar for Stellar. I'm not seeing the need for parsers exactly.
>>
>>  I don't want to squash the forward thinking aspect here; we should be
>
> broad
>>  and think about the end, ideal state. I just want to make sure we think
>>  through something that we can iterate on as an initial state that still
>>  solves your problem, MVP style.
>>
>>  On Fri, Mar 10, 2017 at 3:39 PM, Otto Fowler 
>>  wrote:
>>
>>  > "The Apache NiFi NAR ‘system’ allows for the packaging and loading of
>>  java
>>  > resources with classloader isolation.
>>  > Although technically it is the Service Provider api that makes the
>>  > ‘plugins’ part of the system, you can view them
>>  > together, and thus look at the NAR features as a system to create,
>>  > package, load, and execute plugins in a java system
>>  > while maintaining classloader isolation and dependency separation.
>>  >
>>  > While the NiFi problem case ( many plugins possibly executing in the
>
> same
>>  > vm ) is not universal, the functionality provided
>>  > by NAR is commonly needed, and is indeed functionality that I am
>>  currently
>>  > looking at implementing in the Apache Metron project.”
>>  >
>>  > This is how I put it to Joe.
>>  >
>>  > I think what you are proposing would work. I think what I have done up
>>  > until now will pretty much work. What I have been thinking about and
>>  > considering
>>  > is the difference between getting ‘something that works’, and maybe
>>  > something better.
>>  >
>>  > So if you look at nar there is the ‘packaging’ part, and the class
>>  loading
>>  > part.
>>  > We are already doing almost the same thing with the assembly of the
>>  > .tag.gz. The Nar is a next step to this which adds more metadata and
>
> the
>>  > dependency repo.
>>  > More of a refinement than a change.
>>  >
>>  > As far as class loading, the Nar is a more refined system for deploying
>>  > and consuming jars and dependencies, and setting up classloader
>>  instances.
>>  > It has more
>>  > functionality than we need at the moment in storm, but in other
>
> services
>> 

Re: [DISCUSS] SIDELOADING PARSERS: Packaging and Loading and Extensions [oh.my]

[Otto Fowler]

RE: questions -

- The archetype will produce NARs so having it with the VFSClassloader
doesn’t work, unless
I do an archetype without nars, but that would be so short lived I wouldn’t
bother.
- My goal is to do it in such a way that I could propose / submit the
changes made back to nifi ( worse case is work alike ).
It is not out of the questions that between metron-nar ( or whatever I call
it ) and nifi-nar, we come up with
a 3rd thing that both switch to, that is generic and cross project.  To
that end, in the work I’m doing I am keeping dependencies
to metron or nifi to zero.

I am still doing some poc work.  But the nar work ( that I will be putting
into jira ) will be:

* nar-utils ( the class loaders and expanders ) to a top level but not
dependent metron library
* refactor nar -utils to be independent of the nifi-properties and more
generically configurable
* replace the nifi file utils and file / uri class loaders with VFS
FileSystemManager based implementation. ( this will work for them too, and
I will be posting it back to them )
* fork the nifi maven plugin and make it configurable as to output name
extension  and metadata names ( XXX-ID instead of Nar-ID ) - this too I
will propose as a PR back


I feel the right thing to do, when landing the parsers as separate, is not
what I have done so far.   Each parser should be a product of the archetype
themselves.
That is how I intend to proceed.


On March 14, 2017 at 17:42:28, Kyle Richardson (kylerichards...@gmail.com)
wrote:

I like the direction of using NAR. The key benefits I see (there are
others, but I picked by favorites):

1. Move away from the uber jars in Storm. Granted, this should be possible
with modifications from the new Stellar classloader. My hope is that this
would allow us to stop shading the parser project packages.
2. Allow parser development to be somewhat independent of the core code
base. I do think we'll have to address versioning and backwards
compatibility.
3. "Drop in" extensions to Metron. Future expansion beyond parsers.

Questions:
- Is there a way to do 1 using the VFS Classloader and land the archetype
as an MVP? If so, we could avoid shading as part of the archetype and maybe
iterate from there on it. Do you think this would still be too much change
to the archetype after putting it out there?
- How would we adapt NAR from Nifi without frankensteining it? I'm all for
code reuse but, ideally, we'd like Nifi and Metron's versions of NAR to end
up converging in the end.

On Fri, Mar 10, 2017 at 3:43 PM, Casey Stella  wrote:

> I would definitely agree that moving forward we should consider something
> like Nar for Stellar. I'm not seeing the need for parsers exactly.
>
> I don't want to squash the forward thinking aspect here; we should be
broad
> and think about the end, ideal state. I just want to make sure we think
> through something that we can iterate on as an initial state that still
> solves your problem, MVP style.
>
> On Fri, Mar 10, 2017 at 3:39 PM, Otto Fowler 
> wrote:
>
> > "The Apache NiFi NAR ‘system’ allows for the packaging and loading of
> java
> > resources with classloader isolation.
> > Although technically it is the Service Provider api that makes the
> > ‘plugins’ part of the system, you can view them
> > together, and thus look at the NAR features as a system to create,
> > package, load, and execute plugins in a java system
> > while maintaining classloader isolation and dependency separation.
> >
> > While the NiFi problem case ( many plugins possibly executing in the
same
> > vm ) is not universal, the functionality provided
> > by NAR is commonly needed, and is indeed functionality that I am
> currently
> > looking at implementing in the Apache Metron project.”
> >
> > This is how I put it to Joe.
> >
> > I think what you are proposing would work. I think what I have done up
> > until now will pretty much work. What I have been thinking about and
> > considering
> > is the difference between getting ‘something that works’, and maybe
> > something better.
> >
> > So if you look at nar there is the ‘packaging’ part, and the class
> loading
> > part.
> > We are already doing almost the same thing with the assembly of the
> > .tag.gz. The Nar is a next step to this which adds more metadata and
the
> > dependency repo.
> > More of a refinement than a change.
> >
> > As far as class loading, the Nar is a more refined system for deploying
> > and consuming jars and dependencies, and setting up classloader
> instances.
> > It has more
> > functionality than we need at the moment in storm, but in other
services
> > where multiple parsers or plugin types may need to be loaded, it would
> make
> > more sense.
> > Rest may be that case. Stellar may be that case too, if anyone ever
> > writes a stellar function with different dependencies than the
platform.
> >
> >
> >
> > On March 10, 2017 at 14:32:00, Casey Stella (ceste...@gmail.com) wrote:
> >
> > So, 

Re: [DISCUSS] SIDELOADING PARSERS: Packaging and Loading and Extensions [oh.my]

[Kyle Richardson]

I like the direction of using NAR. The key benefits I see (there are
others, but I picked by favorites):

1. Move away from the uber jars in Storm. Granted, this should be possible
with modifications from the new Stellar classloader. My hope is that this
would allow us to stop shading the parser project packages.
2. Allow parser development to be somewhat independent of the core code
base. I do think we'll have to address versioning and backwards
compatibility.
3. "Drop in" extensions to Metron. Future expansion beyond parsers.

Questions:
- Is there a way to do 1 using the VFS Classloader and land the archetype
as an MVP? If so, we could avoid shading as part of the archetype and maybe
iterate from there on it. Do you think this would still be too much change
to the archetype after putting it out there?
- How would we adapt NAR from Nifi without frankensteining it? I'm all for
code reuse but, ideally, we'd like Nifi and Metron's versions of NAR to end
up converging in the end.

On Fri, Mar 10, 2017 at 3:43 PM, Casey Stella  wrote:

> I would definitely agree that moving forward we should consider something
> like Nar for Stellar.  I'm not seeing the need for parsers exactly.
>
> I don't want to squash the forward thinking aspect here; we should be broad
> and think about the end, ideal state.  I just want to make sure we think
> through something that we can iterate on as an initial state that still
> solves your problem, MVP style.
>
> On Fri, Mar 10, 2017 at 3:39 PM, Otto Fowler 
> wrote:
>
> > "The Apache NiFi NAR ‘system’ allows for the packaging and loading of
> java
> > resources with classloader isolation.
> > Although technically it is the Service Provider api that makes the
> > ‘plugins’  part of the system, you can view them
> > together, and thus look at the NAR features as a system to create,
> > package, load, and execute plugins in a java system
> > while maintaining classloader isolation and dependency separation.
> >
> > While the NiFi problem case ( many plugins possibly executing in the same
> > vm ) is not universal, the functionality provided
> > by NAR is commonly needed, and is indeed functionality that I am
> currently
> > looking at implementing in the Apache Metron project.”
> >
> > This is how I put it to Joe.
> >
> > I think what you are proposing would work.  I think what I have done up
> > until now will pretty much work.  What I have been thinking about and
> > considering
> > is the difference between getting ‘something that works’, and maybe
> > something better.
> >
> > So if you look at nar there is the ‘packaging’ part, and the class
> loading
> > part.
> > We are already doing almost the same thing with the assembly of the
> > .tag.gz.  The Nar is a next step to this which adds more metadata and the
> > dependency repo.
> > More of a refinement than a change.
> >
> > As far as class loading, the Nar is a more refined system for deploying
> > and consuming jars and dependencies, and setting up classloader
> instances.
> > It has more
> > functionality than we need at the moment in storm, but in other services
> > where multiple parsers or plugin types may need to be loaded, it would
> make
> > more sense.
> > Rest may be that case.  Stellar may be that case too, if anyone ever
> > writes a stellar function with different dependencies than the platform.
> >
> >
> >
> > On March 10, 2017 at 14:32:00, Casey Stella (ceste...@gmail.com) wrote:
> >
> > So, my question is whether we really need nar here. We have a
> classloading
> > mechanism that will allow us to deploy just the parser logic just added
> > into master for stellar, should we be considering another one?
> >
> > I would understand using nar if we needed to have multiple nars around
> > that
> > needed isolation from one another, but in the parser topology, we get
> that
> > isolation naturally. It seems to me that, at least for a MVP, we should
> > use the existing classloader that we just added. That being said, I might
> > be missing something, so let me know your thoughts.
> >
> > Casey
> >
> > On Fri, Mar 10, 2017 at 2:18 PM, Matt Foley  wrote:
> >
> > > I like the approach. I think Nar constitutes a production-quality
> > > existing solution meeting highly similar needs to Metron’s.
> > >
> > > Just a ‘btw’ regarding Joe’s input that I transmitted:
> > > - Joe made clear that he was only giving his personal opinion, since of
> > > course no individual can speak for the community.
> > > - Joe also felt that if Metron succeeded in re-using the Nar system
> > > without having to change it too much, that that would be a good
> > supporting
> > > argument for later proposing that it become a separate child project.
> > > - Whereas if we or they tried to break it out as a separate project
> now,
> > > we would have to do all the community-building work around it, as well
> > as
> > > the technical work of adapting it for a different environment from
> 

Re: [DISCUSS] SIDELOADING PARSERS: Packaging and Loading and Extensions [oh.my]

[Casey Stella]

I would definitely agree that moving forward we should consider something
like Nar for Stellar.  I'm not seeing the need for parsers exactly.

I don't want to squash the forward thinking aspect here; we should be broad
and think about the end, ideal state.  I just want to make sure we think
through something that we can iterate on as an initial state that still
solves your problem, MVP style.

On Fri, Mar 10, 2017 at 3:39 PM, Otto Fowler 
wrote:

> "The Apache NiFi NAR ‘system’ allows for the packaging and loading of java
> resources with classloader isolation.
> Although technically it is the Service Provider api that makes the
> ‘plugins’  part of the system, you can view them
> together, and thus look at the NAR features as a system to create,
> package, load, and execute plugins in a java system
> while maintaining classloader isolation and dependency separation.
>
> While the NiFi problem case ( many plugins possibly executing in the same
> vm ) is not universal, the functionality provided
> by NAR is commonly needed, and is indeed functionality that I am currently
> looking at implementing in the Apache Metron project.”
>
> This is how I put it to Joe.
>
> I think what you are proposing would work.  I think what I have done up
> until now will pretty much work.  What I have been thinking about and
> considering
> is the difference between getting ‘something that works’, and maybe
> something better.
>
> So if you look at nar there is the ‘packaging’ part, and the class loading
> part.
> We are already doing almost the same thing with the assembly of the
> .tag.gz.  The Nar is a next step to this which adds more metadata and the
> dependency repo.
> More of a refinement than a change.
>
> As far as class loading, the Nar is a more refined system for deploying
> and consuming jars and dependencies, and setting up classloader instances.
> It has more
> functionality than we need at the moment in storm, but in other services
> where multiple parsers or plugin types may need to be loaded, it would make
> more sense.
> Rest may be that case.  Stellar may be that case too, if anyone ever
> writes a stellar function with different dependencies than the platform.
>
>
>
> On March 10, 2017 at 14:32:00, Casey Stella (ceste...@gmail.com) wrote:
>
> So, my question is whether we really need nar here. We have a classloading
> mechanism that will allow us to deploy just the parser logic just added
> into master for stellar, should we be considering another one?
>
> I would understand using nar if we needed to have multiple nars around
> that
> needed isolation from one another, but in the parser topology, we get that
> isolation naturally. It seems to me that, at least for a MVP, we should
> use the existing classloader that we just added. That being said, I might
> be missing something, so let me know your thoughts.
>
> Casey
>
> On Fri, Mar 10, 2017 at 2:18 PM, Matt Foley  wrote:
>
> > I like the approach. I think Nar constitutes a production-quality
> > existing solution meeting highly similar needs to Metron’s.
> >
> > Just a ‘btw’ regarding Joe’s input that I transmitted:
> > - Joe made clear that he was only giving his personal opinion, since of
> > course no individual can speak for the community.
> > - Joe also felt that if Metron succeeded in re-using the Nar system
> > without having to change it too much, that that would be a good
> supporting
> > argument for later proposing that it become a separate child project.
> > - Whereas if we or they tried to break it out as a separate project now,
> > we would have to do all the community-building work around it, as well
> as
> > the technical work of adapting it for a different environment from NiFi.
> > - So he recommended to copy and appropriate it for now.
> > - Which I also agree with.
> >
> > Thanks,
> > --Matt
> >
> > On 3/10/17, 7:42 AM, "Otto Fowler"  wrote:
> >
> > As previously discussed here, I have been working on side loading of
> > parsers. The goals of this work are:
> > * Make it possible of developers to create, maintain and deploy parsers
> > outside of the Metron code tree and not have to fork
> > * Create maven archetype support for developers of parsers
> > * Introduce a parser ‘lifecycle’ to support multiple instances and
> > configurations, states of being installed, under configuration, and
> > deployed
> > etc.
> >
> > I would like to have some discussion based on where I am after rebasing
> > onto METRON-671 which revamps deployment to be totally ambari based.
> >
> >
> > Packaging and Loading and Extensions
> >
> > I have mentioned previously, and we have discussed on list wanting to
> > move
> > away from uber’s for somethings to using custom class loaders ( from
> > hdfs
> > possibly ).
> > We also want the REST api to work with 3rd parter parsers
> > We would like to reduce the size of having so many ubers in the build
> > We would benefit from tooling around 

Re: [DISCUSS] SIDELOADING PARSERS: Packaging and Loading and Extensions [oh.my]

[Otto Fowler]

"The Apache NiFi NAR ‘system’ allows for the packaging and loading of java
resources with classloader isolation.
Although technically it is the Service Provider api that makes the
‘plugins’  part of the system, you can view them
together, and thus look at the NAR features as a system to create, package,
load, and execute plugins in a java system
while maintaining classloader isolation and dependency separation.

While the NiFi problem case ( many plugins possibly executing in the same
vm ) is not universal, the functionality provided
by NAR is commonly needed, and is indeed functionality that I am currently
looking at implementing in the Apache Metron project.”

This is how I put it to Joe.

I think what you are proposing would work.  I think what I have done up
until now will pretty much work.  What I have been thinking about and
considering
is the difference between getting ‘something that works’, and maybe
something better.

So if you look at nar there is the ‘packaging’ part, and the class loading
part.
We are already doing almost the same thing with the assembly of the
.tag.gz.  The Nar is a next step to this which adds more metadata and the
dependency repo.
More of a refinement than a change.

As far as class loading, the Nar is a more refined system for deploying and
consuming jars and dependencies, and setting up classloader instances.  It
has more
functionality than we need at the moment in storm, but in other services
where multiple parsers or plugin types may need to be loaded, it would make
more sense.
Rest may be that case.  Stellar may be that case too, if anyone ever writes
a stellar function with different dependencies than the platform.



On March 10, 2017 at 14:32:00, Casey Stella (ceste...@gmail.com) wrote:

So, my question is whether we really need nar here. We have a classloading
mechanism that will allow us to deploy just the parser logic just added
into master for stellar, should we be considering another one?

I would understand using nar if we needed to have multiple nars around that
needed isolation from one another, but in the parser topology, we get that
isolation naturally. It seems to me that, at least for a MVP, we should
use the existing classloader that we just added. That being said, I might
be missing something, so let me know your thoughts.

Casey

On Fri, Mar 10, 2017 at 2:18 PM, Matt Foley  wrote:

> I like the approach. I think Nar constitutes a production-quality
> existing solution meeting highly similar needs to Metron’s.
>
> Just a ‘btw’ regarding Joe’s input that I transmitted:
> - Joe made clear that he was only giving his personal opinion, since of
> course no individual can speak for the community.
> - Joe also felt that if Metron succeeded in re-using the Nar system
> without having to change it too much, that that would be a good
supporting
> argument for later proposing that it become a separate child project.
> - Whereas if we or they tried to break it out as a separate project now,
> we would have to do all the community-building work around it, as well as
> the technical work of adapting it for a different environment from NiFi.
> - So he recommended to copy and appropriate it for now.
> - Which I also agree with.
>
> Thanks,
> --Matt
>
> On 3/10/17, 7:42 AM, "Otto Fowler"  wrote:
>
> As previously discussed here, I have been working on side loading of
> parsers. The goals of this work are:
> * Make it possible of developers to create, maintain and deploy parsers
> outside of the Metron code tree and not have to fork
> * Create maven archetype support for developers of parsers
> * Introduce a parser ‘lifecycle’ to support multiple instances and
> configurations, states of being installed, under configuration, and
> deployed
> etc.
>
> I would like to have some discussion based on where I am after rebasing
> onto METRON-671 which revamps deployment to be totally ambari based.
>
>
> Packaging and Loading and Extensions
>
> I have mentioned previously, and we have discussed on list wanting to
> move
> away from uber’s for somethings to using custom class loaders ( from
> hdfs
> possibly ).
> We also want the REST api to work with 3rd parter parsers
> We would like to reduce the size of having so many ubers in the build
> We would benefit from tooling around this, maven building, archetypes
> etc
> We could benefit from explicitly required metadata and information
> We want a generic extension methodology
> We want to be able to upgrade parsers/extensions in some way
>
>
> I have also mentioned that this would look or work a lot like NiFi’s
> NAR
> system.
>
> Now I’m going to put it differently:
>
> I propose that we adapt and introduce the NAR system for Metron
> Extensions,
> starting with parsers, with that adoption
> extended to allow for VFS Classloading from hdfs as we are now doing
> with
> Stellar. And that this is done as a follow on to
> the base mvp side loading work.
>
> This provides a solution to 

Re: [DISCUSS] SIDELOADING PARSERS: Packaging and Loading and Extensions [oh.my]

[Matt Foley]

I like the approach.  I think Nar constitutes a production-quality existing 
solution meeting highly similar needs to Metron’s.

Just a ‘btw’ regarding Joe’s input that I transmitted:
- Joe made clear that he was only giving his personal opinion, since of course 
no individual can speak for the community.
- Joe also felt that if Metron succeeded in re-using the Nar system without 
having to change it too much, that that would be a good supporting argument for 
later proposing that it become a separate child project.
- Whereas if we or they tried to break it out as a separate project now, we 
would have to do all the community-building work around it, as well as the 
technical work of adapting it for a different environment from NiFi.
- So he recommended to copy and appropriate it for now.
- Which I also agree with.

Thanks,
--Matt

On 3/10/17, 7:42 AM, "Otto Fowler"  wrote:

As previously discussed here, I have been working on side loading of
parsers.  The goals of this work are:
* Make it possible of developers to create, maintain and deploy parsers
outside of the Metron code tree and not have to fork
* Create maven archetype support for developers of parsers
* Introduce a parser ‘lifecycle’ to support multiple instances and
configurations, states of being installed, under configuration, and deployed
etc.

I would like to have some discussion based on where I am after rebasing
onto METRON-671 which revamps deployment to be totally ambari based.


Packaging and Loading and Extensions

I have mentioned previously, and we have discussed on list wanting to move
away from uber’s for somethings to using custom class loaders ( from hdfs
possibly ).
We also want the REST api to work with 3rd parter parsers
We would like to reduce the size of having so many ubers in the build
We would benefit from tooling around this, maven building, archetypes etc
We could benefit from explicitly required metadata and information
We want a generic extension methodology
We want to be able to upgrade parsers/extensions in some way


I have also mentioned that this would look or work a lot like NiFi’s NAR
system.

Now I’m going to put it differently:

I propose that we adapt and introduce the NAR system for Metron Extensions,
starting with parsers, with that adoption
extended to allow for VFS Classloading from hdfs as we are now doing with
Stellar. And that this is done as a follow on to
the base mvp side loading work.

This provides a solution to the above issues, and would afford us a great
amount of flexibility going forward.

https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#nars

https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions

The functional concept would be:
 * the archetype and all the parser projects produce nars ( either
including the configuration and patterns or splitting between runtime ( nar
) and static ( tar.gz )
* these are not shaded, but have a ‘repo’ of dependencies for non-metron
jars.  Metron jars are provided and loaded through the classloading.
* possibly the adaption of the Service Provider api/pattern for parsers and
discovery
*  the nar repository/working directory structures would be implemented in
/usr/metron/version/telemetry  ( although discussion on having multiple
extension directories vs. one extension dir are welcome )
* the storm process only references metron-parsers-common
* the parser bolt uses the nar class loading system to load the parser
* the rest api uses the nar class loading system to load the parser
* etc etc
* a new version of a parser is deployed as a nar, when the service
restarts, the new nar is unpacked and replaces the old version in the
working system ( we could change the restart requirement ……)

The nar system gives us something that is:
* production quality
* small enough to grok and extend as opposed to some other solutions
* comes with a highly accessible sister project
* maven plugin tooling to build
* reference archetype for packaging
and other things

There are a few ways we could approach using Nar:

1. ‘fork’ and appropriate the components and ‘metronize’ them
* the maven plugin
* the nar-utils package
2. Ask for and participate in an effort to pull NAR out into it’s own
project,
* make it more generic
* usable by more than one project
* goal to replace NiFi’s use of nar too
3. Create our own generic version as a fork
* use it in metron
* submit to NiFi as a proposal

MattF was nice enough to float this by Joe Witt, who is at Hortonworks and
is the NiFi lead.
He agrees with the idea of making nar usable for multiple projects, but
does not see them