I went ahead and created a Jira ticket mirroring Casey's discussion of the
MVP.  Feel free to add anything of interest there, too.

https://issues.apache.org/jira/browse/METRON-427


Justin


On Fri, Sep 16, 2016 at 9:39 AM, Justin Leet <justinjl...@gmail.com> wrote:

>
> ---------- Forwarded message ----------
> From: zeo...@gmail.com <zeo...@gmail.com>
> Date: Thu, Sep 15, 2016 at 9:02 PM
> Subject: Re: [DISCUSS] Ambari Integration
> To: u...@metron.incubator.apache.org
> Cc: dev@metron.incubator.apache.org
>
>
> Of course I would still need a full list of the repos, and submit proxy
> rules for the Ambari box, but happy to hear it will alleviate the need for
> making the scripts use proxies on the cluster nodes.
>
> Jon
>
> On Thu, Sep 15, 2016, 19:34 Nick Allen <n...@nickallen.org> wrote:
>
> > Jon - Installing Metron on an isolated network becomes much easier with
> > Ambari.  You would just mirror the required RPM repositories.  You can
> then
> > point Ambari to where your repo lives via the installation wizard.  I've
> > done quite a few installs via Ambari on an isolated network and it worked
> > quite well.
> >
> >
> >
> >
> >
> >
> > On Thu, Sep 15, 2016 at 6:50 PM, zeo...@gmail.com <zeo...@gmail.com>
> > wrote:
> >
> >> First of all - very much looking forward to this approach.  I'm not very
> >> familiar with management packs, but I did read some of the
> documentation in
> >> the link you sent.
> >>
> >> Not sure if this is already included in a "minimum viable product," but
> >> at some point I think there needs to be a method of specifying proxies
> >> and/or internal package repos.  I recently did a Metron 0.2.0 install
> >> behind a proxy (hence METRON-409
> >> <https://issues.apache.org/jira/browse/METRON-409>) and it look me a
> >> semi-lengthy amount of time to (1) find all of the destinations I
> needed to
> >> request openings for in the proxy, and (2) modify the ambari scripts to
> >> appropriately use my proxies in the correct way.
> >>
> >> I also have a bit of a concern with upgrades and customizations in
> >> general (Not just how it would work with mpacks).  I have not done any
> of
> >> this to date, but I have rebuilt and redeployed a couple of times and I
> >> needed to modify some of the metron code itself before build/deploy
> >> (because of my concern with it getting overwritten on upgrade if I just
> did
> >> it directly on the cluster).  I would like to see a method of putting in
> >> install-specific files that modify or overwrite parts of the core metron
> >> stack, like changes to znodes, parsers, etc.
> >>
> >> Regarding not managing sensors with Ambari, I agree.  I run a large bro
> >> cluster and it is maintained via Puppet and various other mechanisms -
> no
> >> need for Ambari to bleed over in my case.
> >>
> >> Thanks for the great work.
> >>
> >> Jon
> >>
> >> On Thu, Sep 15, 2016 at 5:10 PM Casey Stella <ceste...@gmail.com>
> wrote:
> >>
> >>> Hi Everyone,
> >>>
> >>> I wanted to solicit some discussion around a feature that is fast
> >>> approaching.  A major pain point in using Metron is installation.
> Thus far
> >>> our only approach to installation has been driven by the developer's
> needs
> >>> to construct a virtual environment to test out changes, which lead us
> to
> >>> either an ansible installation or a manual installation.
> >>>
> >>> Because we want to make sure that the installation of Metron is as easy
> >>> as possible, we have had some great contributions of an additional
> >>> approach, installation via Apache Ambari directly.  Our ansible scripts
> >>> currently rely on Ambari blueprints to set up Hadoop on the cluster
> that it
> >>> is deploying on, so it is not a new dependency, but we're working
> toward a
> >>> full Ambari management pack
> >>> <https://cwiki.apache.org/confluence/display/AMBARI/Management+Packs>
> >>> that will lay down the relevant topologies (parser, enrichment,
> indexing),
> >>> configs, bits and their infrastructural dependencies (ES and mysql) and
> >>> allow the topologies to be started and stopped as minimum viable
> product.
> >>>
> >>> The beginnings of this have started with:
> >>>
> >>>    - Ambari Service Definitions for the Parser topologies
> >>>    <https://github.com/apache/incubator-metron/pull/218>
> >>>    - Ambari Service Definition for the Indexing Topology
> >>>    <https://github.com/apache/incubator-metron/pull/222>
> >>>    - Ambari Service Definition for Elasticsearch
> >>>    <https://github.com/apache/incubator-metron/pull/223>
> >>>
> >>> There will be more to come in the near-term to realize that vision, but
> >>> we wanted to get some reactions.  Past minimum viable product, what do
> you
> >>> guys think we should have and how should it look?
> >>>
> >>> Currently we are treating the domain of the ambari installation as from
> >>> kafka to the indexes, which leaves the sensors unmanaged via ambari.
> Is
> >>> that a good decision?
> >>>
> >>> Are there other pain points that you have had around installation that
> >>> you'd like to see addressed?
> >>>
> >>> The purpose of this discussion thread is to let you guys know that we
> >>> will soon have a new way to install metron, but also to understand
> what the
> >>> future requirements are so we, as a community, can address them.
> >>>
> >>> Best,
> >>>
> >>> Casey
> >>>
> >> --
> >>
> >> Jon
> >>
> >
> >
> >
> > --
> > Nick Allen <n...@nickallen.org>
> >
> --
>
> Jon
>
>

Reply via email to