I am also going to create the ES Template and possibly modify or create a
new dashboard. Full peer.
On September 27, 2016 at 12:17:23, Otto Fowler (ottobackwa...@gmail.com)
wrote:
My wish, is that when I do an ansible-playbook -v -i {my configuration}
metron_full_install.yml to my cluster - or do the full_dev-> vagrant that
my parser / topology is deployed, started and monitored the same way as the
current bro, snort, and yaf parsers are.
I might be misunderstanding something however. I seems to me that all the
examples of adding other parsers are temporary and not permanent because
they do not have the full deployment, kind of push the config and run the
script and you are going. Am I missing something? Would the squid sample
steps result in a parser topology that would survive restarts / reboots etc?
On September 27, 2016 at 12:06:44, James Sirota (jsir...@apache.org) wrote:
Just so I completely understand what you are asking for...you want to know
how to create a new parser topology with the JSON parser and plug it into
Monit so you can monitor and restart it on demand?
27.09.2016, 09:03, "Otto Fowler" <ottobackwa...@gmail.com>:
> Thanks James,
>
> I want to deploy an instance of the JSONMapParser into my POC cluster and
vagrant. I’m trying to work out exactly how to add a new configured parser
instance to the deployment. I think these instructions would be a good
extension to the squid stuff that is already there. If I could get that
going and add a new parser all the way through, then maybe I can contribute
something in that area. The ability to do this will also enable some of
the other work you mentioned.
>
> On September 27, 2016 at 11:51:41, James Sirota (jsir...@apache.org)
wrote:
>
>> There are three types of parsers you can have currently. Our preferred
way is to use Grok parser. The only thing you need to do there is to define
your Grok statement and the parser will uptake it and do the rest. That is
what most of our documentation reflect. The second type of parser that we
have is a java parser, where you actually extend a parser class and define
your own custom parsing logic. We intend this type of parser for high
velocity feeds that require custom parsing logic that is not easily
attainable by Grok. The third type of parser is the one you have been
working on, a Json parser. This is a parser designed to take pre-parsed
JSON for sensors that either log in JSON format natively or have been
pre-parsed for us by some system upstream.
>>
>> Parsers don't integrate with Monit by default. We can come up with some
instructions for you on how to do that.
>>
>> I should also note there are 2 additional parser types that are on the
road map. METRON-295 (scripting bolt), which is a parser that allows you to
uptake something like javascript, lua, etc., for doing the parsing. There
is also METRON-288, which is a XSL parser designed to parse XML documents.
If either of these are of interest to you we would welcome this
contribution and we can work with you to get you started.
>>
>> 26.09.2016, 10:35, "Otto Fowler" <ottobackwa...@gmail.com>:
>>> Are all the steps required to add a parser documented anywhere? The
squid
>>> document starts the topology, but I don’t think that integrates it in
with
>>> monit for example. Or does that actually happen?
>>
>> -------------------
>> Thank you,
>>
>> James Sirota
>> PPMC- Apache Metron (Incubating)
>> jsirota AT apache DOT org
-------------------
Thank you,
James Sirota
PPMC- Apache Metron (Incubating)
jsirota AT apache DOT org
