You certainly _could_ create a new parser using pure grok, however, for the ASA logs that would become a pretty hairy grok expression, due to the fact that the format of the second ‘half’ of the message is dependent on the device tag.
The best bet is almost certainly to modify the patterns for the existing asa parser in this instance, which will be a lot easier. Simon > On 8 Mar 2017, at 01:05, Ali Nazemian <alinazem...@gmail.com> wrote: > > What If I create that by adding another Sensor with a type of Grok and create > all of the configurations manually? Like the following manual which has been > provided for Squid Grok parser. > > https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source > > <https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source> > > Regards, > Ali > > On Wed, Mar 8, 2017 at 11:49 AM, Simon Elliston Ball > <si...@simonellistonball.com <mailto:si...@simonellistonball.com>> wrote: > Currently you can't do his easily though the ui with the asa sensor, since it > is essentially a double grok parser. You will need to alter the underlying > grok files used by the asa parser. > > Hence you will need to backup the patterns on hdfs to preserve them between > upgrades as stands. > > Simon > > Sent from my iPhone > > On 8 Mar 2017, at 00:26, Ali Nazemian <alinazem...@gmail.com > <mailto:alinazem...@gmail.com>> wrote: > >> Hi Simon, >> >> How can I manage that through Management-UI? I thought I would be able to >> create another sensor inside Management-UI and use it as a Grok parser. >> Then, add Grok statements to it. I don't want to overwrite the default >> Metron Parsers. They will be disappeared once we update our platform. I >> might need to have different ASA devices with different types of parsing >> based on the client. >> >> Regards, >> Ali >> >> On Wed, Mar 8, 2017 at 11:18 AM, Simon Elliston Ball >> <si...@simonellistonball.com <mailto:si...@simonellistonball.com>> wrote: >> Hi Ali, >> >> For the grok parser there are essentially two passes at the grok pattern. >> The first determines the header and the device type, which then controls the >> pattern used for the second pass, which is device specific. Both patterns >> depend on the standard grok patterns in /patterns/asa on either the hdfs >> patterns path or the resources within compiled jars. >> >> If you wish to extend these you could just edit the patterns on the hdfs >> location (if for example you have additional fields you need to extract >> beyond the device defaults). This could be brittle to parser upgrades so >> it's worth tracking any future changes to be underlying base patterns. >> >> Simon >> >> On 8 Mar 2017, at 00:13, Ali Nazemian <alinazem...@gmail.com >> <mailto:alinazem...@gmail.com>> wrote: >> >>> Hi Kyle, >>> >>> Thank you very much. I should have asked the question earlier. We have done >>> the most of the Grok statement implementations so far! I haven't checked >>> the source code for any Grok sample parser. >>> >>> For the Grok deployment, are you saying that we need to put all of the grok >>> statements inside the Metron Management-UI and we only need to create a >>> Kafka topic for that? >>> >>> Regards, >>> Ali >>> >>> On Tue, Mar 7, 2017 at 11:26 PM, Kyle Richardson <kylerichards...@gmail.com >>> <mailto:kylerichards...@gmail.com>> wrote: >>> Hi Ali, >>> >>> There is a grok-based ASA parser included in the Metron code base that you >>> can try out. If you find it's missing patterns or requires modifications, >>> I'd be happy to work with you to improve on it. >>> >>> You should be able to test it out by creating a new Kafka topic 'asa' and >>> pointing your raw logs there. Let me know if you run into any issues. >>> >>> Thanks, >>> Kyle >>> >>> On Mon, Mar 6, 2017 at 9:51 PM, Ali Nazemian <alinazem...@gmail.com >>> <mailto:alinazem...@gmail.com>> wrote: >>> Hi all, >>> >>> I am building a customized version of ASA parser using Grok statements. I >>> have prepared the Grok requirements so far. I am using the following manual >>> which has been provided for Grok squid parser >>> <https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source>. >>> I couldn't find anything else as an end-to-end manual for deploying a Grok >>> parser, and I have some trouble to map this manual with the Hortonworks >>> Cyber Security release. For example, I couldn't find the step-5 alternative >>> in Hortonworks one. I would be grateful if somebody can provide a link for >>> better and more up-to-date manual for deploying a Grok Parser in Meron 0.3. >>> >>> Regards, >>> Ali >>> >>> >> >> >> >> -- >> A.Nazemian > > > > -- > A.Nazemian
