Hi Kyle, Thank you very much for your offer. I do appreciate it. There are few requirements at this moment. We only need to capture the following information as a part of parsing:
-devicename -src_zone -dst_zone -session_info Regards, Ali On Thu, Mar 9, 2017 at 4:40 AM, Kyle Richardson <kylerichards...@gmail.com> wrote: > Agreed. One of the main drivers for the existing ASA parser is that it was > too complex to implement with a single Grok expression. > > One caveat with the existing ASA parser, right now it only loads the > patterns file from the jar. This is because adding a new message pattern > also requires an update to the static map used by the parser. While this is > not idea long term, I would be happy to work with you to either (1) > incorporate any additional message patterns you need or (2) to enhance the > parser to load patterns from other locations (e.g. HDFS). > > -Kyle > > On Wed, Mar 8, 2017 at 6:06 AM, Simon Elliston Ball < > si...@simonellistonball.com> wrote: > >> You certainly _could_ create a new parser using pure grok, however, for >> the ASA logs that would become a pretty hairy grok expression, due to the >> fact that the format of the second ‘half’ of the message is dependent on >> the device tag. >> >> The best bet is almost certainly to modify the patterns for the existing >> asa parser in this instance, which will be a lot easier. >> >> Simon >> >> On 8 Mar 2017, at 01:05, Ali Nazemian <alinazem...@gmail.com> wrote: >> >> What If I create that by adding another Sensor with a type of Grok and >> create all of the configurations manually? Like the following manual which >> has been provided for Squid Grok parser. >> >> https://cwiki.apache.org/confluence/display/METRON/Adding+a+ >> New+Telemetry+Data+Source >> >> Regards, >> Ali >> >> On Wed, Mar 8, 2017 at 11:49 AM, Simon Elliston Ball < >> si...@simonellistonball.com> wrote: >> >>> Currently you can't do his easily though the ui with the asa sensor, >>> since it is essentially a double grok parser. You will need to alter the >>> underlying grok files used by the asa parser. >>> >>> Hence you will need to backup the patterns on hdfs to preserve them >>> between upgrades as stands. >>> >>> Simon >>> >>> Sent from my iPhone >>> >>> On 8 Mar 2017, at 00:26, Ali Nazemian <alinazem...@gmail.com> wrote: >>> >>> Hi Simon, >>> >>> How can I manage that through Management-UI? I thought I would be able >>> to create another sensor inside Management-UI and use it as a Grok parser. >>> Then, add Grok statements to it. I don't want to overwrite the default >>> Metron Parsers. They will be disappeared once we update our platform. I >>> might need to have different ASA devices with different types of parsing >>> based on the client. >>> >>> Regards, >>> Ali >>> >>> On Wed, Mar 8, 2017 at 11:18 AM, Simon Elliston Ball < >>> si...@simonellistonball.com> wrote: >>> >>>> Hi Ali, >>>> >>>> For the grok parser there are essentially two passes at the grok >>>> pattern. The first determines the header and the device type, which then >>>> controls the pattern used for the second pass, which is device specific. >>>> Both patterns depend on the standard grok patterns in /patterns/asa on >>>> either the hdfs patterns path or the resources within compiled jars. >>>> >>>> If you wish to extend these you could just edit the patterns on the >>>> hdfs location (if for example you have additional fields you need to >>>> extract beyond the device defaults). This could be brittle to parser >>>> upgrades so it's worth tracking any future changes to be underlying base >>>> patterns. >>>> >>>> Simon >>>> >>>> On 8 Mar 2017, at 00:13, Ali Nazemian <alinazem...@gmail.com> wrote: >>>> >>>> Hi Kyle, >>>> >>>> Thank you very much. I should have asked the question earlier. We have >>>> done the most of the Grok statement implementations so far! I haven't >>>> checked the source code for any Grok sample parser. >>>> >>>> For the Grok deployment, are you saying that we need to put all of the >>>> grok statements inside the Metron Management-UI and we only need to create >>>> a Kafka topic for that? >>>> >>>> Regards, >>>> Ali >>>> >>>> On Tue, Mar 7, 2017 at 11:26 PM, Kyle Richardson < >>>> kylerichards...@gmail.com> wrote: >>>> >>>>> Hi Ali, >>>>> >>>>> There is a grok-based ASA parser included in the Metron code base that >>>>> you can try out. If you find it's missing patterns or requires >>>>> modifications, I'd be happy to work with you to improve on it. >>>>> >>>>> You should be able to test it out by creating a new Kafka topic 'asa' >>>>> and pointing your raw logs there. Let me know if you run into any issues. >>>>> >>>>> Thanks, >>>>> Kyle >>>>> >>>>> On Mon, Mar 6, 2017 at 9:51 PM, Ali Nazemian <alinazem...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> I am building a customized version of ASA parser using Grok >>>>>> statements. I have prepared the Grok requirements so far. I am using the >>>>>> following manual which has been provided for Grok squid parser >>>>>> <https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source>. >>>>>> I couldn't find anything else as an end-to-end manual for deploying a >>>>>> Grok >>>>>> parser, and I have some trouble to map this manual with the Hortonworks >>>>>> Cyber Security release. For example, I couldn't find the step-5 >>>>>> alternative >>>>>> in Hortonworks one. I would be grateful if somebody can provide a link >>>>>> for >>>>>> better and more up-to-date manual for deploying a Grok Parser in Meron >>>>>> 0.3. >>>>>> >>>>>> Regards, >>>>>> Ali >>>>>> >>>>> >>>>> >>>> >>> >>> >>> -- >>> A.Nazemian >>> >>> >> >> >> -- >> A.Nazemian >> >> >> >
