My bad, the pattern surpasses names of capture groups.
AUTHLOG1 %{NUMBER:timestamp} %{SYSLOGHOST:syslog_host}
%{DATA:syslog_program}(?:\[%{POSINT}\])?: %{WORD:login} password for
%{USERNAME:username} from %{IP:ip} %{GREEDYDATA}
AUTHLOG2 %{NUMBER:timestamp} %{SYSLOGHOST:syslog_host}
%{DATA:syslog_program}(?:\[%{POSINT}\])?: %{WORD:login} closed for user
%{USERNAME:username}
AUTHLOG (%{AUTHLOG1}|%{AUTHLOG2})
should work… though to be honest, your patterns look a little unusual. You seem
to have logs with a timestamp in epoch at the front, which is a very weird way
to setup syslog, so the issue might be that your patterns flat out don’t match
the logs.
Simon
> On 23 Oct 2017, at 10:36, tkg_cangkul <yuza.ras...@gmail.com> wrote:
>
> Hi Simon,
>
> I've tried your suggestion but i have an error msg like below :
>
> <Screenshot from 2017-10-23 16:34:45.png>
>
> On 23/10/17 16:22, Simon Elliston Ball wrote:
>> That is not valid grok. Pattern names should be unique in the grok.
>>
>> What you probably mean is something like:
>>
>> AUTHLOG1 %{NUMBER:timestamp} %{SYSLOGHOST:syslog_host}
>> %{DATA:syslog_program}(?:\[%{POSINT}\])?: %{WORD:login} password for
>> %{USERNAME:username} from %{IP:ip} %{GREEDYDATA}
>> AUTHLOG2 %{NUMBER:timestamp} %{SYSLOGHOST:syslog_host}
>> %{DATA:syslog_program}(?:\[%{POSINT}\])?: %{WORD:login} closed for user
>> %{USERNAME:username}
>> AUTHLOG (?:%{AUTHLOG1}|%{AUTHLOG2})
>>
>> Simon
>>
>>
>>> On 23 Oct 2017, at 08:53, tkg_cangkul <yuza.ras...@gmail.com>
>>> wrote:
>>>
>>> FYI,
>>>
>>> i've trying to using Grok parser metron with multiple pattern in single
>>> file but it doesn't work. this is my sample grok pattern on
>>> /apps/metron/patterns/authlog :
>>>
>>> AUTHLOG %{NUMBER:timestamp} %{SYSLOGHOST:syslog_host}
>>> %{DATA:syslog_program}(?:\[%{POSINT}\])?: %{WORD:login} password for
>>> %{USERNAME:username} from %{IP:ip} %{GREEDYDATA}
>>> AUTHLOG %{NUMBER:timestamp} %{SYSLOGHOST:syslog_host}
>>> %{DATA:syslog_program}(?:\[%{POSINT}\])?: %{WORD:login} closed for user
>>> %{USERNAME:username}
>>>
>>> When the sensor started, the second grok pattern doesn't work. Only first
>>> pattern works.
>>> There is an error message like this on storm logs:
>>>
>>> Caused by: java.lang.RuntimeException: Grok statement produced a null
>>> message.
>>>
>>>
>>> On 23/10/17 10:49, tkg_cangkul wrote:
>>>
>>>> Hi Wasim,
>>>>
>>>> thx for your reply.
>>>> So it means i should use logstash parser for metron?
>>>> Is there any documentation about use logstash parser for metron?
>>>> I didn't found any documentation about that on metron.
>>>> i just find logstash basic parser but there is no documentation about that.
>>>>
>>>>
>>>>
>>>> On 23/10/17 10:33, Wasim Halani wrote:
>>>>
>>>>> Hi Youzha,
>>>>>
>>>>> It should be possible to add multiple patterns in a single config file.
>>>>> For reference, you can check out the use of multiple patterns in a repo I
>>>>> maintain [1].
>>>>> You would find the patterns in [2] useful for your use-case.
>>>>>
>>>>> However, do note that there is a cost to every grok failure [3] - so you
>>>>> need to ensure that your most common event patterns are at the top of the
>>>>> list.
>>>>>
>>>>> As a side-note, if you have any logstash parsers which are not available
>>>>> in the repo, please feel to submit a PR to [4]
>>>>>
>>>>>
>>>>> [1]
>>>>> https://bitbucket.org/networkintelligence/logstash-configs/raw/aae3d61bb6c53beb0678536e2e9b33d7996e2960/cisco-asa.conf
>>>>>
>>>>> [2]
>>>>> https://bitbucket.org/networkintelligence/logstash-configs/raw/aae3d61bb6c53beb0678536e2e9b33d7996e2960/linux-system.conf
>>>>>
>>>>> [3]
>>>>> https://www.elastic.co/blog/do-you-grok-grok
>>>>>
>>>>> [4]
>>>>> https://bitbucket.org/networkintelligence/logstash-configs/
>>>>>
>>>>>
>>>>> Regards,
>>>>> ---
>>>>> Wasim Halani
>>>>>
>>>>> http://twitter.com/washalsec
>>>>> http://securitythoughts.wordpress.com
>>>>>
>>>>> ----------
>>>>> To keep silent when you can say something wise and useful is as bad as
>>>>> keeping on propagating foolish and unwise thoughts. -- Imam Ali (p.b.u.h.)
>>>>>
>>>>> On Mon, Oct 23, 2017 at 8:08 AM, Youzha
>>>>> <yuza.ras...@gmail.com>
>>>>> wrote:
>>>>> Hi, is that possible to using multiple pattern grok parser ini 1 pattern
>>>>> file?
>>>>> i’m trying to parsing authlog file in /var/log/secure into metron. the
>>>>> problem is there are different structures of logs inside /var/log/secure.
>>>>> any suggest for this pls?
>>>>>
>>>>>
>>>>> Best Regards,
>>>>>
>>>>>
>>>>>
>
