I know it is a little bit old threat, but I did have a similar issue today
(graylog 0.20.1). I tried to make a rule
which machtes the String "connect from localhost[127.0.0.1]". My simplest
approach was
m : Message(message matches ".*localhost.*")
but it didn't work. What did work is
m: Message(message matches ".*localhost.*(.|\n|\r)*" ) // I think it should
be m: Message(message matches ".*localhost.*(.|\\n|\\r)*" ) <= didn't try
I think it must be a multiline string so I ended up with this
m : Message(getField("facility") == "mail" && message matches
"(?s).*connect.*from.*localhost.*")
this works too.
My message is created by postfix and send via rsyslog to graylog. I have no
glue why it is a multiline string or some thins similar.
Maybe there can be an option in graylog to debug rules a little bit more
easaly.
Thanks Martin for the hint, without this I would not have found the issue.
regards
dieter
Am Dienstag, 24. September 2013 21:00:51 UTC+2 schrieb Martin René
Mortensen:
>
> I finally figured it out - somehow some appliances put something extra in
> the end of the message, so Drools matches doesnt work properly. If you
> enclose your pattern with .* and (.|\n|\r)* it will succeed!.
>
> Example:
>
> m : LogMessage ( fullMessage matches ".*%ASA(.|\n|\r)*" )
>
> Enjoy!
>
> /Martin
>
> On Saturday, 21 September 2013 19:30:27 UTC+2, Martin René Mortensen wrote:
>>
>> I'm trying to write some Drools rules to fix formatting of incoming logs
>> (especially from ASA firewalls), but I can't seems to be able to match a
>> LogMessage on its shortMessage, longMessage. I can match the facility
>> without problems.
>> I have searched for drools syntaxes and all the examples I can find on
>> the internet (starting with
>> https://github.com/Graylog2/graylog2-server/wiki/Message-processing-rewriting)
>>
>> - but nothing seems to help. Maybe the field names themsevles are
>> incorrect? how do I tell ?
>>
>> My squid rule works fine.
>>
>> I use graylog2-server 0.12 with elasticsearch 0.20.4 - debug output says
>> nothing incorrect, message just gets put normally without any form of
>> rewrite - I tried putting println statements into the then group, but it
>> doesnt get triggered at all.
>> Any tips ? If you cant match on short or fullMessage, the point of the
>> drools rules are somewhat moot.
>>
>> /etc/graylog2.d/rules/graylog2.drl:
>>
>> import org.graylog2.plugin.logmessage.LogMessage
>> import java.util.regex.Matcher
>> import java.util.regex.Pattern
>> import java.net.InetAddress
>> import java.text.DateFormat
>> import java.text.SimpleDateFormat
>> import java.text.ParseException
>>
>> /*
>> Raw Syslog: squid[2099]: 1339551529.881 55647 1.2.3.4 TCP_MISS/200 22
>> GET http://www.google.com/
>>
>> squid\[\d+\]: (\d+\.\d+) *(\d+) *(\d+.\d+.\d+.\d+) *(\w+\/\w+) (\d+)
>> (\w+) (.*)
>> matched: 13:1339551529.881
>> matched: 29:55647
>> matched: 35:1.2.3.4
>> matched: 47:TCP_MISS/200
>> matched: 60:22
>> matched: 64:GET
>> matched: 68:http://www.google.com/
>> */
>>
>> rule "Squid Logging to GELF"
>> when
>> m : LogMessage( facility == "local5" )
>> then
>> Matcher matcher = Pattern.compile("squid\\[\\d+\\]: (\\d+.\\d+)
>> *(\\d+) *(\\d+.\\d+.\\d+.\\d+) *(\\w+\\/\\w+) (\\d+) (\\w+)
>> (.*)").matcher(m.getShortMessage());
>>
>> if (matcher.find()) {
>> m.setFacility("squid");
>> InetAddress addr = InetAddress.getByName(matcher.group(3));
>> String host = addr.getHostName();
>> m.setHost(host);
>> m.setShortMessage(matcher.group(6) + " " + matcher.group(7));
>> m.addAdditionalData("_status",matcher.group(4));
>> m.addAdditionalData("_size",matcher.group(5));
>> }
>>
>> end
>>
>> rule "ASA logs rewrite"
>> when
>> m : LogMessage ( shortMessage matches "%ASA" )
>> then
>> Matcher matcher = Pattern.compile("^(?<date>\\w+ \\d+
>> \\d+ \\d+:\\d+:\\d+) (?<host>\\w+) :
>> (?<mesgid>%ASA-\\d-\\d+):(?<mesg>.+)").matcher(m.getShortMessage());
>> if (matcher.find()) {
>> DateFormat dfm = new SimpleDateFormat("MMM dd
>> yyyy HH:mm:ss");
>> long unixtime=0;
>> try
>> {
>> unixtime =
>> dfm.parse(matcher.group(1)).getTime();
>> unixtime = unixtime/1000;
>> m.setCreatedAt(unixtime);
>> }
>> catch (ParseException e)
>> {
>> e.printStackTrace();
>> }
>> m.setHost(matcher.group('host'));
>>
>> m.addAdditionalData("_id",matcher.group('mesgid'));
>> m.setShortMessage(matcher.group('mesg'));
>> }
>> end
>>
>> rule "ASA access-list permit fields"
>> when
>> m : LogMessage ( fullMessage matches "%ASA-4-106100" )
>> then
>> Matcher matcher = Pattern.compile("%ASA-4-106100:
>> access-list (?<rule>[\\w\\d.-]+) (?<permdeny>\\w+) (?<proto>\\w+)
>> (?<srcif>\\w+)/(?<src>[0-9a-fA-F.:]+)\\((?<srcport>\\d+)\\) ->
>> (?<dstif>\\w+)/(?<dst>[0-9a-fA-F.:]+)\\((?<dstport>\\d+)\\)").matcher(m.getFullMessage());
>> if (matcher.find()) {
>>
>> m.addAdditionalData("_rule",matcher.group('rule'));
>>
>> m.addAdditionalData("_permitdeny",matcher.group('permdeny'));
>>
>> m.addAdditionalData("_proto",matcher.group('proto'));
>>
>> m.addAdditionalData("_src_if",matcher.group('srcif'));
>> m.addAdditionalData("_src",matcher.group('src'));
>>
>> m.addAdditionalData("_src_port",matcher.group('srcport'));
>>
>> m.addAdditionalData("_dst_if",matcher.group('dstif'));
>> m.addAdditionalData("_dst",matcher.group('dst'));
>>
>> m.addAdditionalData("_dst_port",matcher.group('dstport'));
>> }
>> end
>>
>> rule "ASA access-list deny fields"
>> when
>> m : LogMessage ( fullMessage matches "%ASA-4-106023" )
>> then
>> Matcher matcher = Pattern.compile("%ASA-4-106023:
>> (?<permdeny>\\w+) (?<proto>\\w+) src
>> (?<srcif>\\w+):(?<src>[0-9a-fA-F.:]+)/(?<srcport>\\d+) dst
>> (?<dstif>\\w+):(?<dst>[0-9a-fA-F.:]+)/(?<dstport>\\d+) by access-group
>> \"(?<rule>.+)\" ").matcher(m.getFullMessage());
>> if (matcher.find()) {
>>
>> m.addAdditionalData("_rule",matcher.group('rule'));
>>
>> m.addAdditionalData("_permitdeny",matcher.group('permdeny'));
>>
>> m.addAdditionalData("_proto",matcher.group('proto'));
>>
>> m.addAdditionalData("_src_if",matcher.group('srcif'));
>> m.addAdditionalData("_src",matcher.group('src'));
>>
>> m.addAdditionalData("_src_port",matcher.group('srcport'));
>>
>> m.addAdditionalData("_dst_if",matcher.group('dstif'));
>> m.addAdditionalData("_dst",matcher.group('dst'));
>>
>> m.addAdditionalData("_dst_port",matcher.group('dstport'));
>> }
>> end
>>
>
--
You received this message because you are subscribed to the Google Groups
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
