On Fri, Jul 16, 2010 at 6:43 AM, <[email protected]> wrote:
> On Thu, 15 Jul 2010, Joel Merrick wrote:
>
>> Hi list,
>>
>> I'm trying to build a service to enable the quick searching of mail
>> logs, for our support team to use. We get quite a lot of log
>> generation (about 2G of mysql data a day)
>>
>> Searching these becomes really inefficient after a while, even though
>> there's extra keys and indexes in the db.
>>
>> I'd like to try and parse the syslog event using rsyslog and get the
>> message ID out of the payload and add it as an indexed field, which
>> should speed up queries (so we can stitch together a full email
>> transaction)
>
> this shouldn't be _too_ hard, depending on where the message ID is in the
> messages you are logging
>
> just create your own template that writes the message ID as a separate
> field.
>
The position of the message ID's are always pretty consistent but not exactly.
I've got a PoC ruby daemon listening on a named pipe and regex'ing out
the message ID's already.
Shawn's very kindly told me about Solr, so I'm going to give that a whirl today.
If I could get away from using the ruby daemon and use rsyslog
properly, that'd be good.. however it's working and can easily handle
the load
The regexp in ruby I'm using is;
\w{6}-\w{6}-\w{2}
Could this be done for the templates?
--
$ echo "kpfmAdpoofdufevq/dp/vl" | perl -pe 's/(.)/chr(ord($1)-1)/ge'
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
