On Thu, 2013-01-31 at 15:26 +0000, Ben Bradley wrote: > On Thu, 31 Jan 2013 13:44:03 +0000 > Rainer Gerhards <[email protected]> wrote: > > > > I guess it all comes down to performance testing, but 10GB would probably > > > mean ~20M logs or something like that. If the majority of those will be > > > sent during the day (say 10 hours), my poor math says if you handle > > > 500-600 > > > logs/sec you should be fine. > > > > seeing that number, I'd say it requires quite some regexpes to get > > rsyslog to sweat. HOWEVER... do we really need regexpes? Can you post a > > couple of samples? > > > > Rainer > > Great news. I'll be testing this over the next few days/weeks. > > Here's a sample log line as it comes in to rsyslog from Apache logging to > /bin/logger... > http://pastebin.com/649fbqQ7 > > <134>Jan 30 14:09:30 LWEB03 apache-access[www.apachevhostname.com]: > 84.184.148.184 - - [30/Jan/2013:14:09:30 +0000] "GET > /fileadmin/images/bg-footerBar.gif HTTP/1.1" 404 244 > "http://www.website.com/latest-news/article/newsarticle/article-name-in-here/"; > "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/5.0; SLCC2; .NET > CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; > .NET4.0C; InfoPath.1)" www.apachevhostname.com 16992 > > It's an Apache combined log line with vhost and request time in microseconds > added to the end. > At the moment I'm building a regular expression to capture each of those > fields from the log line. > That sounds a bit like we should be able to grab this even with the current version and mmnormalize - maybe with a bit larger rulebase than actually would be needed. I'll see if we can give it a try and report back.
How would you ideally like to see this after conversion? Rainer > Cheers, Ben > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
