Sent from phone, thus brief. Am 09.10.2013 21:39 schrieb "Robert" <[email protected]>: > > If I write out to /dev/null the only way for me to see what is being processed will be thorugh the pstats? >
Yup but imho that's all you need to have. It's much more precise than counting lines. .. > >> >> ----- Original Message ----- >> >> From: David Lang >> >> Sent: 10/09/13 02:49 PM >> >> To: Robert >> >> Subject: Re: [rsyslog] Tr : Re: perfomance tweaking (fwd) >> >> >> >> thanks for including the config, it's hard to keep track of everything. >> >> the 'unexpected end of file' error is because rsyslog still has the file open at >> the time you are doing the zgrep, if you rotate the files (mv the file then hit >> rsyslog with kill -HUP) rsyslog will write the remainder of the file and you >> will not get that error. >> >> The fact that all the per second numbers are eactly 200000 or 300000 is very odd >> >> what did top look like during the test? >> >> and just to confirm, this is rsyslog 7.5 right? >> >> if you remove ziplevel, it will disable gzip entirely. I expect that when you do >> that you will find iostat going to 100%, but you can give it a try. >> >> could you do a zgrep for a time frame when the queue was full (this will give us >> an idea of the throughput), say around 14:12 or so >> >> David Lang >> >> On Wed, 9 Oct 2013, Robert wrote: >> >> > Date: Wed, 09 Oct 2013 14:40:13 -0400 >> > From: Robert <[email protected]> >> > To: David Lang <[email protected]> >> > Cc: Rainer Gerhards <[email protected]>, rmkml <[email protected]> >> > Subject: Re: [rsyslog] Tr : Re: perfomance tweaking (fwd) >> > >> > Hello Everyone, >> > >> > I have been away for business, but I'm back in action, attached you will find the stats for the rsyslog server, [we are currently throwing 395k/eps to the server ~ 985Mb] and below you will find the current config (just to refreshen the memory) >> > >> > >> > #### MODULES #### >> > >> > module(load="imuxsock") # needs to be done just once Roberto 8-9-13 >> > module(load="imklog") >> > module(load="impstats" interval="30" log.syslog="off" >> > log.file="/var/log/pstats") >> > >> > # Provides UDP syslog reception >> > module(load="imudp" >> > batchSize="1000" >> > TimeRequery="100" >> > SchedulingPolicy="fifo" >> > SchedulingPriority="10") >> > input(type="imudp" port="514" ) >> > >> > #### GLOBAL DIRECTIVES #### >> > >> > # Use imprecise-old-style timestamp format >> > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat >> > >> > $MainMsgQueueType FixedArray >> > $MainMsgQueueSize 20000000 >> > $MainMsgQueueWorkerThreads 2 >> > $MainMsgQueueDequeueBatchSize 1000 >> > $InputUDPMaxSessions 40000000 >> > >> > #### RULES #### >> > >> > >> > >> > action (type="omfile" >> > File="/var/log/test/f_all" >> > ziplevel="1" >> > ASyncWriting="on") >> > >> > >> > Lastly these are the result from the zgrep command: >> > >> > >> > >> > [rcortiz@simon test]$ sudo zgrep 14:07:53 f_all | wc -l >> > >> > gzip: f_all: unexpected end of file >> > 200001 >> > [rcortiz@simon test]$ sudo zgrep 14:07:54 f_all | wc -l >> > >> > gzip: f_all: unexpected end of file >> > 200000 >> > [rcortiz@simon test]$ sudo zgrep 14:07:55 f_all | wc -l >> > >> > gzip: f_all: unexpected end of file >> > 200000 >> > [rcortiz@simon test]$ sudo zgrep 14:08:00 f_all | wc -l >> > >> > gzip: f_all: unexpected end of file >> > 300000 >> > [rcortiz@simon test]$ sudo zgrep 14:08:10 f_all | wc -l >> > >> > gzip: f_all: unexpected end of file >> > 200000 >> > [rcortiz@simon test]$ sudo zgrep 14:08:20 f_all | wc -l >> > >> > gzip: f_all: unexpected end of file >> > 200000 >> > [rcortiz@simon test]$ sudo zgrep 14:08:30 f_all | wc -l >> > >> > gzip: f_all: unexpected end of file >> > 200000 >> > [rcortiz@simon test]$ sudo zgrep 14:08:40 f_all | wc -l >> > >> > gzip: f_all: unexpected end of file >> > 200001 >> > [rcortiz@simon test]$ sudo zgrep 14:08:50 f_all | wc -l >> > >> > gzip: f_all: unexpected end of file >> > 200000 >> > [rcortiz@simon test]$ >> > >> > >> > I am curious if we need to remove the zipping part of the action also maybe adding the recvbuffer size? >> > >> > Robert. >> > > > > > > > > > > Robert. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
