Thanks so Much Andre We have implemented the suggests parameters. Now we have about 20 MB per minute: is it a good performance for rsyslog agent or is it possibile improve the speed? Our csv files are around 1 GB and aren't a few
*Fabio Danìa* Information & Communication Technology Authentication & Network services *On behalf of** FCA Item* Corso Luigi Settembrini 167, Ingresso 19 10135 Torino – ITALY On Mon, Apr 19, 2021 at 9:13 AM Andre Lorbach <[email protected]> wrote: > Dear Fabio, > > Please see my comments below: > > > -----Ursprüngliche Nachricht----- > > Von: rsyslog <[email protected]> Im Auftrag von Fabio > > Dania via rsyslog > > Gesendet: Freitag, 16. April 2021 19:41 > > An: rsyslog-users <[email protected]> > > Cc: Fabio Dania <[email protected]> > > Betreff: [rsyslog] Rsyslog agent 7.0 - Increase Processed files Speed > > > > Good Morning All > > We have a lot of log files to process and send to siem, the csv files are > > about > > 1 GB dimension and are download on filesystem every 15 minutes > > > > How can we increase the line (then files) processed for seconds? > > For highest performance, keep Overrun Prevention Delay (ms) Setting to 0 - > this will remove any delay from the processing. > Then I recommend to enable "Delay writing Last File Position" and set it to > 100 which means the last processed file position will only be written every > 100 lines or after a timeout (If all lines are processed). > > > The Lines count seems not very performed. > > The count does not reflect the lines processed but the actual byte position > of the last processed line within the file. > > > The second question is related to the processed file: Where can we find > > the > > filename of the processed files? > > If you are using registry configuration mode, they can be find in the > registry under: > > HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Adiscon\RSyslogAgentData\Services\{GUIDOFSERVICE} > > Each processed file will have its own node there with nLastPos:Low and > nLastPos:High as DWORD values. > nLastPos:Low is the file position if the file does not exceed 4GB. > > > > Best regards, > Andre Lorbach > -- > Adiscon GmbH > Mozartstr. 21 > 97950 Großrinderfeld, Germany > Ph. +49-9349-9298530 > Geschäftsführer/President: Rainer Gerhards Reg.-Gericht Mannheim, HRB > 560610 > Ust.-IDNr.: DE 81 22 04 622 > Web: www.adiscon.com - Mail: [email protected] > > Informations regarding your data privacy policy can be found here: > https://www.adiscon.com/data-privacy-policy/ > > This e-mail may contain confidential and/or privileged information. If you > are not the intended recipient or have received this e-mail in error please > notify the sender immediately and delete this e-mail. Any unauthorized > copying, disclosure or distribution of the material in this e-mail is > strictly forbidden. > > Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte > Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail > irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und > vernichten Sie diese E-Mail. Das unerlaubte Kopieren und die unbefugte > Weitergabe dieser E-Mail sind nicht gestattet. > _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
