Hi Risto, thank you for positive answer - dumping period in minutes in enough, not needed many times per second. And also for useful tips - I already noticed JSON option, just considering, that default is maybe more universally usable, because extra modules (JSON.pm) may not be installed or installable on any systems (not only due technical reasons, but also security policies - more code auditing needed). Maybe the question also could be, how frequent are changes of dump file formats during SEC development, which could break monitoring implemented for particular version of SEC.
Richard ut 28. 1. 2020 o 12:40 Risto Vaarandi <risto.vaara...@gmail.com> napísal(a): > hi Richard, > > as I understand from your post, you would like to create SEC dump files > periodically, in order to monitor performance of SEC based on these dump > files. Let me first provide some comments on performance related question. > Essentially, creation of dump file involves a pass over all major internal > data structures, so that summaries about internal data structures can be > written into dump file. In fact, SEC traverses all internal data structures > for housekeeping purposes anyway once a second (you can change this > interval with --cleantime command line option). Therefore, if you re-create > the dump file after reasonable time intervals (say, after couple of > minutes), it wouldn't noticeably increase CPU consumption (things would > become different if dumps would be generated many times a second). > > For production use, I would provide couple of recommendations. Firstly, > you could consider generating dump files in JSON format which might make > their parsing easier. That feature was requested couple of years ago > specifically for monitoring purposes ( > https://sourceforge.net/p/simple-evcorr/mailman/message/36334142/), and > you can activate JSON format for dump file with --dumpfjson command line > option. You could also consider using --dumpfts option which adds numeric > timestamp suffix to dump file names (e.g., /tmp/sec.dump.1580210466). SEC > does not overwrite the dump file if it already exists, so having timestamps > in file names avoids the need for dump file rotation (you would still need > to delete old dump files from time to time, though). > > Hopefully this information is helpful. > > kind regards, > risto > > Kontakt Richard Ostrochovský (<richard.ostrochov...@gmail.com>) kirjutas > kuupäeval E, 27. jaanuar 2020 kell 22:04: > >> Greetings, friends, >> >> there is an idea to monitor high-volume event flows (per rules or log >> files, e.g. in bytes per time unit) from dump files from *periodic >> dumping*. >> >> The question is, if this is recommended for *production use* - answer >> depends at least on if dump creation somehow slows down or stops SEC >> processing, or it has not significant overhead (also for thousands of >> rules, complex configurations). >> >> Are there some best practises for (or experiences with) performance >> self-monitoring and tuning of SEC? >> >> Thank you. >> >> Richard >> _______________________________________________ >> Simple-evcorr-users mailing list >> Simple-evcorr-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users >> >
_______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
