[Simple-evcorr-users] RegExp modifiers

2019-12-08 Thread Richard Ostrochovský
Hello colleagues, I was searching for the answer here: https://simple-evcorr.github.io/man.html https://sourceforge.net/p/simple-evcorr/mailman/simple-evcorr-users/ and haven't found the answer, so I'am putting new question here: Does SEC in pattern= parameters support RegExp modifiers (

[Simple-evcorr-users] "multi-line" and multi-file logs - out of box

2019-11-26 Thread Richard Ostrochovský
Hello guys, in fact, this question is about 2 independent things, but I see interesting parallels to think about both topics together: I know, that it je possible to create SEC rules configurations (correlators), to process: - "multi-line" logs (means having message separator other than "\n")

Re: [Simple-evcorr-users] "multi-line" and multi-file logs - out of box

2019-11-27 Thread Richard Ostrochovský
Risto, thank you for your pre-analysis about multi-lines with regexp, and also for suggestions about multi-files yet more sophisticated solution. My comments are also inline: st 27. 11. 2019 o 15:07 Risto Vaarandi napísal(a): > hi Richard, > ... > In the current code base, identifying the end

Re: [Simple-evcorr-users] "multi-line" and multi-file logs - out of box

2019-11-30 Thread Richard Ostrochovský
hi Risto, thank you for sharing. You are undoubtedly right, it is common, that "delimiter" pattern uses to be beginning part of the message (we can't say this about "\n", which is not "part of the beginning"... delimiting the end of message is then optional, in cases where detecting the beginning

[Simple-evcorr-users] generate SEC configurations automatically

2019-11-30 Thread Richard Ostrochovský
Hello friends, this post loosely follows this one: https://sourceforge.net/p/simple-evcorr/mailman/message/36867007/. Being monitoring consultant and developer, I have an idea to hide complexity of SEC configurations, and still to allow to configure it also for "regular" administrators without

Re: [Simple-evcorr-users] generate SEC configurations automatically

2019-12-01 Thread Richard Ostrochovský
hi John and Risto, Being a "regular" administrator, I claim admins >> that can't program/use programming methods won't >> be admins much longer. If the companies you work >> for are depending on (rapidly turning over) junior >> admins to administer something as important as >> monitoring and

Re: [Simple-evcorr-users] generate SEC configurations automatically

2019-12-02 Thread Richard Ostrochovský
po 2. 12. 2019 o 14:19 Risto Vaarandi napísal(a): > >> This SEC admin, as I see it, is still also from user perspective, tightly >> bound with SEC technology and user needs to know, how SEC rules and their >> compositions work internally. I am imagining something, that is >> technology-agnostic,

[Simple-evcorr-users] SEC rules performance monitoring and tuning

2020-01-27 Thread Richard Ostrochovský
Greetings, friends, there is an idea to monitor high-volume event flows (per rules or log files, e.g. in bytes per time unit) from dump files from *periodic dumping*. The question is, if this is recommended for *production use* - answer depends at least on if dump creation somehow slows down or

Re: [Simple-evcorr-users] SEC + AI (machine learning)

2020-01-27 Thread Richard Ostrochovský
Thank you, Risto, such overview can serve as starting point, when designing something yet more useful, than static configurations of correlations. I am still not sure, what the optimal goal should be, I need to analyze and study more about these topics. I believe, that I will follow up on

[Simple-evcorr-users] how not to keep monitored files permanently open (not only) on NFS

2020-02-04 Thread Richard Ostrochovský
Hi Risto and friends, I am unsure about one conceptual question about how SEC keeps open monitored files. Using SEC as systemd service, when files stored in NFS (opened via addinput) being watched by SEC are moved elsewhere, and then their removal is tried, NFS persistently keeps .nfsNUMBER

Re: [Simple-evcorr-users] how not to keep monitored files permanently open (not only) on NFS

2020-02-07 Thread Richard Ostrochovský
hi Risto, thank you for your helpful explanation about inner functionality of SEC. Closed files in my case were not existing, that was the reason. Richard ut 4. 2. 2020 o 23:09 Risto Vaarandi napísal(a): > hi Richard, > > I have never used SEC for monitoring files on NFS file systems, but I

Re: [Simple-evcorr-users] how not to keep monitored files permanently open (not only) on NFS

2020-02-07 Thread Richard Ostrochovský
to (immediately?) close (re)moved file, instead of keeping original i-node open until its recreation in its original location. pi 7. 2. 2020 o 12:38 Richard Ostrochovský napísal(a): > hi Risto, > > thank you for your helpful explanation about inner functionality of SEC. &

Re: [Simple-evcorr-users] SEC rules performance monitoring and tuning

2020-02-04 Thread Richard Ostrochovský
.1580210466). SEC > does not overwrite the dump file if it already exists, so having timestamps > in file names avoids the need for dump file rotation (you would still need > to delete old dump files from time to time, though). > > Hopefully this information is helpful. > > kind r

[Simple-evcorr-users] action-list checking if log file is already open by SEC

2020-02-20 Thread Richard Ostrochovský
Hello Risto and friends, having mechanism for dynamic opening (addinput) and closing (dropinput) files, I would like to be able to check, if the file is already opened, before trying to open it again, to avoid it. That way I would like to eliminate this error message from SEC log (present also in

Re: [Simple-evcorr-users] RegExp modifiers

2020-01-20 Thread Richard Ostrochovský
/?node=Categorized%20Questions%20and%20Answers. Maybe I would help with something like this, when I will have less busy time.) po 9. 12. 2019 o 16:43 Risto Vaarandi napísal(a): > hi Richard, > > Kontakt Richard Ostrochovský () kirjutas > kuupäeval E, 9. detsember 2019 kell 01:57

[Simple-evcorr-users] SEC + AI (machine learning)

2020-01-20 Thread Richard Ostrochovský
Hello, this is free continuation of https://sourceforge.net/p/simple-evcorr/mailman/message/36867012/. That post was about possibilities of user-friendly configurations of event correlations outside of SEC (without knowing SEC syntax and low-level principles), and generation of SEC rules from

[Simple-evcorr-users] log files existence and accessibility

2020-04-06 Thread Richard Ostrochovský
Hello friends, I am thinking about how to monitor not only events from log files, but also those files existence and accessibility (for user running SEC) - in cases, where this is considered to be a problem. As I saw in the past, these were logged into SEC log file, but higher debug level was

Re: [Simple-evcorr-users] log files existence and accessibility

2020-04-08 Thread Richard Ostrochovský
That is one way how to check input files without forking anything from > SEC. However, if the root cause of the problem is related to SELinux, it is > probably much better to adjust SELinux configuration (perhaps by changing > file contexts), so that the problem would not appear. > > k

Re: [Simple-evcorr-users] SEC CPU utilization

2020-04-01 Thread Richard Ostrochovský
contain messages from a specific > application which are processed by few specific rule files, a separate SEC > process could be started for handling these messages with given rule files. > In that way, it might be possible to divide the rule files and input files > into several independent groups,

Re: [Simple-evcorr-users] action-list checking if log file is already open by SEC

2020-04-01 Thread Richard Ostrochovský
nal is received, SEC will stop monitoring input > files set up with 'addinput' action. However, on receiving HUP signal SEC > will also drop all its contexts, so there is no need to take any extra > steps in that case. > > Hope this helps, > risto > > Kontakt Richard Ost