hi Richard, these are interesting questions and you can find my comments inline:
Hello guys, > > ... > > My question is, if you see, how some of this things could be accomplished > in more generic way, without special configurations of correlation rules. > It would be great having SEC supporting such use cases "out of box", e.g. > by: > > - having configurable line delimiter pattern (regular expression) > In the current code base, identifying the end of each line is done with a simple search for newline character. The newline is searched not with a regular expression, but rather with index() function which is much faster. It is of course possible to change the code, so that a regular expression pattern is utilized instead, but that would introduce a noticeable performance penalty. For example, I made couple of quick tests with replacing the index() function with a regular expression that identifies the newline separator, and when testing modified sec code against log files of 4-5 million events, cpu time consumption increased by 25%. Introducing a custom delimiter also raises another important question -- should it also be used for actions that utilize newline as a natural separator? For example, 'add' and 'event' actions split data into events by newline (if present in the data), 'spawn' action assumes that events from a child process are separated by newlines, etc. Should all these actions be changed to use the new separator? Given the performance penalty and other delimiter related questions, this idea needs careful thinking before implementing it in the code. (Before moving forward, it would be also interesting to know how many users would see this idea worth implementing.) > - accepting wildcard pattern as specification of input log file, to > "monitor them all" (also dynamically adding newly created files matching > wildcard and removing disappeared) > It would be easier to implement that functionality, since input file patterns are re-evaluated on some signals, and in principle it is possible to invoke similar code after short time periods (e.g., once in 5 seconds). However, sec-2.8.X has 'addinput' and 'dropinput' actions which offer more general interface for dynamically adding and dropping inputs. For example, it is possible to start an external script with 'spawn' action which can detect input files not just with wildcard match but also more advanced criteria, and generate synthetic events like NEWFILE_<file> for input files that need opening. These synthetic events can be easily captured by a rule which invokes 'addinput' action for relevant files. I acknowledge that this functionality is somewhat different from providing wildcards in command line and requires writing your own script, but you can actually do more advanced things here. kind regards, risto > I don't have clue, how hard would be implementation of such things > directly in SEC (maybe question to Risto?), or if do you see also other, > more straightforward, solutions, without bringing more complexity to SEC > rules, I would be grateful for your know-how sharing. > > Have a nice days. > > Richard > >
_______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
