On Thu, 7 Feb 2013, Risto Vaarandi wrote: > On 02/06/2013 11:59 PM, David Lang wrote:
>> I think this sort of thing would be useful, a lot of 'action scripts' end up >> being trivial wrappers to do these sorts of things, and opening/closing files >> and starting programs can have a surprisingly large effect on performance (I >> suspect that SEC has other bottlenecks, but even if it's not a bottleneck, >> speeding it up will leave more time for the bottleneck code :-) >> >> I would not try to match up command lines, as you note it's hard to tell when >> you really have two identical ones, and it makes it hard to have multiple >> connectons open to the same destination (important for network connections) >> >> This crys out for a filehandle approach, have something define the output and >> then write2 can use it. You can have calendar events close and re-open the >> files >> for log rotation > > The use of filehandles introduces another problem. Suppose the > connection to other host is closed for some reason. If connections > should be explicitly created by the end user with 'open' action which > also creates a handle, it would also be the responsibility of the end > user to check the connection status and re-create it if necessary. On > the other hand, if 'write2' action is reopening connections when > necessary, the end user would be free of defining any extra connection > management rules. change the statement from 'file handle' to 'connection handle', when the connection is defined, it gets opened. If something happens to close it, write2 should re-open it. but write2 would reopen a connection if it's closed when write2 tries to use it. this would look basically the same as a filehandle that can never close to the user, but would operate diffently under the covers. sorry if the term 'filehandle' lead to narrowing the behavior too much. >> now, as to specific output options. My first thing is make this extendable, >> people will have lots of strange outputs they want to talk to. Make it so >> they >> can define a perl subroutine and use that. >> >> you need pipe to command line (think |gzip -c ->/var/log/alerts.gz) >> > > I am actually also thinking of reusing the already existing 'spawn' > action for arbitrary command lines. One very common approach of using > 'spawn' has been to start a command line with it which is reading its > input through a named pipe. In other parts of the ruleset, the 'write' > action can then be used to write into this particular named pipe. This > will drop the need for distinguishing command lines from each other and > having handles for them, since the filename of the named pipe provides a > natural unique identifier. > > In the current version, there has been one issue related to using > 'spawn' in the above way -- since 'write' opens and closes the named > pipe at each write operation, a process listening to the pipe will see > EOF after each event and could terminate (this will happen to many > common tools like 'logger' or 'cat'). However, if the 'write' action > would keep the pipe constantly open, many unix command line tools would > not need such frequent restarting. This can work, but unless SEC also creates the named pipe, it seems like there are more steps needed to set it up than should be needed. It would also be nice if there was some way to create a pool of processes, so that if one is blocked, the output goes to the second one. >> you need send to syslog, but this may not be just send the raw string. Syslog >> message formatting is common across the different transport types, so it >> would >> be more 'open syslog destination' than 'open tcp destination' >> >> In formatting the syslog message, options that jump out at me include: >> >> facility/priority >> >> syslog tag (program name) >> >> source hostname (it's useful to be able to generate alert messages with >> the >> name of the relavent server, not just the SEC box) >> >> syslog format (historic syslog rfc3164, new syslog rfc5424, lumberjack >> JSON, >> CEE, CEF, ....) >> >> how to handle multiline (report) data (send as multiple log messages, >> send as >> one message with newlines, send as one message with a different separator >> character,...) >> >> for these, there should be a default set at open time, but it can make sense >> to >> override these for particular messages. >> >> Much of this formatting stuff is useful even if just writing to files, >> so it >> may be that the right thing is to define the transport at open time and then >> have a function call to format the message at log time. >> >> >> thoughts? >> >> David Lang >> > > I'd leave the message formatting completely open to the end user. First, > I think it would be quite hard to figure out defaults for many cases > (e.g. how to handle scenarios where a newline is inside the message > text). Also, the number of potential formats can be very large. > Fortunately, for syslog it is very easy to manually craft an rfc3164 and > rfc5424 message, and I've often used telnet and nc for connecting remote > syslog servers and interactively typing in valid messages. So it is > probably not very hard to craft a proper syslog message from the sec rule... I almost agree with you, the user should format the message, but they should have some helper functions pre-defined to deal with the most common cases. These could be in an include file if you want them defined in the config namespace rather than the core namespace. These would both satisfy a lot of users for the common case, and provide examples for people to hack on when they need something more complicated. David Lang ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb _______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
