[Simple-evcorr-users] sec-2.7.3 released

2013-05-30 Thread Risto Vaarandi
hi all, today, sec-2.7.3 has been released which adds several improvements to the previous version: - sockets created by 'tcpsock', 'udpsock', 'ustream' and 'udgram' actions are managed in a better way, allowing for timely detection of peer shutdowns and restarts - improvements to 'write' and 'spa

[Simple-evcorr-users] process interactivity check for SIGINT overloading

2013-05-28 Thread Risto Vaarandi
hi all, as you know, since the 2.5 version there has been a simple check inside the code for process interactivity, in order to use the INT signal for changing the logging level. Currently, this test is done in a simple way through the use of -t STDIN (tests if STDIN is connected to terminal).

Re: [Simple-evcorr-users] SEC on the big data security log management

2013-05-05 Thread Risto Vaarandi
2013/5/5 David Lang > On Fri, 3 May 2013, John Zhang wrote: > > Hi everyone, >> >> I am researching the big data security log management, such as Kibana + >> ElasticSearch + Logstash for my security log management, I need event >> correlation on this platform, i know SEC( >> http://simple-evcor

Re: [Simple-evcorr-users] SEC on the big data security log management

2013-05-02 Thread Risto Vaarandi
hi John, if you plan to use Logstash for feeding Elasticsearch database, SEC can be quite easily connected to it, since Logstash supports receiving data through wide variety of inputs. Depending on your system and log data volumes, you could have just one SEC instance which correlates all your even

[Simple-evcorr-users] asking for user opinions regarding tcp socket handling

2013-04-26 Thread Risto Vaarandi
hi all, as you know, the support for tcp sockets was added recently to the SEC code which allows for integrations with applications capable of receiving data over TCP connections (like Graphite and various syslog servers). The current implementation of 'tcpsock' action is fairly basic and only

Re: [Simple-evcorr-users] PairWithWindow logic timing (continue and continue2).

2013-04-12 Thread Risto Vaarandi
hi Gary, the 'continue' and 'continue2' parameters specify that the event should be passed to the following rules after the pattern has matched and the rule has processed for the event. Here the processing means only immediate activities, and everything which takes time is done asynchronously. For

[Simple-evcorr-users] sec-2.7.2

2013-04-12 Thread Risto Vaarandi
hi all, the 2.7.2 version of sec has been released which fixes a bug in parsing the 'rewrite' action. Due to this bug, 'rewrite' was not read in and an incorrect error message was reported. The new version with a bugfix is available at: http://sourceforge.net/projects/simple-evcorr/files/sec/2.

Re: [Simple-evcorr-users] trap suppression and threshold

2013-03-30 Thread Risto Vaarandi
d from the Single rule and use it in the SingleWith2Thresholds rule with the appropriate alarm string for /usr/OV/bin/event. regards, risto 2013/3/30 Risto Vaarandi > ...the ruleset seems to be not recognized as plain text with the .conf > extension: > > type=SingleWith2Thresholds >

Re: [Simple-evcorr-users] trap suppression and threshold

2013-03-30 Thread Risto Vaarandi
ype=RegExp pattern=GENERATE_CPU_ALARM_(\w+vp\d+) desc=$1 high CPU realarm action=shellcmd /usr/OV/bin/event -e NDWN_EV -h $1 \ -d "TEST EVENT: 2 $1 system-critical-00030: SYSTEM CPU utilization is high."; \ create CPU_REALARM_$1 1800 (event GENERATE_CPU_ALARM_$1) 2013/3/30 R

Re: [Simple-evcorr-users] trap suppression and threshold

2013-03-30 Thread Risto Vaarandi
hi Vernon, as John and David have already suggested, you need to fix the 'desc' keyword of the counting rule, since this sets the scope of event correlation. Using $0 for 'desc' is not a good idea, since $0 match variable holds the *entire* matching line, including timestamps and CPU utilization va

Re: [Simple-evcorr-users] log / sec question

2013-03-28 Thread Risto Vaarandi
On 03/25/2013 11:30 PM, ward.p.fonte...@wellsfargo.com wrote: > It's three different servers running the same application (dev, test, prod). > What I'm looking to do - and it does do by the way - is rotate the > application log based on "Shutdown complete" showing in the logs. I just > wanted to ma

Re: [Simple-evcorr-users] How to write an event to syslog-ng

2013-03-15 Thread Risto Vaarandi
On 03/15/2013 04:50 PM, John P. Rouillard wrote: > > In message<5142e00c.3050...@seb.ee>, > Risto Vaarandi writes: >> On 12/15/2012 12:01 AM, mindman101 wrote: >>> Does anyone know how to send an event to syslog-ng under sec rules? >>> >>> I m

Re: [Simple-evcorr-users] How to write an event to syslog-ng

2013-03-15 Thread Risto Vaarandi
2013/3/15 John P. Rouillard : > > In message <5142e00c.3050...@seb.ee>, > Risto Vaarandi writes: >>On 12/15/2012 12:01 AM, mindman101 wrote: >>> Does anyone know how to send an event to syslog-ng under sec rules? >>> >>> I mean, something like: &g

Re: [Simple-evcorr-users] How to write an event to syslog-ng

2013-03-15 Thread Risto Vaarandi
On 12/15/2012 12:01 AM, mindman101 wrote: > > Hello list! > > Does anyone know how to send an event to syslog-ng under sec rules? > > I mean, something like: > > action: send_syslog(facility, level, event) > > Thanks for your support. In the 2.7.1 version, one efficient way for this is communicati

[Simple-evcorr-users] sec-2.7.1 released

2013-03-14 Thread Risto Vaarandi
I am sorry for the double posting, but somehow I forgot the subject line from the previous mail :( On 03/14/2013 05:12 PM, Risto Vaarandi wrote: > hi all, > > SEC-2.7.1 has been released and is available from: > http://sourceforge.net/projects/simple-evcorr/files/sec/2.7.1/sec-

[Simple-evcorr-users] (no subject)

2013-03-14 Thread Risto Vaarandi
hi all, SEC-2.7.1 has been released and is available from: http://sourceforge.net/projects/simple-evcorr/files/sec/2.7.1/sec-2.7.1.tar.gz/download The main changes in this version are four new actions for writing data to TCP, UDP and UNIX sockets. The actions are fairly simple and assume the data

Re: [Simple-evcorr-users] PairWithWindow Help Needed.

2013-03-13 Thread Risto Vaarandi
On 03/13/2013 05:25 PM, Boyles, Gary P wrote: > Risto, > Yes, I'm specifying integers through a web interface. At this point though, > it is through > a list of integers rather than free-form integers. > > I'm trying to minimize the number of rules written, by utilizing hash-tables > to decide o

Re: [Simple-evcorr-users] PairWithWindow Help Needed.

2013-03-13 Thread Risto Vaarandi
ow rule through two Single rules, with a context of variable lifetime acting as a connector between two rules. Since contexts accept variables for their lifetime, that might be a solution for the problem. kind regards, risto > > > -Original Message- > From: Risto Vaarandi [

Re: [Simple-evcorr-users] PairWithWindow Help Needed.

2013-03-08 Thread Risto Vaarandi
urposes, or do you intend to modify the window size? kind regards, risto > > Thanks a lot for your help. > > Gary Boyles > > > -Original Message- > From: Risto Vaarandi [mailto:risto.vaara...@gmail.com] > Sent: Tuesday, March 05, 2013 4:56 AM > To: Boyles, Gary P &

Re: [Simple-evcorr-users] Beginner needs simple help about pattern matching with ftp logs

2013-03-05 Thread Risto Vaarandi
> Again, thanks for this great information. Here is another question from > a newbie: > > If I want one config file but two log files, i.e., the rules in the > config file pull from two different log files, what is the syntax for that? > > I now have the following two config files and, although I

Re: [Simple-evcorr-users] PairWithWindow Help Needed.

2013-03-05 Thread Risto Vaarandi
hi Gary, you have run into a subtle issue that is related to pattern2 processing when the same Pair* rule has started several operations. The desc field of your rule is working properly, since you are seeing different PairWithWindow operations in memory for different nodes. However, when you have s

Re: [Simple-evcorr-users] extending output types (RabbitMQ)

2013-02-21 Thread Risto Vaarandi
idea would help solve the issues... >> >>>> Therefore, would there be any interest to an output action which would >>>> leave the output open/running/connected, so that the next access of the >>>> same output would not have to do reopen/restart/connec

Re: [Simple-evcorr-users] extending output types (RabbitMQ)

2013-02-12 Thread Risto Vaarandi
and the consideration of > integrating it (or as a plug-in) with sec. > > Regards, > Steve Busko | NOC Tools Manager > >> -Original Message- >> From: Risto Vaarandi [mailto:risto.vaara...@seb.ee] >> Sent: Wednesday, February 06, 2013 12:38 PM >> To:

Re: [Simple-evcorr-users] extending output types (RabbitMQ)

2013-02-12 Thread Risto Vaarandi
On 02/11/2013 11:04 PM, Clayton Dukes wrote: > Hi Steve, > I'm a little confused. Why would you need RabbitMQ to insert only 40 > events/sec? > My syslog tool (LogZilla) does 15k events/sec into MySQL without using > Rabbit. > We are working now (using RabbitMQ) on getting that number to around > 1

Re: [Simple-evcorr-users] extending output types

2013-02-07 Thread Risto Vaarandi
On 02/07/2013 01:28 PM, David Lang wrote: > On Thu, 7 Feb 2013, Risto Vaarandi wrote: > >> >> I am actually also thinking of reusing the already existing 'spawn' >> action for arbitrary command lines. One very common approach of using >> 'spawn'

Re: [Simple-evcorr-users] extending output types

2013-02-07 Thread Risto Vaarandi
On 02/06/2013 11:59 PM, David Lang wrote: > On Wed, 6 Feb 2013, Risto Vaarandi wrote: > >> hi all, >> >> as you know, currently there are number of outputs which are not >> directly supported by sec. For example, with 'pipe' action one can feed >> on

Re: [Simple-evcorr-users] extending output types

2013-02-06 Thread Risto Vaarandi
nt-queue). > If you meant pipes to external programs, then one pipe for each distinct command line. kind regards, risto > > > -----Original Message- > From: Risto Vaarandi [mailto:risto.vaara...@seb.ee] > Sent: Wednesday, February 06, 2013 9:38 AM > To: Simple-evcorr-users &

Re: [Simple-evcorr-users] Using environment variable in write within rules.

2013-02-06 Thread Risto Vaarandi
hi Gary, environment variables are a concept related to a UNIX shell, and inside sec they are not visible. Of course, if you trigger command lines from sec which get interpreted by the shell, the environment variables make sense again. For example, env MYVAR="this is a test" sec-2.7.0/sec -conf=te

[Simple-evcorr-users] extending output types

2013-02-06 Thread Risto Vaarandi
hi all, as you know, currently there are number of outputs which are not directly supported by sec. For example, with 'pipe' action one can feed only one event to an external program (with most programs exiting after the 'pipe' action closes the pipe). Also, the 'write' action opens a given fi

[Simple-evcorr-users] Debian package information for sec-2.7.0

2013-02-01 Thread Risto Vaarandi
To: Risto Vaarandi Hi Risto, I rolled package that can be installed for stable and frozen versions (6.0 onwards): http://liiwi.idle.fi/sec/ You can point people to there for now. I'll will upload next version to unstable as normal after some small cleanups on packaging during next we

[Simple-evcorr-users] SEC-2.7.0

2013-01-29 Thread Risto Vaarandi
hi all, SEC-2.7.0 has been released and can be downloaded at http://sourceforge.net/projects/simple-evcorr/files/sec/2.7.0/sec-2.7.0.tar.gz/download kind regards, risto -- Master Visual Studio, SharePoint, SQL, ASP.NET

Re: [Simple-evcorr-users] Cleanliness is next to high performance

2012-12-28 Thread Risto Vaarandi
On 12/28/2012 02:47 AM, John P. Rouillard wrote: > >> In my own environment, I've seen accidental differences of 8-10KB >> which disappear when a new dump file is created in the next second. >> >> However, it is important to know that the read offset is not always >> reflecting the amount of alread

Re: [Simple-evcorr-users] Cleanliness is next to high performance

2012-12-27 Thread Risto Vaarandi
hi John, in the 2.7 version there is an alternative for measuring the lag. I've added few lines of code to the dump file creation function which record the size and current read offset for the input file (this is done only for regular files, not FIFOs). As a result, the recorded statistics looks li

[Simple-evcorr-users] SEC-2.7.alpha2 released

2012-12-27 Thread Risto Vaarandi
format change for highlighting frequently matching rules, but also a small fix for syslog logging. Also, I take the opportunity and wish all list members all the best for 2013. kind regards, risto On 11/26/2012 11:55 AM, Risto Vaarandi wrote: > hi all, > > the first alpha version of SE

Re: [Simple-evcorr-users] More table like display of stats

2012-12-06 Thread Risto Vaarandi
2012/12/6 John P. Rouillard : > > In message <50c06562.8030...@seb.ee>, > Risto Vaarandi writes: >>On 12/05/2012 05:17 AM, John P. Rouillard wrote: >>> Currently the output of a state dump (generated with a kill -USR1) >>> looks like: >>> >>&

Re: [Simple-evcorr-users] More table like display of stats

2012-12-06 Thread Risto Vaarandi
On 12/05/2012 05:17 AM, John P. Rouillard wrote: > > Currently the output of a state dump (generated with a kill -USR1) > looks like: > > Rule 1 at line 31 (Clear EVENT_PROCESSED) has matched 6431597 events > Rule 2 at line 51 (Skip all processing for slapd) has matched 14126179 events > Rule 3 at

Re: [Simple-evcorr-users] sec 2.7alpha ideas/notes

2012-12-05 Thread Risto Vaarandi
On 12/05/2012 05:38 AM, John P. Rouillard wrote: > > Hi Risto and all: > > I have been looking at the 2.7 alpha release and I have a few > comments. The functions that deal with setting a variables from some > context seem to have different calling conventions > > copy, empty, pop, shift > > a

Re: [Simple-evcorr-users] Fix logging for SIGINT handler

2012-12-04 Thread Risto Vaarandi
On 12/04/2012 01:45 AM, John P. Rouillard wrote: > > In message > , > Risto Vaarandi writes: > >> 2012/12/3 John P. Rouillard: >>> One of the nice things about SIGINT handiling when sec is running as a >>> daemon is that you can increase debugging. >>

Re: [Simple-evcorr-users] Fix logging for SIGINT handler

2012-12-04 Thread Risto Vaarandi
On 12/04/2012 01:45 AM, John P. Rouillard wrote: > > In message > , > Risto Vaarandi writes: > >> 2012/12/3 John P. Rouillard: >>> One of the nice things about SIGINT handiling when sec is running as a >>> daemon is that you can increase debugging. >>

Re: [Simple-evcorr-users] Fix logging for SIGINT handler

2012-12-03 Thread Risto Vaarandi
2012/12/3 John P. Rouillard : > > One of the nice things about SIGINT handiling when sec is running as a > daemon is that you can increase debugging. > > One of the bad things is that it increments and wraps the debug value. > Once you wrap to 1 you don't see any more indications of the debug > lev

Re: [Simple-evcorr-users] Anyone using JSON with SEC and SEC alpha.

2012-11-29 Thread Risto Vaarandi
2012/11/29 Boyles, Gary P : > Hi, > > > > I was wondering if anyone is using JSON for events with SEC, and if so… how > had you implemented it. > > > > In addition, David Lang wrote this about the SEC alpha version. I was > wondering if there are any examples around this? > > A SEC rule snippet wo

Re: [Simple-evcorr-users] SEC-2.7.alpha1

2012-11-26 Thread Risto Vaarandi
On 11/26/2012 06:36 PM, John P. Rouillard wrote: > > In message<50b33ca3.7050...@seb.ee>, > Risto Vaarandi writes: >> the first alpha version of SEC-2.7 has been released and is available at >> http://sourceforge.net/projects/simple-evcorr/files/sec/2.7.alpha1/sec-2.7

[Simple-evcorr-users] SEC-2.7.alpha1

2012-11-26 Thread Risto Vaarandi
hi all, the first alpha version of SEC-2.7 has been released and is available at http://sourceforge.net/projects/simple-evcorr/files/sec/2.7.alpha1/sec-2.7.alpha1.tar.gz/download It has quite many new features, including 14 new actions which allow for more advanced operations on contexts, but a

Re: [Simple-evcorr-users] dealing with JSON based logs

2012-11-05 Thread Risto Vaarandi
with the following structure {"host":{"ip":["127.0.0.1","127.0.0.2"],"dns":"localhost"}} the following line is echoed back: 127.0.0.1 127.0.0.2 localhost Of course, since the above parsing is done without any knowledge about the fi

Re: [Simple-evcorr-users] dealing with JSON based logs

2012-11-05 Thread Risto Vaarandi
On 11/04/2012 11:58 PM, da...@lang.hm wrote: > On Sun, 4 Nov 2012, da...@lang.hm wrote: > >> On Sun, 4 Nov 2012, Risto Vaarandi wrote: >> >>> 2012/10/20: >>>> On Sat, 20 Oct 2012, Risto Vaarandi wrote: >>>> >>> hi David, >>>

Re: [Simple-evcorr-users] dealing with JSON based logs

2012-11-05 Thread Risto Vaarandi
ing a preliminary check for $+{: in the string with a switch to simpler regular expression will help here. with kind regards, risto On 11/05/2012 12:33 AM, John P. Rouillard wrote: > > On Sat, 20 Oct 2012, Risto Vaarandi wrote: >> >> hi David, >> I have completed some wor

Re: [Simple-evcorr-users] dealing with JSON based logs

2012-11-04 Thread Risto Vaarandi
2012/11/4 Mark D. Nagel : > On 11/4/2012 3:04 AM, Risto Vaarandi wrote: >> I have completed some work on the alpha version of the next release, >> and it supports returning hashes from PerlFunc patterns. If the user >> defines a custom parsing scheme inside PerlFunc pattern, k

[Simple-evcorr-users] dealing with JSON based logs

2012-11-04 Thread Risto Vaarandi
2012/11/4 : > On Sun, 4 Nov 2012, da...@lang.hm wrote: > >> On Sun, 4 Nov 2012, Risto Vaarandi wrote: >> >>> 2012/10/20 : >>>> >>>> On Sat, 20 Oct 2012, Risto Vaarandi wrote: >>>> >>> hi David, >>> I have completed

[Simple-evcorr-users] dealing with JSON based logs

2012-11-04 Thread Risto Vaarandi
2012/10/20 : > On Sat, 20 Oct 2012, Risto Vaarandi wrote: > >> As for the naming convention -- Perl has a requirement that match >> variable names can consist of alphanumerals and underscores. >> Currently, similar sanity check has been implemented for SEC named >> m

Re: [Simple-evcorr-users] SingleWithSuppress "Window" Parameter.

2012-11-02 Thread Risto Vaarandi
hi Gary, the value for the window field has to be an integer constant. However, suppression of repeated events can also be easily done with contexts, and their lifetimes are freely adjustable. kind regards, risto 2012/11/3 Boyles, Gary P : > Is there a way to make a window a variable in a “SingleW

Re: [Simple-evcorr-users] SingleWithSuppress Question.

2012-11-01 Thread Risto Vaarandi
hi Gary, the scope of event suppression is entirely set with the 'desc' field of the rule. Each time a new event comes in, 'desc' is evaluated and combined with the rule file name and rule number. If there is an event correlation operation already running for the resulting value, event is consumed

Re: [Simple-evcorr-users] How to delete all contexts that match pattern ?

2012-10-25 Thread Risto Vaarandi
On 10/25/2012 01:30 PM, Robert Charroux wrote: > Hi, i'm trying to find a way to delete a certain amount of contexts a > rule created. this based on a daily task. > the contexts are created like this by the rule : already_got_$2 , where > $2 is a hostname > > So, what i'd like to do is something li

Re: [Simple-evcorr-users] dealing with JSON based logs

2012-10-20 Thread Risto Vaarandi
2012/10/20 Mark D. Nagel : > On 10/20/2012 1:40 AM, Risto Vaarandi wrote: >> a separate rule type is one option, but in my other mail I was >> actually suggesting your approach -- to have an improved PerlFunc >> pattern type, from which named match variables could be returned

[Simple-evcorr-users] dealing with JSON based logs

2012-10-20 Thread Risto Vaarandi
2012/10/20 : > On Fri, 19 Oct 2012, Risto Vaarandi wrote: > >> On 10/18/2012 03:20 AM, da...@lang.hm wrote: >>> There is a lot of activity nowdays around 'structured' logging, with a lot >>> of the effort going into having systems generate logs in the JSO

Re: [Simple-evcorr-users] dealing with JSON based logs

2012-10-20 Thread Risto Vaarandi
2012/10/20 Mark D. Nagel : > [meant to send this to the list, not only Risto -- sorry for the dup, > Risto!] > > On 10/18/2012 12:02 PM, Risto Vaarandi wrote: >> I am thinking about having a rule like >> >> type=Parse >> ptype=PerlFuncN >> pattern=s

Re: [Simple-evcorr-users] Use Of User-Defined % variables in multiple rules???

2012-10-19 Thread Risto Vaarandi
2012/10/19 Boyles, Gary P : > Hi, > > I’ve got a question about creating and using % variables in my rules. > > > > I was under the impression that the %variable contents were accessible from > other rules if in the same configuration file. hi Gary, the variables which the user can create in actio

Re: [Simple-evcorr-users] dealing with JSON based logs

2012-10-19 Thread Risto Vaarandi
On 10/18/2012 03:20 AM, da...@lang.hm wrote: > There is a lot of activity nowdays around 'structured' logging, with a lot > of the effort going into having systems generate logs in the JSON format > (see the stuff from systemd, project lumberjack, etc). Some of this is > going to be showing up in t

Re: [Simple-evcorr-users] dealing with JSON based logs

2012-10-18 Thread Risto Vaarandi
2012/10/18 Risto Vaarandi : > On 10/18/2012 03:20 AM, da...@lang.hm wrote: >> There is a lot of activity nowdays around 'structured' logging, with a lot >> of the effort going into having systems generate logs in the JSON format >> (see the stuff from systemd, pr

Re: [Simple-evcorr-users] SingleWithSuppress Question.

2012-10-18 Thread Risto Vaarandi
hi Gary it is possible, if you employ the EventGroup rule which is a generalization of both SingleWithThreshold and SingleWithSuppress. This question was asked about a year ago, so I will also post a link to the relevant post: http://sourceforge.net/mailarchive/message.php?msg_id=27913498 Shortly,

Re: [Simple-evcorr-users] dealing with JSON based logs

2012-10-18 Thread Risto Vaarandi
On 10/18/2012 03:20 AM, da...@lang.hm wrote: > There is a lot of activity nowdays around 'structured' logging, with a lot > of the effort going into having systems generate logs in the JSON format > (see the stuff from systemd, project lumberjack, etc). Some of this is > going to be showing up in t

Re: [Simple-evcorr-users] Use Of lcall.

2012-10-15 Thread Risto Vaarandi
On 10/12/2012 07:47 PM, Boyles, Gary P wrote: > John, > Thanks for the information. > > Is it possible to set %xx variables as part of a pattern return, or $n, or is > that also just reserved for "action" statements. > > For example... this code works, but I'd like to retain the node and class >

Re: [Simple-evcorr-users] Setup and use of context and variables.

2012-10-10 Thread Risto Vaarandi
hi Gary, below are short answers to your questions: 2012/10/10 Boyles, Gary P : > Hi, > > I’ve got a question or two on CONTEXT and PATTERNS. > > > > I’ve written the following rule, and it appears to work fine… that is, with > the hostname in $4 it returns a data-center and pod that the hostname

Re: [Simple-evcorr-users] Need help on using perlfunc with pattern.

2012-10-04 Thread Risto Vaarandi
On 10/03/2012 02:07 AM, Boyles, Gary P wrote: > All, > > I’ve got this simple rule, and I keep getting the error… > > *Use of uninitialized value $node in exists at (eval 13) line 1.* > > Here is the rule. Any obvious errors you can see? After having a quick look into the function, the following f

Re: [Simple-evcorr-users] SEC system requirements and limitations on servers and alerts

2012-10-04 Thread Risto Vaarandi
On 10/02/2012 11:38 PM, Joseph Guanzon wrote: > Hi Guys, > > Is there known system requirement when using SEC to monitor large quantity of > servers like how many cpu/memory would be needed for 300 to 500 servers and > or 500 to 1000 servers monitored? Can SEC be able to summarize log file > al

Re: [Simple-evcorr-users] Context and desc.

2012-09-15 Thread Risto Vaarandi
2012/9/15 : > On Fri, 14 Sep 2012, Boyles, Gary P wrote: > >> Can someone point me to documentation that describes context and desc? >> >> There seems to be more uses for it that I can find documentation on. > > the tutorials linked to from the SEC page do a pretty good job of covering > them. >

Re: [Simple-evcorr-users] Patch to allow setting just context action and not change lifetime

2012-09-14 Thread Risto Vaarandi
hi John, I think it is a nice additional feature, and it is possible to implement it without the fear of side effects. With early versions of SEC context lifetimes had to be numerical constants which would have meant a change in several places of code. Fortunately, with more recent versions the

Re: [Simple-evcorr-users] Jump rule oddness

2012-09-14 Thread Risto Vaarandi
On 09/14/2012 12:25 AM, John P. Rouillard wrote: > > Hi all: > > I have the following setup: > > ruleset 01 does some processing > > ruleset 05 has a jump rule in it that jumps to the END cf set > >(rules to spit out new event and clear some contexts) > >type= jump >continue = dontcont

Re: [Simple-evcorr-users] Get date in mmddyyyy format

2012-09-11 Thread Risto Vaarandi
hi Akash, date numbers in file names can be handled with a symbolic link which can be created from Calendar rule each midnight. Here are links to relevant posts in mailing list archive: http://sourceforge.net/mailarchive/message.php?msg_id=26661175 http://sourceforge.net/mailarchive/message.php?msg

Re: [Simple-evcorr-users] In-Memory Hash Array for event-enhancement.

2012-09-07 Thread Risto Vaarandi
On 09/06/2012 11:59 PM, da...@lang.hm wrote: > On Thu, 6 Sep 2012, Boyles, Gary P wrote: > >> Hi, >> >> I'm new to SEC and have one problem I'm hoping can be solved within SEC. >> >> I want to load a (hash) table on SEC startup that contains the host >> (e.g. foobar.x.y.com) as the key, and a value

Re: [Simple-evcorr-users] Integarting SEC with other monitoring tools

2012-08-30 Thread Risto Vaarandi
monitoring console on 7x24 (or at least 5x8) basis. In small companies of 5-10 persons where there is no dedicated helpdesk, it is often better to send e-mails to relevant persons. with kind regards, risto > > > -Original Message- > From: Risto Vaarandi [mailto:risto.vaara.

Re: [Simple-evcorr-users] Integarting SEC with other monitoring tools

2012-08-30 Thread Risto Vaarandi
2012/8/30 Joseph Guanzon : > Hi Guys, > > > > I’m trying to look for a monitoring tool that I can integrate with other > monitoring tools like HP Openview, HP OVO, Geneos Active Console ( ITRS ) > and BMC Patrol and I can also configure to generate ticket automatically. > hi Joseph, like other ans

Re: [Simple-evcorr-users] Incremental parsing of an event using multiple rules

2012-08-15 Thread Risto Vaarandi
2012/8/14 John P. Rouillard : > Hi all: > > I know this should be doable, and I know my current thinking is > barking up the wrong tree, but my brain is locked into doing something > and it's not quite working. > > Here's what happened. In order to get the output from postgres into > the proper for

[Simple-evcorr-users] a paper on SEC from ISSA journal

2012-08-11 Thread Risto Vaarandi
hi all, a paper about SEC has been published in the August 2012 issue of the iSSA journal (a joint work with Michael Grimaila) which is now also available at the SEC home page. The paper is titled "Security Event Processing with Simple Event Correlator" and describes three security event processing

Re: [Simple-evcorr-users] SEC graceful termination question...

2012-07-20 Thread Risto Vaarandi
2012/7/20 Mike Ellis : > Hi All, I'm new to SEC... I like it so far. > > Still doing mostly simple things, but I'm starting to gradually get more > complex. > > I ran into something that didn't act exactly as I expected and wanted to see > how others are dealing with this. > > -- > > Sending the SE

Re: [Simple-evcorr-users] sec not catching new lines on file

2012-06-06 Thread Risto Vaarandi
On 06/05/2012 01:10 PM, Pedro Rafael Alves Simoes wrote: > Hello, > > I have a problem with while reading new lines from a file wih sec. > This is the scenario: > > Log file with lines: > > F, [2012-06-05T10:55:10.096883 #5420] FATAL -- : RESTART_ME > rinda_take: druby://localhost:12345 - # Connect

Re: [Simple-evcorr-users] Counting in Pair/Threshold

2012-05-24 Thread Risto Vaarandi
On 05/24/2012 02:42 PM, Richard Jones wrote: > Hi All, > > I have a question. From the man page: > > The SingleWithThreshold rule runs event correlation operations for > counting repeated instances of the same event during T seconds, and > taking an action if N events are observed. > >

Re: [Simple-evcorr-users] Using SEC in offline mode and writing rules for complex event co-relation

2012-05-21 Thread Risto Vaarandi
want to spam so capturing in one single line > > Another question I have is, how can I capture N same consecutive lines as one > single line like "XYZ repeats 10 times consecutively" For this, you can employ RegExpN pattern type (e.g., for use RegExp10 for matching N consecutive li

Re: [Simple-evcorr-users] Out of sequence logs

2012-05-21 Thread Risto Vaarandi
2012/5/18 Richard Jones : > On Thu, May 17, 2012 at 05:00:51PM +0300, Risto Vaarandi wrote: >> hi Richard, is my understanding correct that the first two log >> messages are coming from one server, and are thus always appearing in >> this order? If the third message is the

Re: [Simple-evcorr-users] Using SEC in offline mode and writing rules for complex event co-relation

2012-05-19 Thread Risto Vaarandi
hi Jyothi, for offline processing, the use of Perl variables for counting is indeed the best approach. In fact, you can optimize this ruleset a bit by capturing the last two rules into one rule (I haven't tested it, but the idea is to increment the counter in the context evaluation): type=single p

Re: [Simple-evcorr-users] Out of sequence logs

2012-05-17 Thread Risto Vaarandi
hi Richard, is my understanding correct that the first two log messages are coming from one server, and are thus always appearing in this order? If the third message is the one that might appear before (or in between) of them, then you could try EventGroup rule for which is designed for matching ev

Re: [Simple-evcorr-users] Using SEC in offline mode and writing rules for complex event co-relation

2012-05-08 Thread Risto Vaarandi
hi Jyothi, the main working mode of SEC is real-time event log processing -- events are correlated as they come in, and intervals between events are measured according to the real system clock. Although you can do some event matching for past logs, much of the event correlation features would not w

Re: [Simple-evcorr-users] Evaluating variables in a context

2012-05-02 Thread Risto Vaarandi
On 04/30/2012 08:07 PM, mindman101 wrote: > Hello Risto, > > Thanks for your answer. However, I was trying the rules and I got > unexpected results. For example: I forgot to change the action of the first rule. If the reporting is moved outside from the action-on-expiration list of the linkDown

Re: [Simple-evcorr-users] Evaluating variables in a context

2012-04-24 Thread Risto Vaarandi
On 04/21/2012 02:21 AM, mindman101 wrote: > Hello list! > > I configured three single rules: the first one trigger a context and the > next two add a text to the context when there is a match. > > In the first rule I copied the content of the context to a variable. > > I know I should use a miniper

Re: [Simple-evcorr-users] Does anybody know where James Brown's 2 part tutorial went to?

2012-04-23 Thread Risto Vaarandi
On 04/24/2012 12:52 AM, John P. Rouillard wrote: > Hi all: > > Back 5 or more years ago James Brown created a very nice tutorial on > using SEC and included a number of neat extentions including linking > it into a database etc. > > Does anybody know where that went to. I know he moved it somewhere

Re: [Simple-evcorr-users] Rules to increment/decrement a counter, add, remove items from a list, and fire on counter threshold.

2012-04-03 Thread Risto Vaarandi
On 04/02/2012 06:38 PM, John Grasett wrote: > Rules to increment/decrement a counter, add, remove items from a list, > and fire on counter threshold. > > Given a number of servers, and rules in existance already: > > 1. detect OOM, add a context: OOM_on_serverX (!OOM_on_severX is ther > also to pre

Re: [Simple-evcorr-users] Determining when sec is falling behind

2012-03-22 Thread Risto Vaarandi
On 03/21/2012 10:05 PM, John P. Rouillard wrote: > >> There is one crucial difference >> between the number of processed lines and file position, though. The >> former reflects lines successfully read and processed from a given file. >> However, it is possible that the file position is located beyo

Re: [Simple-evcorr-users] Determining when sec is falling behind

2012-03-21 Thread Risto Vaarandi
On 03/20/2012 09:54 PM, John P. Rouillard wrote: > > Hi all: > > When sec creates a dump file, the input sources are reported as: > > Input sources: > > /var/log/messages (status: Open, type: regular file, device/inode: > 64774/8339528,

Re: [Simple-evcorr-users] multiple instances of sec getting spawned?

2012-03-08 Thread Risto Vaarandi
On 03/08/2012 05:54 AM, Joe Prosser wrote: > Hi Folks, > I'm seeing a situation where somehow multiple instances of a sec > process are getting spawned when there should only be one. > > The extra copies most of the time disappear, but sometimes they don't > and there are hundreds of them and they

Re: [Simple-evcorr-users] Trying to alert on the ratio between two types of logs

2012-03-05 Thread Risto Vaarandi
On 02/29/2012 01:49 AM, da...@lang.hm wrote: > I want to setup an alert based on too many of one type of log showing up > compared to another type of log during a window > > Ideally, with the appropriate log messages being in a report > > For example, I want to look at the number of successful and

Re: [Simple-evcorr-users] using context to check next sequence number in flow

2012-03-01 Thread Risto Vaarandi
hi Robert, you are having this problem because %a variable is an action list variable -- it can only be employed within action lists, not context expressions. Instead of %a, I would recommend to use a Perl variable. For example, instead of returning an incremented value from the 'lcall' action, you

[Simple-evcorr-users] anyone willing to update sec package for debian?

2012-02-10 Thread Risto Vaarandi
hi all, as you might know, SEC has been packaged for a number of major Linux distros, including Debian, and the Ubuntu package is a copy from Debian repository. Unfortunately, the Debian package has not been updated for two years and it has become quite stale. Moreover, the most recent version

Re: [Simple-evcorr-users] Scanning the logs using SEC

2012-02-03 Thread Risto Vaarandi
Single rules from them. hope this helps, risto > > -Original Message- > From: Risto Vaarandi [mailto:risto.vaara...@seb.ee] > Sent: Thursday, February 02, 2012 2:43 AM > To: simple-evcorr-users@lists.sourceforge.net > Subject: Re: [Simple-evcorr-users] Scanning the

Re: [Simple-evcorr-users] Reset command

2012-02-02 Thread Risto Vaarandi
On 02/02/2012 04:13 PM, l2 l2 wrote: > I can see that the 'reset' command only supports resetting rules > within the same configuration file. > > Nevertheless, I'm looking for a way to do exactly that... resetting a > rule from another configuration file. > > I was wondering if someone could help m

Re: [Simple-evcorr-users] Scanning the logs using SEC

2012-02-01 Thread Risto Vaarandi
On 02/01/2012 11:22 PM, ashok.vaira...@emc.com wrote: > Hello Team, > > We are planning to use this tool to triage and debug in our product. But > we do have few queries. Can you please clarify it? > > 1. Is it possible to use SEC to scan for specific pattern from a group > of log files? I do notic

Re: [Simple-evcorr-users] Re : Tr : Re : SEC & unix sockets

2012-01-24 Thread Risto Vaarandi
On 01/24/2012 12:43 AM, sylver_b wrote: > Jan 01:29:33.498/GLOBAL/ser: RECEIVED message from 91.x.x.x:33583:|INVITE > sip:39329172@sip.x SIP/2.0|Supported:|Allow: INVITE, ACK, OPTIONS, > CANCEL, BYE|Contact: sip:131400@91.x.x.x:33583|Via: SIP/2.0/UDP > 91.x.x.x:33583;branch=z9hG4bKe65d47

Re: [Simple-evcorr-users] Tr : Re : SEC & unix sockets

2012-01-23 Thread Risto Vaarandi
s fraudulent and blocked . We know how to > disconnect a call in real time if the callID is provided. What would be the > best way to implement such rule with SEC? > > a SIP module would be an amazing addition to SEC ..  thank you > > > De 

[Simple-evcorr-users] SEC-2.6.2 released

2012-01-23 Thread Risto Vaarandi
hi all, SEC-2.6.2 has been released which is available at: http://sourceforge.net/projects/simple-evcorr/files/sec/2.6.2/sec-2.6.2.tar.gz/download The changes and improvements in this version are the following: 1) Support for mutually exclusive --jointbuf and --nojointbuf options: the --nojointb

Re: [Simple-evcorr-users] Howto count pair correlation

2012-01-21 Thread Risto Vaarandi
hi, if you would like to keep track of all PIDs of hanging processes, generate an error on the appearance of first hanging process, and to generate OK on the disappearance of the last hanging process, you could use this ruleset: type=single ptype=regexp pattern=Another process is running: PID \[(\

Re: [Simple-evcorr-users] openSUSE Packages available

2012-01-20 Thread Risto Vaarandi
On 01/19/2012 11:56 PM, Malcolm wrote: > On Thu, 19 Jan 2012 16:34:08 -0500 > "John P. Rouillard" wrote: >> >> Might including the systemd compatibility components in the sec >> distribution be worthwhile (c.f. contrib/startup.freebsd and >> contrib/startup.solaris) if Malcolm is willing? >> > Hi

Re: [Simple-evcorr-users] openSUSE Packages available

2012-01-19 Thread Risto Vaarandi
Malcolm, thanks a lot for your work! In fact, quite recently I noticed that the suse package is very old, and was wondering whether anyone would be willing to update it :) with kind regards, risto 2012/1/19 Malcolm : > Hi List > I've been using SEC for a number of years on both Solaris and Linux.

<    1   2   3   4   5   6   7   8   9   10   >