In message <e1781bcc862e47809a0a8d87e18af...@wsp5042a.baltic.seb.net>,
Risto Vaarandi writes:
>I am currently working on the 2.7.7 version, and a recent e-mail
>exchange with one of the users has inspired me to think about
>changing default values for --bufsize and --jointbuf/--nojointbuf
>options.
> [...]
>Nevertheless, when bufsize=1 would be default, some of the code for
>the single-line case could be factored out and written more
>efficiently, which would allow for some performance gains for
>single-line scenario. The downside of changing the default from
>bufsize=10 to bufsize=1 would be the need to set --bufsize explicitly
>on command line, in order to make pattern types like RegExp3,
>NRegExp2, and PerlFunc5 to work. So far, there has been rarely a need
>for this, since --bufsize=10 has been sufficient for most of the
>cases.
>[...]
>Also, currently SEC assumes --jointbuf option by default which means
>that in the case of multi-line matching all events are stored into
>the same input buffer. Nevertheless, in this case --nojointbuf would
>make more sense, since that creates a separate buffer for each input
>source, allowing multiline patterns to work on data from one source
>only. Since with bufsize=1 there is no difference between --jointbuf
>and --nojointbuf, the --nojointbuf option would be a more reasonable
>default.
I think this makes sense. What I suggest is:
if --buffsize > N and any RegExpN, PerlFuncN ... multiline match rules
are defined, SEC should throw an error ("Use --bufsize=N on command
line, currentl setting is too small to allow this rule to work").Then
SEC should exit. I would conssider that a fatal error. This is a
little different from how SEC currently handles invalid rules, but I
think this change would warrant SEC exiting.
If bufsize > 1 then --nojointbuf becomes the default (it might as well
be the default for size=1 as well, but it's effectively a noop in that
case so it doesn't matter). I have never had multiline matches with
multiple inputs work reliably. There was always a race condition. I
have always used nojointbuf with multiple input streams and multiline
match rules.
So I think those are both good ideas. One thing I will note is that a
project I am working on supplies events in a multiline json data
structure, so making sure that the refactoring doesn't make multiline
process/matching worse is critical to me.
--
-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.
------------------------------------------------------------------------------
New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
