hi Jim,
the solutions from my previous post were provided without knowing the root
cause of the issue. If the main reason for the problem is the nature of the
timestamps in the logs (that is, they are provided with an accuracy of a
second), I would recommend using a different logging scheme with
high-resolution timestamps. For example, as David has mentioned in his
post, using syslog servers with RFC5424-based syslog protocol (
https://datatracker.ietf.org/doc/html/rfc5424) provides such timestamps
out-of-the-box. I am not sure whether you can change the logging scheme for
these particular application messages, but if you can, these timestamps are
highly useful for many purposes like the investigation of past system
faults and security incidents.
kind regards,
risto
hi Jim,
>
> let me provide some suggestions how to accomplish this task. First, you
> could utilize the context based approach that you have described in your
> post. In that case, the numeral that has been retrieved from the context
> with the 'pop' action needs to be incremented, and you can do it with the
> 'lcall' action that invokes the relevant Perl code. Here is one example (I
> have added several of extra actions into the rule for initializing the
> context):
>
> type=single
> ptype=regexp
> pattern=call_id (\S+)
> desc=call id $1
> action=exists %e call_uniqueid_seq_$1; \
> if %e ( none ) else ( add call_uniqueid_seq_$1 0); \
> pop call_uniqueid_seq_$1 %sequence_num; \
> lcall %sequence_num %sequence_num -> ( sub { $_[0] + 1 } ); \
> add call_uniqueid_seq_$1 %sequence_num
>
> The first two actions check if the context call_uniqueid_seq_* for the
> given call exists, and if not, 0 is written into that context to initialize
> it:
>
> exists %e call_uniqueid_seq_$1; \
> if %e ( none ) else ( add call_uniqueid_seq_$1 0);
>
> The 'pop' and 'add' actions (the 3rd and 5th action in the above rule) are
> identical to the actions from your email.
>
> The 'lcall' action (4th action in the above rule) increments the counter
> value retrieved from the context by executing the following Perl one-line
> function:
> sub { $_[0] + 1 }
> That function takes the value of its parameter (that is, %sequence_num)
> and returns an incremented value which is stored back into %sequence_num.
> Note that %sequence_num appears twice in the 'lcall' action, since it acts
> both as an input parameter for the Perl function and as the variable where
> the return value from the function is stored.
>
> There is also an alternative way for addressing the same assignment. This
> approach involves keeping the counter for each call ID not in a SEC
> context, but rather in a Perl hash table. That approach is more efficient,
> since the counter value is not copied back and forth between the Perl
> function and the SEC context each time it needs to be incremented. Instead,
> the counter value stays in a Perl data structure during these increments.
> Here is a relevant rule example:
>
> type=single
> ptype=regexp
> pattern=call_id (\S+)
> desc=call id $1
> action=lcall %sequence_num $1 -> ( sub { ++$calls{$_[0]} } ); \
> write - the new value for counter for call $1 is %sequence_num
>
> The above example involves the use of a Perl hash (associative array)
> 'calls', where the call ID serves as a key into the hash. For each key
> (call ID), the hash stores the counter for that call ID. Each time the
> function
> sub { ++$calls{$_[0]} }
> gets called, the counter for the relevant call ID (the value of $1) is
> incremented in the hash 'calls', and the result is stored to %sequence_num.
>
> If you just want to retrieve the current value of the counter for some
> call ID (the value of $1) without incrementing it, you can simply invoke
> the following action:
>
> lcall %sequence_num $1 -> ( sub { $calls{$_[0]} } )
>
> With both the context-based and Perl-hash-based approach, one needs to
> think about dropping stale contexts (or stale hash keys) for calls that are
> no longer relevant. With contexts, you can remove them with the 'delete'
> action. In the case you use the Perl hash for storing the counters, you can
> use the following 'lcall' action for deleting a key from the hash 'calls':
>
> lcall %sequence_num $1 -> ( sub { delete $calls{$_[0]} } )
>
> I hope the above examples are useful.
>
> kind regards,
> risto
>
>
>
>
>
>
>
>
> Kontakt Jim Van Meggelen (<jim.vanmegge...@clearlycore.com>) kirjutas
> kuupäeval R, 22. september 2023 kell 19:50:
>
>> I am using SEC for an atypical use case, where rather than using log
>> events to correlate events for things like security events, I am using it
>> to parse through Asterisk logs in order to capture a sequence of phone call
>> events (call comes in, auto attendant answers, user selects a digit, call
>> is transferred, etc). I have SEC write each to a CSV file, which can then
>> be processed downstream for reporting (for example, I produced a Sankey
>> chart using this data).
>>
>> SEC has proven fantastic for this, however one minor issue has been that
>> the timestamps in the logs are not granular to smaller than a second, so
>> it's possible for two or more events to occur within the same second.
>> Generally this doesn't cause a problem, but when sorting the CSV file
>> elsewhere, this can result in out-of-sequence lines, if they both contain
>> the exact same timestamp.
>>
>> So, what I've been trying to figure out is how to store a counter that is
>> tied to the uniqueid of the call, and increment that with each event. I
>> figured I'd be able to do this by storing an integer in the event store of
>> a context (tied to the uniqueid for that call). I can then increment it as
>> the various log lines of the call are processed in turn.
>>
>>
>>
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
