Hi Risto,
wow, thanks a lot for so detailed answer. My answers on your questions
please see below:
1. *But before suggesting anything more concrete, how exactly are input*
*events appearing? *
Linux auditd daemon produces logs for every syscall event configured
in audit.rules and store it in file /var/log/audit/audit.log
2.* Are the events consecutive and always in the same **order?*
The events order is: SYSCALL EXECVE CWD PATH. All have unique
fields: time (epoch format) and event id.
In most case i see SYSCALL CWD PATH (later will aggregate all events to see
better picture).
But we can get EXECVE event and it appears only when program are
being executed.
Also in some cases we can get 2 PATH log messages for same event id. In one
case 1st PATH message is preferable and for another case 2nd one is most
informative.
Anyway I have completed this configuration and seems everything is working.
Will share all my config with community.
The task is:
1. Correlate all audit events and produce one synthetic event
2. Resolve all unreadable field's values in synthetic rule with human
readable.
My config:
-------------------------------------------------------------------------------------------------------
# ---- Rule 1
# This rule for SYSCALL hash generating
# Next, we want to load the hashes at start, restart, soft restart (so that
we can modify the file and send SEC an SIGHUP)
type=Single
desc=Load hashes for SYSCALS at STARTUP
ptype=RegExp
continue=TakeNext
pattern=SEC_STARTUP|SEC_RESTART|SEC_SOFTRESTART
context=SEC_INTERNAL_EVENT
action=logonly; lcall %syscalls -> ( sub { %SyscallHashes = (); @syscalls =
split(/\n/,`/bin/ausyscall --dump`); shift @syscalls; \
foreach (@syscalls) { my ($key, $val) = split(/\s+/, $_);
$SyscallHashes{"$key"} = $val; }; return %SyscallHashes; } )
# ---- Rule 2
# This rule for User hash generating
# Next, we want to load the hashes at start, restart, soft restart
type=Single
desc=Load hashes for UIDS at STARTUP
ptype=RegExp
continue=TakeNext
pattern=SEC_STARTUP|SEC_RESTART|SEC_SOFTRESTART
context=SEC_INTERNAL_EVENT
action=logonly; lcall %uids -> ( sub { %UserHashes = (); open(PASSWD_FILE,
"</etc/passwd"); \
while (<PASSWD_FILE>) { my @user = split /:/; $UserHashes{"$user[2]"} =
$user[0]; }; return %UserHashes; } )
# ---- Rule 3
# This rule for Group hash generating
# Next, we want to load the hashes at start, restart, soft restart
type=Single
desc=Load hashes for GIDS at STARTUP
ptype=RegExp
continue=TakeNext
pattern=SEC_STARTUP|SEC_RESTART|SEC_SOFTRESTART
context=SEC_INTERNAL_EVENT
action=logonly; lcall %guids -> ( sub { %GroupHashes = (); open(GROUP_FILE,
"</etc/group"); \
while (<GROUP_FILE>) { my @group = split /:/; $GroupHashes{"$group[2]"} =
$group[0]; }; return %GroupHashes; } )
# ---- Rule 4
# WAIT FOR SYSCALL EVENT TYPE
# type=SYSCALL msg=audit(1479132918.851:11624): arch=c000003e syscall=2
success=yes exit=4 a0=6ccb00 a1=0 a2=7fa90401f768 a3=7ffe5520ad70 items=1
ppid=2334 pid=11801 auid=1003 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0
sgid=0 fsgid=0 tty=pts0 ses=12 comm="less" exe="/usr/bin/less" key=(null)
# WAIT FOR CWD EVENT TYPE
# type=CWD msg=audit(1479132918.851:11624): cwd="/var/log"
# WAIT FOR PATH EVENT TYPE
# type=PATH msg=audit(1479132918.851:11624): item=0 name="audit/audit.log"
inode=1108139 dev=ca:01 mode=0100600 ouid=0 ogid=0 rdev=00:00 objtype=NORMAL
# Create context for particular auditd log ID. Lifetime of context is 20
seconds
# Correlate messages type=PATH with fields item=(1|0);
objtype=(CREATE|NORMAL|UNKNOWN)
type=EventGroup3
ptype=regexp
pattern=type=CWD msg=audit\((\d+\.\d+):(\d+)\):\s+cwd="(.+?)"
varmap= time=1; event_id=2; cwd=3
count= alias SYSCALL_RULE1_$+{event_id} CWD_$+{event_id}; assign %vcwd
$+{cwd}
ptype2=regexp
pattern2=type=PATH msg=audit\((\d+\.\d+):(\d+)\): (?:item=1
name="(?<obj_name>.+?)".*objtype=CREATE|item=0
name="(?<obj_name>.+?)".*objtype=NORMAL)
varmap2=vPATH; time=1; event_id=2
context2=CWD_$+{event_id}
count2= assign %vpath $+{obj_name}
ptype3=regexp
pattern3=type=SYSCALL msg=audit\((\d+\.\d+):(\d+)\): arch=(\w+)
syscall=(\d+) success=(\w+).+?ppid=(\d+) pid=(\d+) auid=(\d+) uid=(\d+)
gid=(\d+) euid=(\d+) suid=(\d+) fsuid=(\d+) egid=(\d+) sgid=(\d+)
fsgid=(\d+) tty=(\w+) ses=\d+ comm="(\w+)" exe="(.+?)" key=\(.+?\)
varmap3= time=1; event_id=2; arch=3; syscall=4; success=5; ppid=6; pid=7;
auid=8; uid=9; gid=10; euid=11; suid=12; fsuid=13; egid=14; sgid=15;
fsgid=16; tty=17; comm=18; exe=19; key=20
desc=Rules matched $+{time}_$+{event_id}
action=lcall %cur_syscall $+{syscall} -> ( sub { $SyscallHashes{$_[0]} } );
\
lcall %cur_time $+{time} -> ( sub { my $human_time = scalar
localtime($_[0]); return $human_time; } ); \
lcall %cur_arch $+{arch} -> ( sub { if($_[0]='c000003e'){return
'x86_64';}return 'x86';} ); \
lcall %cur_syscall $+{syscall} -> ( sub { $SyscallHashes{$_[0]} } ); \
lcall %cur_auid $+{auid} -> ( sub { $UserHashes{$_[0]} } ); \
lcall %cur_uid $+{uid} -> ( sub { $UserHashes{$_[0]} } ); \
lcall %cur_euid $+{euid} -> ( sub { $UserHashes{$_[0]} } ); \
lcall %cur_suid $+{suid} -> ( sub { $UserHashes{$_[0]} } ); \
lcall %cur_gid $+{gid} -> ( sub { $GroupHashes{$_[0]} } ); \
lcall %cur_egid $+{egid} -> ( sub { $GroupHashes{$_[0]} } ); \
lcall %cur_sgid $+{sgid} -> ( sub { $GroupHashes{$_[0]} } ); \
create FINAL_MATCH_PATH1_$+{time}_$+{event_id}; event type=SYSCALL
msg=audit(%cur_time:$+{event_id}): arch=%cur_arch syscall=%cur_syscall
success=$+{success} ppid=$+{ppid} pid=$+{pid} auid=%cur_auid uid=%cur_uid
gid=%cur_gid euid=%cur_euid suid=%cur_suid fsuid=$+{fsuid} egid=%cur_egid
sgid=%cur_sgid fsgid=$+{fsgid} tty=$+{tty} comm=$+{comm} exe=$+{exe}
key=$+{key} cwd="%vcwd" obj_name="%vpath"
init=create SYSCALL_RULE1_$+{event_id} 20
slide=delete SYSCALL_RULE1_$+{event_id}; reset 0
end=delete SYSCALL_RULE1_$+{event_id}
window=20
On Wed, Nov 16, 2016 at 7:38 PM, Risto Vaarandi <risto.vaara...@gmail.com>
wrote:
> hi Nikolay,
>
> the Eventgroup3 rule that you have written might not be the best
> option for addressing this task, since it has been designed for
> scenarios where events can appear in arbitrary order. However, in the
> case of auditd records the three events should always have a fixed
> order: SYSCALL, CWD, PATH (any event order can of course change if you
> are collecting events from other hosts, since specific
> protocols/implementations can alter original event order).
>
> Also, your Eventgroup3 rule has one caveat -- since the SYSCALL event
> is always coming first, it will initialize the event correlation
> operation, and at that point all match variables are getting
> substituted (that includes the 'action' field). In the 'action' field,
> you have used variables $:{vcwd:cwd} and $+{obj_name}, but these
> variables do not have any values at the time of substitution. For the
> same reason, If the PATH event would come first, you would have only
> $+{obj_name} variable set, and all other variables without values.
> Also, if you store match variables into a pattern match cache with
> 'varmap' statement, they remain accessible during the current input
> line processing, and after all rules have been matched against the
> line, the pattern match cache is cleared. So there is no way to access
> the value of match variable $:{vcwd:cwd} from previous CWD event when
> sec is currently processing the PATH event.
>
> In order to address the matching of three events, there are several
> workarounds:
>
> 1) define count* fields for storing $+{cwd} and $+{obj_name} into
> contexts (e.g., count2=add PATH $+{obj_name} ), and extract the
> content of these contexts when the action is executed (e.g., copy PATH
> %obj_name). This approach works, since the contexts (unlike match
> variables) are persistent entities and disappear only when it is
> explicitly configured.
>
> 2) If the events are always appearing in the same order and no other
> events can appear in between them, these three events can be simply
> handled by one Single rule that uses the RegExp3 pattern (RegExp3
> matches three consecutive lines).
>
> 3) if the given three events are not always consecutive, you could try
> several other strategies like storing match variables into a Perl hash
> table and updating its content, until you have seen all three events.
>
> But before suggesting anything more concrete, how exactly are input
> events appearing? Are the events consecutive and always in the same
> order?
>
> kind regards,
> risto
>
> 2016-11-16 16:22 GMT+02:00 Nikolay Srebniuk
> <nikolay.srebn...@tonicforhealth.com>:
> > Hi Everybody,
> >
> > i'm struggling 2nd day to make this rule to be working, but no results.
> >
> > What I want to achieve:
> > 1. I have 3 auditd log messages: SYSCALL, CWD, PATH
> > 2. I want to correlate these messages by fields time_id and event_id
> > 3. Generate one correlate rules with:
> > a) all fields from SYSCALL message
> > b) with one filed (cwd) from CWD message
> > c) with one filed (name) from PATH message
> >
> > ISSUE:
> > 1. Only fields from SYSCALL message are inserting in CORRELATED event.
> > 2. Fileds from CWD and PATH are emptry
> >
> > Question:
> > How to get this to work? Kindly appreciate your help.
> >
> > Rule:
> > type=EventGroup3
> > ptype=RegExp
> > pattern=type=CWD msg=audit\((\d+\.\d+):(\d+)\):\s+cwd="(.+?)"
> > varmap=vcwd; time=1; event_id=2; cwd=3
> > ptype2=RegExp
> > pattern2=type=PATH msg=audit\((\d+\.\d+):(\d+)\): item=\d+ name="(.+?)"
> > (inode=|objtype=).*
> > varmap2= time=1; event_id=2; obj_name=3
> > ptype3=RegExp
> > pattern3=type=SYSCALL msg=audit\((\d+\.\d+):(\d+)\): arch=(.+?)
> > syscall=(\d+) success=(.+?).+?ppid=(\d+) pid=(\d+) auid=(\d+) uid=(\d+)
> > gid=(\d+) euid=(\d+) suid=(\d+) fsuid=(\d+) egid=(\d+) sgid=(\d+)
> > fsgid=(\d+) tty=(.+?) ses=\d+ comm="(.+?)" exe="(.+?)" key=\(.+?\)
> > varmap3= time=1; event_id=2; arch=3; syscall=4; success=5; ppid=6; pid=7;
> > auid=8; uid=9; gid=10; euid=11; suid=12; fsuid=13; egid=14; sgid=15;
> > fsgid=16; tty=17; comm=18; exe=19; key=20
> > desc=Rules matched $+{time}_$+{event_id}
> > action=create FINAL_MATCH_$+{time}_$+{event_id}; event type=SYSCALL
> > msg=audit($+{time}:$+{event_id}): arch=$+{arch} syscall=$+{syscall}
> > success=$+{success} ppid=$+{ppid} pid=$+{pid} auid=$+{auid} uid=$+{uid}
> > gid=$+{gid} euid=$+{euid} suid=$+{suid} fsuid=$+{fsuid} egid=$+{egid}
> > sgid=$+{sgid} fsgid=$+{fsgid} tty=$+{tty} comm=$+{comm} exe=$+{exe}
> > key=$+{key} cwd=$:{vcwd:cwd} obj_name=$+{obj_name}
> > init=create $+{time}_$+{event_id} 20
> > slide=delete $+{time}_$+{event_id}; reset 0
> > end=delete $+{time}_$+{event_id}
> > window=20
> >
> > Sec service configuration:
> > [Service]
> > Type=forking
> > PIDFile=/run/sec.pid
> > ExecStart=/usr/bin/sec --detach --pid=/run/sec.pid
> > --conf=/etc/sec/auditd-sec.conf --input=/var/log/audit/audit.log
> > --log=/var/log/sec.log --intevents --syslog=20 --debug=6
> > --dump=/tmp/sec_dump --bufsize=100
> >
> > Raw log input:
> > type=SYSCALL msg=audit(1479282206.600:23242): arch=c000003e syscall=2
> > success=yes exit=3 a0=7ffede744e2a a1=0 a2=0 a3=7ffede743d80 items=1
> > ppid=16533 pid=16670
> > auid=1003 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
> tty=pts0
> > ses=10 comm="tail" exe="/usr/bin/tail" key=(null)
> > type=CWD msg=audit(1479282206.600:23242): cwd="/home/user"
> > type=PATH msg=audit(1479282206.600:23242): item=0 name="/var/log/sec"
> > inode=16837888 dev=ca:01 mode=0100644 ouid=0 ogid=0 rdev=00:00
> > objtype=NORMAL
> >
> >
> > --
> > Best regards,
> > Nikolay
> >
> > ------------------------------------------------------------
> ------------------
> >
> > _______________________________________________
> > Simple-evcorr-users mailing list
> > Simple-evcorr-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
> >
>
Wbr,
Nikolay
------------------------------------------------------------------------------
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
