Daniel -
These machines are agents. As far as I can tell, they're all running:
[r...@yamaguchi ~]# ps -ef|grep ossec
root 4594 1 0 Mar15 ? 00:00:00 /var/ossec/bin/ossec-execd
ossec 4598 1 0 Mar15 ? 00:00:00 /var/ossec/bin/ossec-agentd
root 4602 1 0 Mar15 ? 00:00:00
/var/ossec/bin/ossec-logcollector
root 4606 1 0 Mar15 ? 00:00:34
/var/ossec/bin/ossec-syscheckd
so I do a stop everything:
[r...@yamaguchi ~]# /etc/init.d/ossec stop
Stopping OSSEC: [ OK ]
[r...@yamaguchi ~]# ps -ef|grep ossec
[r...@yamaguchi ~]#
and they all go away, and all of the lock files disappear. So, starting
over everything:
[r...@yamaguchi ~]# /etc/init.d/ossec start
Starting OSSEC: [ OK ]
[r...@yamaguchi ~]# ps -ef|grep ossec
root 30569 1 0 15:05 ? 00:00:00 /var/ossec/bin/ossec-execd
ossec 30573 1 0 15:05 ? 00:00:00 /var/ossec/bin/ossec-agentd
root 30577 1 0 15:05 ? 00:00:00
/var/ossec/bin/ossec-logcollector
root 30581 1 0 15:05 ? 00:00:00
/var/ossec/bin/ossec-syscheckd
[r...@yamaguchi ~]#
... but the process still is locked:
[r...@yamaguchi etc]# tail /var/log/ossec.log
2009/03/16 15:05:48 ossec-execd: INFO: Started (pid: 30569).
2009/03/16 15:05:53 ossec-syscheckd: INFO: Started (pid: 30581).
2009/03/16 15:05:53 ossec-rootcheck: INFO: Started (pid: 30581).
2009/03/16 15:05:54 ossec-logcollector(1950): INFO: Analyzing file:
'/var/log/messages'.
2009/03/16 15:05:54 ossec-logcollector(1950): INFO: Analyzing file:
'/var/log/auth.log'.
2009/03/16 15:05:54 ossec-logcollector(1950): INFO: Analyzing file:
'/var/log/snort/alert'.
2009/03/16 15:05:54 ossec-logcollector: INFO: Started (pid: 30577).
2009/03/16 15:06:02 ossec-logcollector: WARN: Process locked. Waiting for
permission...
2009/03/16 15:14:50 ossec-syscheckd: INFO: Starting syscheck scan (db).
2009/03/16 15:14:50 ossec-syscheckd: WARN: Process locked. Waiting for
permission...
Versions. ossec-agentd 2.0, trying to talk to a 2.0 server. ossec.conf:
[r...@yamaguchi etc]# cat ossec.conf
<ossec_config>
<client>
<server-ip>192.168.1.200</server-ip>
</client>
<syscheck>
<!-- Frequency that syscheck is executed - default to every 6 hours -->
<frequency>21600</frequency>
<!-- Directories to check (perform all possible verifications) -->
<directories check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
<directories check_all="yes">/bin,/sbin</directories>
<!-- Files/directories to ignore -->
<ignore>/etc/mtab</ignore>
<ignore>/etc/mnttab</ignore>
<ignore>/etc/hosts.deny</ignore>
<ignore>/etc/mail/statistics</ignore>
<ignore>/etc/random-seed</ignore>
<ignore>/etc/adjtime</ignore>
<ignore>/etc/httpd/logs</ignore>
<ignore>/etc/utmpx</ignore>
<ignore>/etc/wtmpx</ignore>
<ignore>/etc/cups/certs</ignore>
<ignore>/etc/dumpdates</ignore>
<ignore>/etc/svc/volatile</ignore>
<!-- Windows files to ignore -->
<ignore>C:\WINDOWS/System32/LogFiles</ignore>
<ignore>C:\WINDOWS/Debug</ignore>
<ignore>C:\WINDOWS/WindowsUpdate.log</ignore>
<ignore>C:\WINDOWS/iis6.log</ignore>
<ignore>C:\WINDOWS/system32/wbem/Logs</ignore>
<ignore>C:\WINDOWS/system32/wbem/Repository</ignore>
<ignore>C:\WINDOWS/Prefetch</ignore>
<ignore>C:\WINDOWS/PCHEALTH/HELPCTR/DataColl</ignore>
<ignore>C:\WINDOWS/SoftwareDistribution</ignore>
<ignore>C:\WINDOWS/Temp</ignore>
<ignore>C:\WINDOWS/system32/config</ignore>
<ignore>C:\WINDOWS/system32/spool</ignore>
<ignore>C:\WINDOWS/system32/CatRoot</ignore>
</syscheck>
<rootcheck>
<rootkit_files>/var/ossec/etc/shared/rootkit_files.txt</rootkit_files>
<rootkit_trojans>/var/ossec/etc/shared/rootkit_trojans.txt</rootkit_trojans>
</rootcheck>
<!-- Files to monitor (localfiles) -->
<localfile>
<log_format>syslog</log_format>
<location>/var/log/messages</location>
</localfile>
<localfile>
<log_format>syslog</log_format>
<location>/var/log/auth.log</location>
</localfile>
<localfile>
<log_format>snort-fast</log_format>
<location>/var/log/snort/alert</location>
</localfile>
</ossec_config>
[r...@yamaguchi etc]#
and the error log says:
[r...@yamaguchi etc]# grep -E "ERROR|WARN" /var/ossec/logs/ossec.log
2009/03/15 03:31:41 ossec-logcollector: WARN: Process locked. Waiting for
permission...
2009/03/15 03:32:18 ossec-logcollector: WARN: Process locked. Waiting for
permission...
2009/03/15 03:42:56 ossec-syscheckd: WARN: Process locked. Waiting for
permission...
2009/03/16 15:06:02 ossec-logcollector: WARN: Process locked. Waiting for
permission...
2009/03/16 15:14:50 ossec-syscheckd: WARN: Process locked. Waiting for
permission...
and that's all.
Thanks for any help or pointers - greatly appreciated!
-- tim --
>
>
> Hi Tim,
>
> Were these systems installed as agents or "local" types? Generally you
> get this "locked"
> warning, when analysisd (or ossec-agentd on the agent) can not be
> accessed. Can you
> see if they are running and maybe try restarting it all?
>
> If that still doesn't work, please share you config, version of ossec
> and full log dumps (generally a
> cat /var/ossec/logs/ossec.log | grep -e "ERROR|WARN" should
> be enough).
>
> Thanks,
>
> Thanks,
>
> --
> Daniel B. Cid
> dcid ( at ) ossec.net
>
> On Sun, Mar 15, 2009 at 11:35 AM, Tim Boyer
> <[email protected]> wrote:
> >
> > I've been avoiding SELinux like the plague - it's
> permissive on this one,
> > too:
> >
> > [r...@yamaguchi ~]# getenforce
> > Permissive
> >
> > -- tim --
> >
> >>
> >> This sounds like maybe an SELinux issue. Check
> /var/log/messages for
> >> avc denies related to this. Is it possible SELinux is
> >> permissive/disabled on the other servers not these ones?
> >>
> >> Ben
> >>
> >> On Sat, 2009-03-14 at 20:17 -0400, Tim Boyer wrote:
> >> > I've got a half-dozen RHEL5.3 systems running OSSEC just
> >> fine. And two
> >> > RHEL5.3 systems that never will start up at all.
> >> >
> >> > There's no OSSEC processes running:
> >> >
> >> > [r...@yamaguchi ~]# ps -ef|grep ossec
> >> > root 19348 32346 0 17:55 pts/0 00:00:00 grep ossec
> >> >
> >> > and nothing in the locks directory:
> >> >
> >> > [r...@yamaguchi ~]# ls -la /var/ossec/var/run/
> >> > total 16
> >> > drwxrwx--- 2 root ossec 4096 Mar 14 17:53 .
> >> > dr-xr-x--- 3 root ossec 4096 Mar 14 17:53 ..
> >> >
> >> > but while the other systems start up and check in right
> >> away, these two seem
> >> > to hang forever:
> >> >
> >> > [r...@yamaguchi ~]# /etc/init.d/ossec start
> >> > Starting OSSEC:
> [ OK ]
> >> > [r...@yamaguchi ~]#
> >> >
> >> > watch tail /var/log/ossec.log
> >> >
> >> > 009/03/14 17:57:03 ossec-execd: INFO: Started (pid: 19451).
> >> > 2009/03/14 17:57:07 ossec-syscheckd: INFO: Started (pid: 19463).
> >> > 2009/03/14 17:57:07 ossec-rootcheck: INFO: Started (pid: 19463).
> >> > 2009/03/14 17:57:09 ossec-logcollector(1950): INFO:
> Analyzing file:
> >> > '/var/log/me
> >> > ssages'.
> >> > 2009/03/14 17:57:09 ossec-logcollector(1950): INFO:
> Analyzing file:
> >> > '/var/log/au
> >> > th.log'.
> >> > 2009/03/14 17:57:09 ossec-logcollector(1950): INFO:
> Analyzing file:
> >> > '/var/log/sn
> >> > ort/alert'.
> >> > 2009/03/14 17:57:09 ossec-logcollector: INFO: Started
> (pid: 19459).
> >> > 2009/03/14 17:57:23 ossec-logcollector: WARN: Process
> >> locked. Waiting for
> >> > permis
> >> > sion...
> >> > 2009/03/14 18:05:46 ossec-syscheckd: INFO: Starting
> >> syscheck scan (db).
> >> > 2009/03/14 18:05:46 ossec-syscheckd: WARN: Process locked.
> >> Waiting for
> >> > permissio
> >> > n...
> >> >
> >> > Two hours later, it's still sitting there waiting for permission.
> >> >
> >> > Pointers in the right direction greatly appreciated...
> >> >
> >> >
> >> >
> >> >
> >
> >
