I'm running OSSEC 2.4.1 at work, and I've seen some cases where
centralized file distribution isn't working when a new agent is
connected to the server. I'll outline the problem and then a
workaround which seems to force syncing to start, but I think there is
a bug hiding in here somewhere.
When the problem occurs, I install the agent and it connects to the
server. I can see events coming over into the server logs, but files
do not sync from the server's ossec/etc/shared directory back to the
agent. I've tried restarting the server, restarting the client, even
reinstalling the client, but nothing seemed to force syncing to start.
If you want to know what I tried to make it work, look at at the notes
below, otherwise skip down to 'THE WORKAROUND'.
NOTES:
* The agents I've had problems with are Solaris 8/9 boxes. The
install was done with the 'precompiled' agent install strategy - I
compiled the 2.4.1 source on a Solaris 10 box using the 'setagent' and
USER_BINARYINSTALL options, and then tar'd the result up and used that
to install on older Sol 8/9 boxes.
* I've not seen the problem on all Sol 8/9 boxes; in fact, syncing has
worked correctly on almost all boxes.
* manage_agents -i on the server showed the clients were connected,
but did not have the md5 sum showing that they were syncing.
* On the same day these problems occurred, I was doing agent installs
on Redhat Linux boxes with no problem in syncing.
* The latest two boxes that had a problem syncing could not initially
connect to the ossec server because a firewall was blocking UDP 1514.
I thought perhaps something critical to that first sync might occur
during that first connection to the server, so I reinstalled the agent
on one of the boxes and reimported the old agent key from the server.
This resulted in 'duplicate counter' errors on the server, so I
deleted the agent key from the server and imported the new key into
the client. This did not fix the sync problem.
* Oddly, on these last two problem boxes, one of them synced up
without any interference about 18 hours after the initial install.
Its sister box did not. To be fair, these boxes were installed by two
different people, so there might be some other factor in the procedure
to cover that. Still, the box that synced up on it's own took 18
hours to do so, and there was no admin-initiated change to the shared
files on the server end.
In trying to figure out why that first sync wasn't working, I spent
some time staring at src/remoted/manager.c, which appears to be in
charge of file sync from the server side. There's a comment at line
454 that gave me pause:
/* New agents only have merged.mg. */
That doesn't actually appear to be true - under 2.4.1, new agent
installs have a number of files in ossec/etc/shared, including files
such as cis_debian_linux_rcl.txt, rootkit_files.txt etc - but I don't
see a merged.mg file in there. That gave me a clue for something to
try.
THE WORKAROUND
I went to my problematic Solaris client, did a 'touch /var/ossec/etc/
shared/merged.mg' and restarted the agent. Within a few minutes,
sure enough, that empty merged.mg was replaced, and all the other
shared files came in as well.
My best guess at this point is that the problem was indeed because I
had a firewall blocking the agents from connecting to the server on
the first run. When I tried reinstalling the client to attempt to
prove that point, I didn't do it cleanly: I didn't delete the client
from the server first before trying to reinstall the client. I have
the feeling syncing would have worked if I'd first deleted the agent
info from the server, then re-created it, and then reinstalled the
agent on the client box.
-- Paul Holbrook
