Ok I think the below was caused by the owner/group being changed from root:ossec to root:root. I changed client.keys back to root:ossec, and ossec-remoted fired up and is staying up.
Hopefully this was the issue all along, unfortunately permissions was not something I was checking during the troubleshooting process. On Wed, Jan 5, 2011 at 3:14 PM, Patrick Melvin <[email protected]> wrote: > One last weird update...it was at 1342 agents. Added 1 to bring the > total to 1343 and it was working then the process died. Now when I > restart ossec, I'm back to where I was before where ossec-remoted > isn't running: > > 2011/01/05 16:08:03 ossec-remoted: DEBUG: Starting ... > 2011/01/05 16:08:03 ossec-remoted: INFO: Started (pid: 9045). > 2011/01/05 16:08:03 ossec-remoted: DEBUG: Forking remoted: '0'. > > > $ ps -ef | grep remoted > <blank> > > > On Wed, Jan 5, 2011 at 2:23 PM, Patrick Melvin <[email protected]> > wrote: >> Hi Dan, thanks for the response. I was running into issues that >> remoted would complain about not having an IP address in the remote >> section (with that said, on my CentOS5 setup, the remote section is >> left as default and it runs fine...another issue isolated to Ubuntu?) >> >> ossec-remoted(1501): ERROR: No IP or network allowed in the access >> list for syslog. No reason for running it. Exiting. >> >> Once I added the IP, it went away. I'll get the port removed, was >> grasping at straws when I added it ;) >> After retesting what you stated, I see that the error was specifically >> for the syslog section (even though it went away when it was added to >> either the syslog or secure section). I'll remove syslog from the >> configuration files going forward. >> >> Anyways, on to the Ubuntu issue: >> >> In the logs file, when debug is enabled, this is all I get when it >> fails to start up and bind to 1514/udp: >> (Note: This is what the output looks like when the remote syslog >> section is still included in the ossec.conf. When working correctly, >> it shows 2 remoted processes) >> >> 2011/01/05 14:45:12 ossec-remoted: DEBUG: Starting ... >> 2011/01/05 14:45:12 ossec-remoted: INFO: Started (pid: 4255). >> 2011/01/05 14:45:12 ossec-remoted: DEBUG: Forking remoted: '0'. >> 2011/01/05 14:45:12 ossec-remoted: Remote syslog allowed from: >> '192.168.0.0/16' >> 2011/01/05 14:45:12 ossec-remoted: DEBUG: Forking remoted: '1'. >> >> $ ps -ef | grep remoted >> ossecr 4256 1 0 14:45 ? 00:00:00 /var/ossec/bin/ossec-remoted >> -d >> >> >> $ sudo lsof -P -a -i -c ossec-remoted >> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME >> ossec-rem 4256 ossecr 4u IPv4 19255 0t0 UDP *:514 >> >> >> (This is the output when remote syslog section is removed from ossec.conf) >> >> 2011/01/05 14:56:22 ossec-remoted: DEBUG: Starting ... >> 2011/01/05 14:56:22 ossec-remoted: INFO: Started (pid: 4765). >> 2011/01/05 14:56:22 ossec-remoted: DEBUG: Forking remoted: '0'. >> >> $ ps -ef | grep remoted >> <blank> >> >> $ sudo lsof -P -a -i -c ossec-remoted >> <blank> >> >> >> When I delete an agent from client.keys and (in this case) go back >> down to 1015, these are the outputs: >> >> 2011/01/05 15:03:16 ossec-remoted: DEBUG: Starting ... >> 2011/01/05 15:03:16 ossec-remoted: INFO: Started (pid: 5350). >> 2011/01/05 15:03:16 ossec-remoted: DEBUG: Forking remoted: '0'. >> >> >> $ ps -ef | grep remoted >> ossecr 5351 1 0 15:03 ? 00:00:00 /var/ossec/bin/ossec-remoted >> -d >> >> >> $ sudo lsof -P -a -i -c ossec-remoted >> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME >> ossec-rem 5351 ossecr 4u IPv4 23036 0t0 UDP *:1514 >> >> >> Now I'm really confused. I went through the tests above, and put the >> original client.keys file in place (1300+ agents), and it's working >> now. Just replaced the original configuration file settings (as stated >> in the start of the email) and it still works. >> >> I'm really at a loss... >> >> Thanks in advance, >> Patrick >> >> >> On Wed, Jan 5, 2011 at 1:06 PM, dan <[email protected]> wrote: >>> Hi Patrick, >>> >>> On Tue, Jan 04, 2011 at 03:01:30PM -0600, Patrick Melvin wrote: >>>> Hello, >>>> >>>> I believe I'm having an issue with ossec-remoted binding to port 1514/udp. >>>> >>>> >>>> ** Basic Configuration/OS Information ** >>>> >>>> OS: Linux <server_name> 2.6.32-27-server #49-Ubuntu SMP Thu Dec 2 >>>> 02:05:21 UTC 2010 x86_64 GNU/Linux >>>> ossec: 2.5.1 >>>> >>>> make setmaxagents >>>> Specify maximum number of agents: 4096 >>>> ./install.sh >>>> >>>> /etc/security/limits.conf have the following entries (and server was >>>> restarted afterwards): >>>> * soft nofile 4096 >>>> * hard nofile 4096 >>>> >>>> >>>> ** Issue Description ** >>>> >>>> As stated, it appears that while ossec-remoted is running, it is not >>>> binding to 1514/udp. I've been troubleshooting this and have not been >>>> able to get any helpful information out of the logs, debug mode, or >>>> stracing the process. >>>> >>>> >>>> When I run a tcpdump, I see the agents trying to connect to the server >>>> on 1514/udp, but the server responds back with the following: >>>> >>>> ICMP <server_ip> udp port 1514 unreachable, length 109 >>>> >>>> Which indicates there's no process listening on the port. netstat does >>>> not show 1514 in use. >>>> >>>> >>>> I verified 1514/udp connectivity by utilizing netcat (nc) and >>>> successfully connected to the server on 1514/udp. >>>> >>>> >>>> strace show's the following for ossec-remoted: >>>> recvfrom(4, >>>> >>>> >>>> I have the following in ossec.conf with relation to remoted: >>>> <remote> >>>> <connection>syslog</connection> >>>> </remote> >>>> >>> >>> Are you using OSSEC to collect syslog messages? If not, you don't need >>> this. If you are using it, put it after the secure connection type, and >>> add allowed-ips. >>> >>>> <remote> >>>> <connection>secure</connection> >>>> <allowed-ips>192.168.0.0/16</allowed-ips> >>>> <port>1514</port> >>>> <local_ip>(server_ip_address)</local_ip> >>>> </remote> >>>> >>> >>> You shouldn't need the allowed-ips in the secure section, I think it's >>> only for syslog. You also don't need to put the port in there if you're >>> using the default. >>> >>>> >>>> If there's any addtional information that might be helpful, please let me >>>> know. >>>> >>>> I've been researching using google but have found no resolutions to >>>> this specific problem. Any ideas? >>>> >>>> Thanks in advance, >>>> Patrick >>> >>> Look for entries for ossec-remoted in /var/ossec/logs/ossec.log. You may >>> need to enable debugging for that daemon (run it with the -d flag) to >>> get something more useful. >>> >>> I'm currently using both syslog and secure on one of my servers, so it >>> is possible. I haven't tried it with so many clients though. >>> >>> dan >>> >> >
