[ossec-list] Re: Modification to Correct IP Connectivity Issues on BSD Servers
Brent, Yes, I am familiar with PR 1259. I tried that first, but it did not work for me. The problem for me was that FreeBSD defines AI_V4MAPPED in . As the code is written in net_os.c, if you have this defined on your operating system then it uses AF_INET6 with the AI_V4MAPPED flag. The getaddrinfo() request will retrieve all IPv6 addresses on your system, including IPv4 addresses remapped in IPv6 format. Unfortunately, as soon as it finds one valid address, it only uses that one address. I was not using IPv6, but FreeBSD automatically generated an IPv6 address for me based on my MAC address (useless for my purposes, but conforms to RFC 2462). As such, OSSEC would always find the bogus IPv6 address and use it for port 1514. The IBM article I linked to in my previous post goes into a lot more detail on AF_INET6 with AI_V4MAPPED versus AF_UNSPEC. A shortcut would have been to simply modify the code to use AF_UNSPEC instead of AF_INET6, but then because it only returns the first thing it finds it is still a crapshoot. I could also force it to read IPv4 addresses only by using AF_INET, but then down the road, if I found an IPv6 client, I would have to modify the source code again. A better approach was to use ALL of the configured (AI_ADDRCONFIG flag) addresses available on the system. So that is what I did. It can now support both IPv4 and IPv6 on all configured interfaces. This will be useful with pfSense eventually, although I have not tested it on that platform yet (pfSense is FreeBSD with kernel tweaks). If you are having connectivity issues, this is what I would recommend - download the modified OSEC 2.9.3 distribution from my server (I call it 2.9.3.1 with the modifications). Because you are running Linux, it will attempt to use the original AF_INET6 and AI_V4MAPPED behavior, but I put a define in the Makefile you can enable to override that and force it to use AF_UNSPEC instead. It will find and use all of the available configured addresses on your system. Simply follow this procedure: 1. Download the modified distribution (11 files changed) from https://networkalarmcorp.com/dnld/ossec-hids-2.9.3.1.tgz 2. Unpack the tarball in a directory on your system using "tar xvzf ossec-hids-2.9.3.1.tgz" (the distribution will unpack inside a subdirectory named ossec-hids-2.9.3) 3. Change into the directory "cd ossec-hids-2.9.3" 4. Edit the src/Makefile and remove the hash mark in front of #DEFINES+=NOV4MAP on line 47 and save your changes (this replaces AF_INET6 with AF_UNSPEC). 5. Run the install.sh script in the top directory and answer the questions (you want a "server" install). Make sure you are root when you run install.sh. If you do not have "." in your path, you will have to use "./install.sh" to start it., It should compile, link, and install without any problems. 6. Configure OSSEC in your chosen installation directory. Note that you need to configure at least one client before you can run OSSEC. When OSSEC is running, use "netstat -an" to view your listening sockets - you will see udp4 and udp6 bound to the same port 1514. Shoot me an email if you encounter any errors. Best, Dave Stoddard Network Alarm Corporation https://networkalarmcorp.com https://redgravity.net Office: 301-850-0668 x101 Email: dgs at networkalarmcorp dot com On Thursday, May 3, 2018 at 12:16:44 PM UTC-4, Brent Morris wrote: > > Dave - first, thank you very much for such a detailed explanation and > submitting this back to OSSEC for the community! > > I just attempted an upgrade from 2.8.3. to 2.9.3 and had, what sounds > like, similar issues. Our CentOS 6.5 server didn't have ipv6 support > installed or enabled. I recompiled with a workaround noted in this github > issue - https://github.com/ossec/ossec-hids/pull/1259 - and was able to > get syslog clients working, but my agents would still not connect. > > My agent logs contained the same errors... trying to connnect. Connected, > and then waiting for reply. I ended up restoring my old server. > > Have you looked at the pull request I linked above? Are the issues > related? > > Thanks again! > > On Wednesday, May 2, 2018 at 6:25:18 PM UTC-7, Dave Stoddard wrote: >> >> I just submitted a pull request to correct a connectivity issue I was >> having with OSSEC servers running on FreeBSD 11.1. I mentioned this on the >> email list a few weeks ago, but it took some time to do the work to >> identify and correct the problem. I encountered a significant issue in >> OSSEC 2.9.3 and the current OSSEC beta (GitHub repository) release that >> affects the ability of OSSEC clients to be able to connect to FreeBSD >> servers. The issue did not exist for OSSEC 2.8.3 possibly due to the fact >> that it was IPv4 only. This connectivity issue probably affects all current >> BSD derivatives and possibly some of the more obscure releases of Linux as >> well. I have identified and resolved the issue in OSSEC by rewriting some >> of
[ossec-list] Re: Modification to Correct IP Connectivity Issues on BSD Servers
Dave - first, thank you very much for such a detailed explanation and submitting this back to OSSEC for the community! I just attempted an upgrade from 2.8.3. to 2.9.3 and had, what sounds like, similar issues. Our CentOS 6.5 server didn't have ipv6 support installed or enabled. I recompiled with a workaround noted in this github issue - https://github.com/ossec/ossec-hids/pull/1259 - and was able to get syslog clients working, but my agents would still not connect. My agent logs contained the same errors... trying to connnect. Connected, and then waiting for reply. I ended up restoring my old server. Have you looked at the pull request I linked above? Are the issues related? Thanks again! On Wednesday, May 2, 2018 at 6:25:18 PM UTC-7, Dave Stoddard wrote: > > I just submitted a pull request to correct a connectivity issue I was > having with OSSEC servers running on FreeBSD 11.1. I mentioned this on the > email list a few weeks ago, but it took some time to do the work to > identify and correct the problem. I encountered a significant issue in > OSSEC 2.9.3 and the current OSSEC beta (GitHub repository) release that > affects the ability of OSSEC clients to be able to connect to FreeBSD > servers. The issue did not exist for OSSEC 2.8.3 possibly due to the fact > that it was IPv4 only. This connectivity issue probably affects all current > BSD derivatives and possibly some of the more obscure releases of Linux as > well. I have identified and resolved the issue in OSSEC by rewriting some > of the networking code, however, before I describe the solution, I wanted > to provide some background information on the problem first. > > *Problem Symptoms* > > The server is running FreeBSD 11.1 with all of the current updates. This > is a standard OSSEC server installation with all standard options enabled > and no bells and whistles (no MySQL and no MQ). After correcting for some > BSD-specific issues for header files and libraries, OSSEC installs normally > on the server with the install.sh script and the compilation completes > without any fatal errors (GCC does generate a few warnings on > analysisd/decoders/syscheck.c and analysisd/decoders/syscheck-test.c, but > these are not show stoppers). The server has an assigned IP address of > 192.168.1.80 (IPv4 private IP space) and all of the clients it associates > with are on the local 192.168.1.0/24 network. > > The OSSEC 2.9.3 client software is installed on each Windows workstation > and valid client keys are obtained for each client system. Everything > appears fine, except that there are no alerts logged on the server for any > of the configured clients. In short, the problem manifests itself as an > inability for the client systems to connect to the OSSEC server. In the > ossec.log file on each client there are messages like this that repeat over > and over: > > 2018/03/08 12:11:06 ossec-agentd: INFO: Trying to connect to server > 192.168.1.80, port 1514. > 2018/03/08 12:11:06 INFO: Connected to 192.168.1.80 at address 192.168. > 1.80:1514, port 1514 > 2018/03/08 12:11:27 ossec-agentd(4101): WARN: Waiting for server reply ( > not started). Tried: '192.168.1.80'. > > > 2018/03/08 12:12:41 ossec-agentd: INFO: Trying to connect to server > 192.168.1.80, port 1514. > 2018/03/08 12:12:41 INFO: Connected to 192.168.1.80 at address 192.168. > 1.80:1514, port 1514 > 2018/03/08 12:13:02 ossec-agentd(4101): WARN: Waiting for server reply ( > not started). Tried: '192.168.1.80'. > > > Because the clients use UDP, and UDP is a connectionless protocol, there > is no protocol handshake to ensure a connection has actually been > established like there would be with TCP. As long as the sendto() function > returns a positive integer (the number of bytes sent), the call is deemed > successful. This is true even when packets are not received on the > destination host. To compensate for this, the server is supposed to return > a control message to indicate the packet was received. Because the client > never receives this confirmation, a warning is generated in the client log > "Waiting for server reply (not started)". > > On the server, remoted starts fine. It displays the maximum number of > agents allowed (2048) and logs the fact that it successfully read the > authentication keys file. However, the server logs hint that there is an > issue when remoted starts because each client in the client.keys file > results in a message stating "Assigning counter for agent Workstation01: > '0:0'", or "Assigning sender counter: 0:0". These messages are generated > when a client is defined in the etc/client.keys file, but no initial > connection has ever occurred. This is normal for a first time start of > OSSEC, but if this happens every time you restart OSSEC for every client > defined in the system, your clients are not communicating. > > To identify whether remoted was listening on UDP port 1514, I used the > "netstat -an"
