main mode produces comm losses
Hi, I am running OpenBSD 4.0 with carp+isakmpd+sasyncd+pf on 166MHz Pentium boards. Everything is working well. There are 6 locations, all clustered (2 redundant firewalls). When I fail one cluster the other one takes over with some packet loss. I see the carp is doing its thing. After the failover the new master starts using the SAs from its partner until it establishes its own. For this delta time everything is stable. 10-15 seconds later it starts establishing the main mode keys all at the same time and I can see for 7-9 seconds the CPU utilized 100%. During that time the communication is down again. After this new SAs are established everything goes back to normal. Since I already have the SAs, it is really no need to run the CPU demanding D-H to a point where the CPU is fully used and the packet forwarding is affected. Is there a way to have the CPU demanding main mode done so that the packet forwarding is not affected? I tried to run nice isakmpd but I still get the timeouts when the new IKE and IPSEC SAs are established. I also tried renice-ing process id 13 (crypto) with value -20, but I still get the same result. Thanks, Catalin - All new Yahoo! Mail - - Get a sneak peak at messages with a handy reading pane.
Security associations and SA_FLAG_REPLACED
Hi, I have GW1 and GW2 redundant firewalls (isakmpd+pf+carp+sasyncd) Is there a way to see which security associations are marked as replaced on the backup GW? ipsecctl -s all -v -v shows a lot but it does not seem to show that. On the master (let's say GW1) echo S /var/run/isakmpd.fifo then vi isakmpd.report shows the flags, but I'm interested in the SAs from the backup GW2 which were created by sasyncd. Basically after the old SAs soft time expired and new SAs are created I want to see the old ones marked as SA_FLAG_REPLACED and the new ones SA_FLAG_ALIVE on the backup firewall. Is there a way. Thank you, Catalin - Ask a question on any topic and get answers from real people. Go to Yahoo! Answers.
Carp question and security association mismatch
Hi, I have two firewalls using isakmpd+pf+sasyncd+carp (OpenBSD 4.0) preempt is set to 0 At one end (machine names MAED11 and MAED12) carp0 on external has 172.16.140.145 255.255.255.0 advbase 0 advskew 128 pass gijane vhid 1 carp1 on external has 172.16.160.33 255.255.255.224 advbase 0 advskew 128 vhid 2 On the other end (machine names MAED21 and MAED22) carp0 on external has 172.16.140.148 255.255.255.0 advbase 0 advskew 128 pass gijane vhid 1 carp1 on external has 172.16.160.33 255.255.255.224 advbase 0 advskew 128 vhid 2 I do not want to favor any machine in the cluster. The master stays master until it fails (same thing happens if advbase is set to 1, not 0). Here is an interesting scenario that I observed: 1. I reboot the first MAED11 and MAED21 (first machines in the two clusters) then 20 seconds later I reboot MAED12 and 22 2. MAED11 and MAED21 come back up first and they establish SAs with SPIs spi11-21 and spi21-11. Packets go through. 3. Their carp interface advertise advbase 0 and advskew 240 (~950ms) for about 45 packets then they start advertising every 500 ms (advbase 0 and advskew 128) 4. MAED12 and MAED22 come back before the 45 packets are sent. They become master as they advertise directly with advbase 0 and advskew 128. If I delay the restart of the MAED12 and MAED22 so that the first 45 packets are sent and the new adv rate is advbase 0 advskew 128 (500 ms) the switchover does not occur. Another interesting thing is that in the packets sent on the ext interface, before the takeover I see Auth Type: Simple Text Authentication (1) but after the takeover I see No authentication (0). Since I use a pass on external should I not have authentication on the external? 5. The new masters (MAED12 and MAED 22) establish new SAs with SPIs spi12-22 and spi22-12. But before that they got spi11-12 and spi12-11 using sasyncd 6. I notice that they try to communicate using the new spi12-22 out and the older spi21-11 in. Basically it mixes the SA pairs. The packets stop going through. I believe once new SAs are established the old ones are marked replaced so what happens it makes sense. But why it decides to use the old SPI I do not know. Any ideas? Is there a way to ensure the first firewall that comes up stays master? Why would the SPI mismatch occur? Is sasyncd setting the replaced flag? Thanks for the info. Let me know if you need any info. The firewalls have identical hardware. Regards, Catalin - Instant message from any web browser! Try the new Yahoo! Canada Messenger for the Web BETA
Carp creates a wide route if netmask is not used when carp is configured
Hi, I am using isakmpd+pf+sasyncd+carp to set a VPN network (OpenBSD 4.0) Recently had a problem with carp... Basically ifconfig carp0 inet 172.16.140.1 255.255.255.0 advbase 1 ... versus ifconfig carp0 inet 172.16.140.1 netmask 255.255.255.0 advbase 1 ... The simple fact that I not put the word netmask creates a route 172.16/16 on carp0 interface which was causing a lot of trouble. Same thing with /etc/hostname.carp0 if the word netmask is not there. On the first firewall I have: - external 172.16.140.1/24 - internal 172.16.160.1/24 On the second firewall I have: - external 172.16.140.2/24 - internal 172.16.161.1/24 Problem description: 1. When the firewall is booting, it first configures the IP addresses (/etc/netstart). 2. The moment carp is configured I can see a new route 172.16/16 on interface carp0 (using route -n show). 3. A fraction of a second later a PC behind the first firewall sends a packet from the 172.16.160.2 to 172.16.161.2 (PC behind the second firewall) 4. An entry is added to the routing table assigning the destination IP to carp0 interface (external) since the packet matches the 172.16/16 route. 5. A seconds later the route is added route add -net 172.16.161.0/24 172.16.140.2 but the existing route 172.16.161.2 on interface carp0 is used (which makes sense because the narrower route has preference) Now if I put the work netmask in the hostname.carp0 (external) and hostname.carp1 (internal) this route is not created. A packet coming from a PC behind the firewall would be dropped as there is no route for it. A second later when the route is added the packets will be routed properly. The man carp page is correct but the other documentation www.openbsd.org/faq/pf/carp.html www.countersiege.com/doc/pfsync-carp www.lugbe.ch/action/reports/BSDCluster.pdf (German) http://www.nycbsdcon.org/2005/files/jdixon_firewall_failover.pdf in my opinion do not configure carp properly. What is nasty about this is that carp is configured and ifconfig carp shows everything is OK. If netmask is required than an error should be displayed. If it's optional, then in both cases the result should be the same. I remember a similar problem with carp where the command line parameter order did matter: http://www.webservertalk.com/archive248-2007-3-1848404.html Regards, Catalin - Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail
/usr/ports/net/ntp and VPN (improvement idea and solution)
Hello,
This is used in a VPN network to bind the internal IP address and allow ntpd
running of firewalls to get the time from a time source in a different
protected subnet.
I've changed two files
ntp_io.c
cmd_args.c
in /usr/ports/net/ntp
See the diffs below. Hope they can be added to the ports tree.
Regards,
./catalin
--- ntp_io.c_orig Thu Jul 5 11:42:32 2007
+++ ntp_io.c Thu Jul 5 11:39:47 2007
@@ -112,6 +112,7 @@
int nwilds;/* Total number of wildcard intefaces */
int wildipv4 = -1; /* Index into inter_list for IPv4 wildcard */
int wildipv6 = -1; /* Index into inter_list for IPv6 wildcard */
+extern char* szBindIPAddr; /* SDDEV */
#ifdef REFCLOCK
/*
@@ -1959,6 +1960,24 @@
rtn = getsockname(s, (struct sockaddr *)saddr, saddrlen);
closesocket(s);
+
+/*
+ * SDDEV
+ * If there is an interface specified bind that one instead of
+ * using all the interfaces
+ */
+if ( NULL != szBindIPAddr )
+{
+for (i = 0; i ninterfaces; i++)
+{
+if ( 0 == strcmp( stoa(inter_list[i].sin),
szBindIPAddr ) )
+{
+return (inter_list[i]);
+}
+}
+return ANY_INTERFACE_CHOOSE(addr);
+}
+
#ifndef SYS_WINNT
if (rtn 0)
#else
--- cmd_args.c_orig Thu Jul 5 11:16:02 2007
+++ cmd_args.c Thu Jul 5 11:36:03 2007
@@ -14,16 +14,31 @@
#endif /* SIM */
/*
+ * SDDEV
+ * These three headers are included in order to use inet_pton to verify if
+ * the IP address received as command-line parameter is valid
+ */
+#include sys/socket.h
+#include netinet/in.h
+#include arpa/inet.h
+
+/*
* Definitions of things either imported from or exported to outside
*/
extern char const *progname;
int listen_to_virtual_ips = 1;
+/*
+ * SDDEV
+ * Used to store the IP address received as command-line param.
+ */
+char *szBindIPAddr = NULL;
+
#ifdef SYS_WINNT
extern BOOL NoWinService;
#endif
-static const char *ntp_options =
aAbB:c:C:dD:f:gi:k:l:LmnNO:p:P:qr:s:S:t:T:W:u:v:V:xY:Z:-:;
+static const char *ntp_options =
aAbB:c:C:dD:f:gi:I:k:l:LmnNO:p:P:qr:s:S:t:T:W:u:v:V:xY:Z:-:;
#ifdef HAVE_NETINFO
extern int check_netinfo;
@@ -84,6 +99,28 @@
++errflg;
break;
#endif
+/*
+ * SDDEV
+ * Specify the internal interface
+ */
+case 'I':
+{
+// used here to test if the given IP is correct
+struct sockaddr_in saTmp;
+
+szBindIPAddr = (char*)malloc( 32*sizeof(char) );
+strcpy(szBindIPAddr, ntp_optarg);
+
+/* test if the given IP is correct (a.b.c.d),
with a,b,c,d in [0,255] */
+if ( 1 != inet_pton(AF_INET, szBindIPAddr,
saTmp.sin_addr) )
+{
+msyslog(LOG_ERR, the given interface
is in wrong format);
+fprintf(stderr, the given interface
is in wrong format);
+free(szBindIPAddr);
+szBindIPAddr = NULL;
+}
+}
+break;
case 'L':
listen_to_virtual_ips = 0;
break;
@@ -158,6 +195,7 @@
(void) fprintf(stderr, \t\t[ -f freq_file ] [ -k key_file ] [ -l log_file
]\n);
(void) fprintf(stderr, \t\t[ -p pid_file ] [ -r broad_delay ] [ -s statdir
]\n);
(void) fprintf(stderr, \t\t[ -t trust_key ] [ -v sys_var ] [ -V
default_sysvar ]\n);
+(void) fprintf(stderr, \t\t[ -I ip_addr ]\n); //SDDEV
#if defined(HAVE_SCHED_SETSCHEDULER)
(void) fprintf(stderr, \t\t[ -P fixed_process_priority ]\n);
#endif
@@ -264,6 +302,11 @@
#else
errflg++;
#endif
+
+// SDDEV
+case 'I': /* already done at prescan */
+break;
+
case 'k':
getauthkeys(ntp_optarg);
break;
@@ -414,6 +457,7 @@
(void) fprintf(stderr, \t\t[ -f freq_file ] [ -k key_file ] [ -l log_file
]\n);
(void) fprintf(stderr, \t\t[ -p pid_file ] [ -r broad_delay ] [ -s statdir
]\n);
(void) fprintf(stderr, \t\t[ -t trust_key ] [ -v sys_var ] [ -V
default_sysvar ]\n);
+(void) fprintf(stderr, \t\t[ -I ip_addr ]\n);
#if defined(HAVE_SCHED_SETSCHEDULER)
(void) fprintf(stderr, \t\t[ -P fixed_process_priority ]\n);
#endif
-
Be smarter than spam. See how smart SpamGuard is at giving junk email the boot
with the All-new Yahoo! Mail
OpenBSD 4.0: isakmpd and immediate use of crls (without isakmpd restart)
Hello, I was wondering what is the best way to immediately use a newly received crl that contains a revoked certificate... Basically if I have 3 firewalls and one of them is compromised I will push a new crl on the 2 uncorrupted firewalls. The thing is that (even when I send them a HUP signal) isakmpd only uses the CRL when the next main-mode is performed. One thing I was thinking is to remove all IPSEC SAs echo T /var/run/isakmpd.fifo Then find a way to remove all IKE SAs echo t main * /var/run/isakmpd.fifo -- something like this...I'm don't know yet how I could do that. However, it is a bit inconvenient because the connection between the two good firewalls is broken as well. I found this: http://archives.neohapsis.com/archives/openbsd/2002-10/1327.html but it doesn't help much in this case... I was looking through the isakmpd code and I could force this by changing sa.c file, sa_reinit function to remove all SAs not just phase 2 SAs on SIGHUP when Renegotiate-on-HUP is set. Again that would break all tunnels not just the one to the compromised firewall. But there must be a better way to do this. Thanks, ./catalin - Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail
Re: isakmpd on OpenBSD 3.7 and OpenBSD 4.0
Thanks to Stuart Henderson. On 2007/06/25 11:35, catalin visinescu wrote: I see that OpenBSD 3.7 isakmpd and OpenBSD 4.0 isakmpd do not establish security associations. try -T (disable nat-t) on the 4.0 side. If it works, can you post back to misc@ to get it in the archives please. - Ask a question on any topic and get answers from real people. Go to Yahoo! Answers. Received: from [207.34.229.126] by web39710.mail.mud.yahoo.com via HTTP; Mon, 25 Jun 2007 11:35:19 EDT Date: Mon, 25 Jun 2007 11:35:19 -0400 (EDT) From: catalin visinescu [EMAIL PROTECTED] Subject: isakmpd on OpenBSD 3.7 and OpenBSD 4.0 To: misc@openbsd.org MIME-Version: 1.0 X-Security: message sanitized on shear.ucar.edu See http://www.impsec.org/email-tools/sanitizer-intro.html for details. $Revision: 1.147 $Date: 2004-10-02 11:16:26-07 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit Content-Length: 516 X-Converted-To-Plain-Text: from multipart/alternative by demime 1.01d X-Converted-To-Plain-Text: Alternative section used was text/plain Hello, I see that OpenBSD 3.7 isakmpd and OpenBSD 4.0 isakmpd do not establish security associations. I get an INVALID-PAYLOAD-TYPE message. isakmpd 3.7 does not seem to understand payload RESERVED. Is there a way I can run isakmpd 4.0 downgraded or any other way to get the two of them to work together? Thank you, ./catalin - Ask a question on any topic and get answers from real people. Go to Yahoo! Answers.
isakmpd on OpenBSD 3.7 and OpenBSD 4.0
Hello, I see that OpenBSD 3.7 isakmpd and OpenBSD 4.0 isakmpd do not establish security associations. I get an INVALID-PAYLOAD-TYPE message. isakmpd 3.7 does not seem to understand payload RESERVED. Is there a way I can run isakmpd 4.0 downgraded or any other way to get the two of them to work together? Thank you, ./catalin - Ask a question on any topic and get answers from real people. Go to Yahoo! Answers.
Re: Pinging redundant firewall problem (isakmpd+pf+pfsync+sasyncd+carp)
catalin visinescu [EMAIL PROTECTED] wrote: Hello, Intro: I am using isakmpd+sasyncd+carp+pf+pfsync to have a redundant firewall setup (OpenBSD 4.0). I have two firewall that carp-advertise at the same rate, and not preempt each other. This works fine. isakmpd is using x509 certificates to establish SAs. This is working fine. sasyncd is running on both and they share the SAs properly. pfsync has been configured and it is working well. I have the following setup (netmask is /24 everywhere): Redundant end FW1: Ext IP: 172.16.140.2 (static) Int IP: 172.16.36.2 (static) FW2: Ext IP: 172.16.140.3 (static) Int IP: 172.16.36.3 (static) FW1 and FW2 shared IP addresses (carp) Ext IP: 172.16.140.1 Int IP: 172.16.36.1 Non-redundant end: Ext IP: 172.16.142.1 (static) Int IP: 172.16.40.1 (static) Problem: Assume the gateway that has static IP 172.16.36.2 is the master. I ping from 172.16.40.1 to 172.16.36.1 (or 172.16.36.2) and the ping goes through. The moment I ping the backup (ping -c 1 -I 172.16.40.1 172.16.36.3) I get a reply, but I can no longer ping 172.16.36.2. Now I can only ping the second gateway (goes in through the master, goes out through the backup). Everything goes back to normal (I can ping 172.16.36.2) the moment a new quick mode is finished and new SAs are established. Question: Why is this happening? I would like to have remote access to the backup gateway, for instance for live status polling (that's why I have the static IP addresses), or sync NTP time on firewalls (time source over secure tunnel). I don't mind if when I ping 172.16.36.3 the packet goes in through the first gateway and goes out through the second (because the flows are already set). I just don't want to block the communication on messages to the backup gateway. Can anyone help with this issue? ./catalin Hello, I understand now why this happens... it is a problem with the packet filter not updating the sequence numbers correctly. When I ping the master firewall the sequence numbers used are the same for both directions (SPIs)... (100,100) let's say. When I ping the backup, the request goes through master and goes out through the backup with sequence numbers (101, and 16485). That is normal behaviour and is documented here http://members.iinet.net.au/~nathanael/OpenBSD/sasyncd.html (section 1.5) Let's say 172.16.36.2 is the master... From the non-redundant end: ping -c 1 172.16.40.1 172.16.36.2 OK seq:100 request, 100 reply (sniffing on pfsync0 of the master firewall shows an updated seq # being sent to the backup firewall for that SPI) ping -c 1 172.16.40.1 172.16.36.3 OK seq:101 request, 101+16384=16485 reply (sniffing on pfsync0 of the master firewall shows an update being sent to the backup firewall) (sniffing on pfsync0 of the backup firewall shows an update being sent to the master firewall) NOTE THAT THE MASTER USES THE UPDATE FROM BACKUP. ping -c 1 172.16.40.1 172.16.36.2 OK seq:102 request, 16485+16384= 32869 reply (sniffing on pfsync0 of the master firewall shows an update being sent to the backup firewall) ping -c 1 172.16.40.1 172.16.36.2 OK seq:103 request, 16485+16384= 32870 reply (sniffing on pfsync0 of the master firewall shows an update being sent to the backup firewall) This part is clear... whenever a firewall has something to send, it is adding 1 to the previous sequence # if it sent the last message and it adds 16384 if the sequence # it has was received using pfsync from the other firewall. That I see in if_pfsync.c However if I change the test just a little bit... ping -i .1 172.16.40.1 172.16.36.2 OK seq:100 request, 100 reply, and so on (sniffing on pfsync0 of the master firewall shows an update being sent to the backup firewall) and at some point: ping -c 1 172.16.40.1 172.16.36.3 OK seq:101 request, 101+16384=16485 reply (sniffing on pfsync0 of the backup firewall shows an update being sent to the master firewall) The communication to 172.16.36.2 stops as the master does not get the update of the seq # for that SPI. The update is sent though (sniffing pfsync). As soon as a new SA is established everything (obviously) goes back to normal. THE MASTER DOES NOT USE THE UPDATE FROM THE BACKUP. This is quite bizarre that sending this one packet stops the traffic to 172.16.36.2. I would expect some packets to be lost until the master receives the update from the backup though (up to a second). I will take a look at if_pfsync.c and check why this happens. Hope this helps. ./catalin - Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail
Pinging redundant firewall problem (isakmpd+pf+pfsync+sasyncd+carp)
Hello, Intro: I am using isakmpd+sasyncd+carp+pf+pfsync to have a redundant firewall setup (OpenBSD 4.0). I have two firewall that carp-advertise at the same rate, and not preempt each other. This works fine. isakmpd is using x509 certificates to establish SAs. This is working fine. sasyncd is running on both and they share the SAs properly. pfsync has been configured and it is working well. I have the following setup (netmask is /24 everywhere): Redundant end FW1: Ext IP: 172.16.140.2 (static) Int IP: 172.16.36.2 (static) FW2: Ext IP: 172.16.140.3 (static) Int IP: 172.16.36.3 (static) FW1 and FW2 shared IP addresses (carp) Ext IP: 172.16.140.1 Int IP: 172.16.36.1 Non-redundant end: Ext IP: 172.16.142.1 (static) Int IP: 172.16.40.1 (static) Problem: Assume the gateway that has static IP 172.16.36.2 is the master. I ping from 172.16.40.1 to 172.16.36.1 (or 172.16.36.2) and the ping goes through. The moment I ping the backup (ping -c 1 -I 172.16.40.1 172.16.36.3) I get a reply, but I can no longer ping 172.16.36.2. Now I can only ping the second gateway (goes in through the master, goes out through the backup). Everything goes back to normal (I can ping 172.16.36.2) the moment a new quick mode is finished and new SAs are established. Question: Why is this happening? I would like to have remote access to the backup gateway, for instance for live status polling (that's why I have the static IP addresses), or sync NTP time on firewalls (time source over secure tunnel). I don't mind if when I ping 172.16.36.3 the packet goes in through the first gateway and goes out through the second (because the flows are already set). I just don't want to block the communication on messages to the backup gateway. Can anyone help with this issue? ./catalin - Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail
Pinging redundant firewall problem (isakmpd+pf+pfsync+sasyncd+carp)
Hello, Intro: I am using isakmpd+sasyncd+carp+pf+pfsync to have a redundant firewall setup (OpenBSD 4.0). I have two firewall that carp-advertise at the same rate, and not preempt each other. Basically I don't care which firewall is master and which is backup. This works fine. isakmpd is using x509 certificates to establish SAs. This is working fine. sasyncd is running on both and they share the SAs properly. pfsync has been configured and it is working well. I have the following setup (netmask is /24 everywhere): Redundant end FW1: Ext IP: 172.16.140.2 (static) Int IP: 172.16.36.2 (static) FW2: Ext IP: 172.16.140.3 (static) Int IP: 172.16.36.3 (static) FW1 and FW2 shared IP addresses (carp) Ext IP: 172.16.140.1 Int IP: 172.16.36.1 Non-redundant end: Ext IP: 172.16.142.1 (static) Int IP: 172.16.40.1 (static) Problem: Assume the gateway that has static IP 172.16.36.2 is the master. I ping from 172.16.40.1 to 172.16.36.1 (or 172.16.36.2) and the ping goes through. The moment I ping the backup (ping -c 1 -I 172.16.40.1 172.16.36.3) I get a reply, but I can no longer ping 172.16.36.2. Now I can only ping the second gateway (goes in through the master, goes out through the backup). Everything goes back to normal (I can ping 172.16.36.2) the moment a new quick mode is finished and new SAs are established. Question: Why is this happening? I would like to have remote access to the backup gateway, for instance for live status polling (that's why I have the static IP addresses), or sync NTP time on firewalls (time source over secure tunnel). I don't mind if when I ping 172.16.36.3 the packet goes in through the first gateway and goes out through the second (because the flows are already set). I just don't want to block the communication on messages to the backup gateway. Additional info: 1. FYI... I wanted a faster switch over with time and I had to change carp a bit to allow polling rates of under a second. Also there was a bug where setting the advbase 0 and advskew 100 only set the proper value of advbase the second time ifconfig command is typed. The patches have been submitted to [EMAIL PROTECTED] Marco Pfatschbacher was nice and added the changes. The changes will be found in OpenBSD 4.2. With advbase 0 and advskew 25 the switchover is half a second to a second. 2. I have noted that when sasyncd is copying the SAs on the backup, it does not set the validity of the SAs to the remaining validity time of that SA (for instance when the backup is booting later). The validity time is set as if the SA has just been created. This way the backup will still have in its SADB Security Associations copied from the master that are expired and removed from the master. 3. Another problem (rebooting the master/backup in a given order) can get to pretty bizar situation where a redundant gateway has 4 unidirectional SAs, and it is using one SA from one the first main mode to send, and one SA from the latter main-mode to receive. A ping message does not go through, although both ends have the 4 SAs. This is a topic of its own, if you want to know more I can give you the detailed information how to reproduce it. Many thanks, Catalin - Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail
