Re: CARP load balancing problems under KVM
Many thanks David for your help. I am using virtual switches on all OpenBSD's guest interfaces, except for one which it is the "public" interface connected to my ISP's router. I will try to reconfigure two scenarios: one with OpenBSD 6.7 guests and another with OpenBSD 6.8 guests and I will keep you update. Many thanks. On 9/1/21, 11:14, "David Gwynne" wrote: Hey Carlos, I've spent a bit of time today trying to figure out what's going on here, and haven't found anything that looks wrong with carp in 6.8. I did have a lot of trouble trying to reproduce it though, but that's because some of the switches involved seem to be "helping" and filtering packets sent from a multicast MAC address. I could see the carp interface get arp requests for the shared IP, and reply to them, but I never saw the replies on any other machine. However, I was able to build a test setup with carp on top of nvgre between a bunch of machines, and that abstracted me enough off the physical network to test with. As expected, it all worked fine. The only thing that's changing in your setup is the openbsd version? You're not upgrading the host machines or using different physical switches at the same time or anything? To debug this further I'd like to look at packet captures. Can you tcpump on the carp hosts and the client machines? If possible, captures from a 6.7 setup too would be nice. Cheers, dlg > On 5 Jan 2021, at 1:59 am, Carlos Lopez wrote: > > Good afternoon, > > Any news about this bug? > > On 21/10/20, 12:37, "owner-b...@openbsd.org on behalf of Carlos Lopez" wrote: > >Hi all, > >Before upgrade from OpenBSD 6.7 to OpenBSD 6.8, my pair firewalls was using carp in IP balance mode without problems from several months. These firewalls are installed in a RHEL 8.2 (fully patched) KVM host. > >After upgrading to OpenBSD 6.8, carp ip balance mode doesn’t works. I have tested reconfiguring balance mode for ip-stealth and ip-unicast also and the result is always the same: network packets are not processed by firewalls. But if I configure CARP using “the simple configuration” and one node is master and the other is backup all it is working without problems. > >All CARP interfaces are configured as this one: > >carpdev vio0 balancing ip pass 7254e4bc3024e35490e4b9942f919e9b >inet 172.22.55.30 0xffe0 172.22.55.31 >carpnodes 10:0,11:100 >description "Production Network" > >sysctl.conf file: > >net.inet.carp.preempt=1 >net.inet.carp.log=2 >net.inet.ip.forwarding=1 >net.inet.tcp.mssdflt=1440 >net.inet.ip.redirect=0 >net.inet.ip.mtudisc=0 >net.inet.tcp.rfc3390=1 >net.inet.ip.arptimeout=60 >kern.bufcachepercent=70 >net.inet.icmp.tstamprepl=0 >net.inet.udp.sendspace=262144 >net.inet.udp.recvspace=262144 > > >OpenBSD kvm guest config: > > > obsdfw01 > OpenBSD Security Gateway Cluster > 786432 > 786432 > 1 > >/machine > > >hvm > > > > > > > >Broadwell > > > > > > > > > > > > > > destroy > restart > destroy > > > > > >/usr/libexec/qemu-kvm > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: CARP load balancing problems under KVM
Hey Carlos, I've spent a bit of time today trying to figure out what's going on here, and haven't found anything that looks wrong with carp in 6.8. I did have a lot of trouble trying to reproduce it though, but that's because some of the switches involved seem to be "helping" and filtering packets sent from a multicast MAC address. I could see the carp interface get arp requests for the shared IP, and reply to them, but I never saw the replies on any other machine. However, I was able to build a test setup with carp on top of nvgre between a bunch of machines, and that abstracted me enough off the physical network to test with. As expected, it all worked fine. The only thing that's changing in your setup is the openbsd version? You're not upgrading the host machines or using different physical switches at the same time or anything? To debug this further I'd like to look at packet captures. Can you tcpump on the carp hosts and the client machines? If possible, captures from a 6.7 setup too would be nice. Cheers, dlg > On 5 Jan 2021, at 1:59 am, Carlos Lopez wrote: > > Good afternoon, > > Any news about this bug? > > On 21/10/20, 12:37, "owner-b...@openbsd.org on behalf of Carlos Lopez" > wrote: > >Hi all, > >Before upgrade from OpenBSD 6.7 to OpenBSD 6.8, my pair firewalls was > using carp in IP balance mode without problems from several months. These > firewalls are installed in a RHEL 8.2 (fully patched) KVM host. > >After upgrading to OpenBSD 6.8, carp ip balance mode doesn’t works. I have > tested reconfiguring balance mode for ip-stealth and ip-unicast also and the > result is always the same: network packets are not processed by firewalls. > But if I configure CARP using “the simple configuration” and one node is > master and the other is backup all it is working without problems. > >All CARP interfaces are configured as this one: > >carpdev vio0 balancing ip pass 7254e4bc3024e35490e4b9942f919e9b >inet 172.22.55.30 0xffe0 172.22.55.31 >carpnodes 10:0,11:100 >description "Production Network" > >sysctl.conf file: > >net.inet.carp.preempt=1 >net.inet.carp.log=2 >net.inet.ip.forwarding=1 >net.inet.tcp.mssdflt=1440 >net.inet.ip.redirect=0 >net.inet.ip.mtudisc=0 >net.inet.tcp.rfc3390=1 >net.inet.ip.arptimeout=60 >kern.bufcachepercent=70 >net.inet.icmp.tstamprepl=0 >net.inet.udp.sendspace=262144 >net.inet.udp.recvspace=262144 > > >OpenBSD kvm guest config: > > > obsdfw01 > OpenBSD Security Gateway Cluster > 786432 > 786432 > 1 > >/machine > > >hvm > > > > > > > >Broadwell > > > > > > > > > > > > > > destroy > restart > destroy > > > > > >/usr/libexec/qemu-kvm > > > > > > > function='0x0'/> > > > > > > > function='0x0'/> > > > > > > > > > function='0x0' multifunction='on'/> > > > > > > function='0x1'/> > > > > > > function='0x2'/> > > > > > > function='0x3'/> > > > > > > function='0x4'/> > > > > > > function='0x5'/> > > > > > > function='0x6'/> > > > > > > function='0x7'/> > > > > > > function='0x0' multifunction='on'/> > > > > > > function='0x1'/> > > > > > > function='0x2'/> > > > > > > function='0x3'/> > > > > > > function='0x4'/> > > > > function='0x2'/> > > > > > > > > function='0x0'/> > > > > > > > > function='0x0'/> > > > > > > > > function='0x0'/> > > > > > > > > function='0x0'/> > > > > > > > > function='0x0'/> > >
Re: CARP load balancing problems under KVM
Good afternoon, Any news about this bug? On 21/10/20, 12:37, "owner-b...@openbsd.org on behalf of Carlos Lopez" wrote: Hi all, Before upgrade from OpenBSD 6.7 to OpenBSD 6.8, my pair firewalls was using carp in IP balance mode without problems from several months. These firewalls are installed in a RHEL 8.2 (fully patched) KVM host. After upgrading to OpenBSD 6.8, carp ip balance mode doesn’t works. I have tested reconfiguring balance mode for ip-stealth and ip-unicast also and the result is always the same: network packets are not processed by firewalls. But if I configure CARP using “the simple configuration” and one node is master and the other is backup all it is working without problems. All CARP interfaces are configured as this one: carpdev vio0 balancing ip pass 7254e4bc3024e35490e4b9942f919e9b inet 172.22.55.30 0xffe0 172.22.55.31 carpnodes 10:0,11:100 description "Production Network" sysctl.conf file: net.inet.carp.preempt=1 net.inet.carp.log=2 net.inet.ip.forwarding=1 net.inet.tcp.mssdflt=1440 net.inet.ip.redirect=0 net.inet.ip.mtudisc=0 net.inet.tcp.rfc3390=1 net.inet.ip.arptimeout=60 kern.bufcachepercent=70 net.inet.icmp.tstamprepl=0 net.inet.udp.sendspace=262144 net.inet.udp.recvspace=262144 OpenBSD kvm guest config: obsdfw01 OpenBSD Security Gateway Cluster 786432 786432 1 /machine hvm Broadwell destroy restart destroy /usr/libexec/qemu-kvm /dev/urandom system_u:system_r:svirt_t:s0:c82,c777 system_u:object_r:svirt_image_t:s0:c82,c777 +107:+107 +107:+107 Dmesg output: OpenBSD 6.8 (GENERIC) #97: Sun Oct 4 18:00:46 MDT 2020 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC real mem = 788389888 (751MB) avail mem = 749596672 (714MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5af0 (9 entries) bios0: vendor SeaBIOS version "1.11.1-4.module+el8.1.0+4066+0f1aadab" date 04/01/2014 bios0: Red Hat KVM acpi0 at bios0: ACPI 3.0 acpi0: sleep states S5 acpi0: tables DSDT FACP APIC MCFG acpi0: wakeup devices acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel Core Processor (Broadwell),
Re: CARP load balancing problems under KVM
Good afternoon, Any news about this? Regards. On 21/10/2020, 12:37, "owner-b...@openbsd.org on behalf of Carlos Lopez" wrote: Hi all, Before upgrade from OpenBSD 6.7 to OpenBSD 6.8, my pair firewalls was using carp in IP balance mode without problems from several months. These firewalls are installed in a RHEL 8.2 (fully patched) KVM host. After upgrading to OpenBSD 6.8, carp ip balance mode doesn’t works. I have tested reconfiguring balance mode for ip-stealth and ip-unicast also and the result is always the same: network packets are not processed by firewalls. But if I configure CARP using “the simple configuration” and one node is master and the other is backup all it is working without problems. All CARP interfaces are configured as this one: carpdev vio0 balancing ip pass 7254e4bc3024e35490e4b9942f919e9b inet 172.22.55.30 0xffe0 172.22.55.31 carpnodes 10:0,11:100 description "Production Network" sysctl.conf file: net.inet.carp.preempt=1 net.inet.carp.log=2 net.inet.ip.forwarding=1 net.inet.tcp.mssdflt=1440 net.inet.ip.redirect=0 net.inet.ip.mtudisc=0 net.inet.tcp.rfc3390=1 net.inet.ip.arptimeout=60 kern.bufcachepercent=70 net.inet.icmp.tstamprepl=0 net.inet.udp.sendspace=262144 net.inet.udp.recvspace=262144 OpenBSD kvm guest config: obsdfw01 OpenBSD Security Gateway Cluster 786432 786432 1 /machine hvm Broadwell destroy restart destroy /usr/libexec/qemu-kvm /dev/urandom system_u:system_r:svirt_t:s0:c82,c777 system_u:object_r:svirt_image_t:s0:c82,c777 +107:+107 +107:+107 Dmesg output: OpenBSD 6.8 (GENERIC) #97: Sun Oct 4 18:00:46 MDT 2020 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC real mem = 788389888 (751MB) avail mem = 749596672 (714MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5af0 (9 entries) bios0: vendor SeaBIOS version "1.11.1-4.module+el8.1.0+4066+0f1aadab" date 04/01/2014 bios0: Red Hat KVM acpi0 at bios0: ACPI 3.0 acpi0: sleep states S5 acpi0: tables DSDT FACP APIC MCFG acpi0: wakeup devices acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel Core Processor
Re: CARP load balancing problems under KVM
On 21 Oct 17:58, Uwe Werler wrote: > > Mmh, it seems to have something to do how both nodes calculate to which ip > they respond. When I have two hosts with consecutive addresses carp responds > to one of them. For example: > > carp 172.16.10.15 > > host 1: 172.16.20.11: ping 172.16.10.15 works > host 2: 172.16.20.12: ping 172.16.10.15 doesn't work. > > Tried that from within host in the same subnet and also from other subnets. > Ok, when carpnode 1 becomes the master by: ipconfig -g carp carpdemote executed on the 2nd carpnode the carp address responds to ping from host 1 and host 2. If carpnode 2 becomes the master the carp ip responds neither to echo requests from host 1 nor host 2. -- wq: ~uw
Re: CARP load balancing problems under KVM
On 21 Oct 10:35, Carlos Lopez wrote: > Hi all, > > Before upgrade from OpenBSD 6.7 to OpenBSD 6.8, my pair firewalls was using > carp in IP balance mode without problems from several months. These firewalls > are installed in a RHEL 8.2 (fully patched) KVM host. > > After upgrading to OpenBSD 6.8, carp ip balance mode doesn’t works. I have > tested reconfiguring balance mode for ip-stealth and ip-unicast also and the > result is always the same: network packets are not processed by firewalls. > But if I configure CARP using “the simple configuration” and one node is > master and the other is backup all it is working without problems. > > All CARP interfaces are configured as this one: > > carpdev vio0 balancing ip pass 7254e4bc3024e35490e4b9942f919e9b > inet 172.22.55.30 0xffe0 172.22.55.31 > carpnodes 10:0,11:100 > description "Production Network" > > sysctl.conf file: > > net.inet.carp.preempt=1 > net.inet.carp.log=2 > net.inet.ip.forwarding=1 > net.inet.tcp.mssdflt=1440 > net.inet.ip.redirect=0 > net.inet.ip.mtudisc=0 > net.inet.tcp.rfc3390=1 > net.inet.ip.arptimeout=60 > kern.bufcachepercent=70 > net.inet.icmp.tstamprepl=0 > net.inet.udp.sendspace=262144 > net.inet.udp.recvspace=262144 > > > OpenBSD kvm guest config: > > > obsdfw01 > OpenBSD Security Gateway Cluster > 786432 > 786432 > 1 > > /machine > > > hvm > > > > > > > > Broadwell > > > > > > > > > > > > > > destroy > restart > destroy > > > > > > /usr/libexec/qemu-kvm > > > > > > >function='0x0'/> > > > > > > >function='0x0'/> > > > > > > > > >function='0x0' multifunction='on'/> > > > > > >function='0x1'/> > > > > > >function='0x2'/> > > > > > >function='0x3'/> > > > > > >function='0x4'/> > > > > > >function='0x5'/> > > > > > >function='0x6'/> > > > > > >function='0x7'/> > > > > > >function='0x0' multifunction='on'/> > > > > > >function='0x1'/> > > > > > >function='0x2'/> > > > > > >function='0x3'/> > > > > > >function='0x4'/> > > > >function='0x2'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > > > > > > > > > > > > > > > > > > > keymap='es'> > > > >primary='yes'/> > >function='0x0'/> > > > >function='0x0'/> > > > /dev/urandom > >function='0x0'/> > > > > system_u:system_r:svirt_t:s0:c82,c777 > system_u:object_r:svirt_image_t:s0:c82,c777 > > > +107:+107 > +107:+107 > > > > Dmesg output: > > OpenBSD 6.8 (GENERIC) #97: Sun Oct 4 18:00:46 MDT 2020 > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC > real mem = 788389888 (751MB) > avail mem = 749596672 (714MB) > random: good seed from bootblocks > mpath0 at root > scsibus0 at mpath0: 256 targets > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5af0 (9 entries) > bios0: vendor SeaBIOS version "1.11.1-4.module+el8.1.0+4066+0f1aadab" date > 04/01/2014 > bios0: Red Hat KVM > acpi0 at bios0: ACPI 3.0 > acpi0: sleep states S5 > acpi0: tables DSDT FACP APIC MCFG > acpi0: wakeup devices > acpitimer0 at acpi0: 3579545 Hz, 24 bits > acpimadt0 at acpi0 addr 0xfee0: PC-AT compat > cpu0 at mainbus0: apid 0 (boot processor) > cpu0: Intel Core Processor (Broadwell), 1900.29 MHz, 06-3d-02 > cpu0: >
Re: CARP load balancing problems under KVM
On 21 Oct 10:35, Carlos Lopez wrote: > Hi all, > > Before upgrade from OpenBSD 6.7 to OpenBSD 6.8, my pair firewalls was using > carp in IP balance mode without problems from several months. These firewalls > are installed in a RHEL 8.2 (fully patched) KVM host. > > After upgrading to OpenBSD 6.8, carp ip balance mode doesn’t works. I have > tested reconfiguring balance mode for ip-stealth and ip-unicast also and the > result is always the same: network packets are not processed by firewalls. > But if I configure CARP using “the simple configuration” and one node is > master and the other is backup all it is working without problems. > > All CARP interfaces are configured as this one: > > carpdev vio0 balancing ip pass 7254e4bc3024e35490e4b9942f919e9b > inet 172.22.55.30 0xffe0 172.22.55.31 > carpnodes 10:0,11:100 > description "Production Network" > > sysctl.conf file: > > net.inet.carp.preempt=1 > net.inet.carp.log=2 > net.inet.ip.forwarding=1 > net.inet.tcp.mssdflt=1440 > net.inet.ip.redirect=0 > net.inet.ip.mtudisc=0 > net.inet.tcp.rfc3390=1 > net.inet.ip.arptimeout=60 > kern.bufcachepercent=70 > net.inet.icmp.tstamprepl=0 > net.inet.udp.sendspace=262144 > net.inet.udp.recvspace=262144 > > > OpenBSD kvm guest config: > > > obsdfw01 > OpenBSD Security Gateway Cluster > 786432 > 786432 > 1 > > /machine > > > hvm > > > > > > > > Broadwell > > > > > > > > > > > > > > destroy > restart > destroy > > > > > > /usr/libexec/qemu-kvm > > > > > > >function='0x0'/> > > > > > > >function='0x0'/> > > > > > > > > >function='0x0' multifunction='on'/> > > > > > >function='0x1'/> > > > > > >function='0x2'/> > > > > > >function='0x3'/> > > > > > >function='0x4'/> > > > > > >function='0x5'/> > > > > > >function='0x6'/> > > > > > >function='0x7'/> > > > > > >function='0x0' multifunction='on'/> > > > > > >function='0x1'/> > > > > > >function='0x2'/> > > > > > >function='0x3'/> > > > > > >function='0x4'/> > > > >function='0x2'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > > > > > > > > > > > > > > > > > > > keymap='es'> > > > >primary='yes'/> > >function='0x0'/> > > > >function='0x0'/> > > > /dev/urandom > >function='0x0'/> > > > > system_u:system_r:svirt_t:s0:c82,c777 > system_u:object_r:svirt_image_t:s0:c82,c777 > > > +107:+107 > +107:+107 > > > > Dmesg output: > > OpenBSD 6.8 (GENERIC) #97: Sun Oct 4 18:00:46 MDT 2020 > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC > real mem = 788389888 (751MB) > avail mem = 749596672 (714MB) > random: good seed from bootblocks > mpath0 at root > scsibus0 at mpath0: 256 targets > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5af0 (9 entries) > bios0: vendor SeaBIOS version "1.11.1-4.module+el8.1.0+4066+0f1aadab" date > 04/01/2014 > bios0: Red Hat KVM > acpi0 at bios0: ACPI 3.0 > acpi0: sleep states S5 > acpi0: tables DSDT FACP APIC MCFG > acpi0: wakeup devices > acpitimer0 at acpi0: 3579545 Hz, 24 bits > acpimadt0 at acpi0 addr 0xfee0: PC-AT compat > cpu0 at mainbus0: apid 0 (boot processor) > cpu0: Intel Core Processor (Broadwell), 1900.29 MHz, 06-3d-02 > cpu0: >
Re: CARP load balancing problems under KVM
On 21 Oct 10:35, Carlos Lopez wrote: > Hi all, > > Before upgrade from OpenBSD 6.7 to OpenBSD 6.8, my pair firewalls was using > carp in IP balance mode without problems from several months. These firewalls > are installed in a RHEL 8.2 (fully patched) KVM host. > > After upgrading to OpenBSD 6.8, carp ip balance mode doesn’t works. I have > tested reconfiguring balance mode for ip-stealth and ip-unicast also and the > result is always the same: network packets are not processed by firewalls. > But if I configure CARP using “the simple configuration” and one node is > master and the other is backup all it is working without problems. > > All CARP interfaces are configured as this one: > > carpdev vio0 balancing ip pass 7254e4bc3024e35490e4b9942f919e9b > inet 172.22.55.30 0xffe0 172.22.55.31 > carpnodes 10:0,11:100 > description "Production Network" > > sysctl.conf file: > > net.inet.carp.preempt=1 > net.inet.carp.log=2 > net.inet.ip.forwarding=1 > net.inet.tcp.mssdflt=1440 > net.inet.ip.redirect=0 > net.inet.ip.mtudisc=0 > net.inet.tcp.rfc3390=1 > net.inet.ip.arptimeout=60 > kern.bufcachepercent=70 > net.inet.icmp.tstamprepl=0 > net.inet.udp.sendspace=262144 > net.inet.udp.recvspace=262144 > > > OpenBSD kvm guest config: > > > obsdfw01 > OpenBSD Security Gateway Cluster > 786432 > 786432 > 1 > > /machine > > > hvm > > > > > > > > Broadwell > > > > > > > > > > > > > > destroy > restart > destroy > > > > > > /usr/libexec/qemu-kvm > > > > > > >function='0x0'/> > > > > > > >function='0x0'/> > > > > > > > > >function='0x0' multifunction='on'/> > > > > > >function='0x1'/> > > > > > >function='0x2'/> > > > > > >function='0x3'/> > > > > > >function='0x4'/> > > > > > >function='0x5'/> > > > > > >function='0x6'/> > > > > > >function='0x7'/> > > > > > >function='0x0' multifunction='on'/> > > > > > >function='0x1'/> > > > > > >function='0x2'/> > > > > > >function='0x3'/> > > > > > >function='0x4'/> > > > >function='0x2'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > >function='0x0'/> > > > > > > > > > > > > > > > > > > > > > > > > > keymap='es'> > > > >primary='yes'/> > >function='0x0'/> > > > >function='0x0'/> > > > /dev/urandom > >function='0x0'/> > > > > system_u:system_r:svirt_t:s0:c82,c777 > system_u:object_r:svirt_image_t:s0:c82,c777 > > > +107:+107 > +107:+107 > > > > Dmesg output: > > OpenBSD 6.8 (GENERIC) #97: Sun Oct 4 18:00:46 MDT 2020 > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC > real mem = 788389888 (751MB) > avail mem = 749596672 (714MB) > random: good seed from bootblocks > mpath0 at root > scsibus0 at mpath0: 256 targets > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5af0 (9 entries) > bios0: vendor SeaBIOS version "1.11.1-4.module+el8.1.0+4066+0f1aadab" date > 04/01/2014 > bios0: Red Hat KVM > acpi0 at bios0: ACPI 3.0 > acpi0: sleep states S5 > acpi0: tables DSDT FACP APIC MCFG > acpi0: wakeup devices > acpitimer0 at acpi0: 3579545 Hz, 24 bits > acpimadt0 at acpi0 addr 0xfee0: PC-AT compat > cpu0 at mainbus0: apid 0 (boot processor) > cpu0: Intel Core Processor (Broadwell), 1900.29 MHz, 06-3d-02 > cpu0: >
CARP load balancing problems under KVM
Hi all, Before upgrade from OpenBSD 6.7 to OpenBSD 6.8, my pair firewalls was using carp in IP balance mode without problems from several months. These firewalls are installed in a RHEL 8.2 (fully patched) KVM host. After upgrading to OpenBSD 6.8, carp ip balance mode doesn’t works. I have tested reconfiguring balance mode for ip-stealth and ip-unicast also and the result is always the same: network packets are not processed by firewalls. But if I configure CARP using “the simple configuration” and one node is master and the other is backup all it is working without problems. All CARP interfaces are configured as this one: carpdev vio0 balancing ip pass 7254e4bc3024e35490e4b9942f919e9b inet 172.22.55.30 0xffe0 172.22.55.31 carpnodes 10:0,11:100 description "Production Network" sysctl.conf file: net.inet.carp.preempt=1 net.inet.carp.log=2 net.inet.ip.forwarding=1 net.inet.tcp.mssdflt=1440 net.inet.ip.redirect=0 net.inet.ip.mtudisc=0 net.inet.tcp.rfc3390=1 net.inet.ip.arptimeout=60 kern.bufcachepercent=70 net.inet.icmp.tstamprepl=0 net.inet.udp.sendspace=262144 net.inet.udp.recvspace=262144 OpenBSD kvm guest config: obsdfw01 OpenBSD Security Gateway Cluster 786432 786432 1 /machine hvm Broadwell destroy restart destroy /usr/libexec/qemu-kvm /dev/urandom system_u:system_r:svirt_t:s0:c82,c777 system_u:object_r:svirt_image_t:s0:c82,c777 +107:+107 +107:+107 Dmesg output: OpenBSD 6.8 (GENERIC) #97: Sun Oct 4 18:00:46 MDT 2020 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC real mem = 788389888 (751MB) avail mem = 749596672 (714MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5af0 (9 entries) bios0: vendor SeaBIOS version "1.11.1-4.module+el8.1.0+4066+0f1aadab" date 04/01/2014 bios0: Red Hat KVM acpi0 at bios0: ACPI 3.0 acpi0: sleep states S5 acpi0: tables DSDT FACP APIC MCFG acpi0: wakeup devices acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel Core Processor (Broadwell), 1900.29 MHz, 06-3d-02 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,RDTSCP,LONG,LAHF,ABM,3DNOWP,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,RDSEED,ADX,SMAP,ARAT,XSAVEOPT,MELTDOWN cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line 16-way L2 cache cpu0: ITLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped cpu0: DTLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 1000MHz ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins acpimcfg0 at acpi0 acpimcfg0: addr 0xb000, bus 0-255 acpiprt0 at acpi0: bus 0 (PCI0) "ACPI0006" at acpi0 not configured acpipci0 at acpi0 PCI0: 0x 0x0011 0x0001 acpicmos0 at acpi0 "PNP0A06" at acpi0 not configured "PNP0A06" at acpi0 not configured "QEMU0002" at acpi0 not configured "ACPI0010" at acpi0 not configured acpicpu0 at acpi0: C1(@1 halt!) cpu0: using Broadwell MDS workaround pvbus0 at mainbus0: KVM pvclock0 at pvbus0 pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel 82G33 Host" rev 0x00 vga1 at pci0 dev 1 function 0
