Re: Poor CARP Interface Performance with NAT
On Tue, Jan 21, 2014 at 03:51:23PM -0800, Gabriel Kuri wrote: I am running obsd 5.4 as my NAT router. I decided to setup a second obsd box and run carp between the two for the external NATed interface (facing the ISP). After I setup everything and switched pf to NAT using the address on the carp interface, I'm seeing about 12Mbps - 13Mbps on the download, I have a 60Mbps pipe (down). When I switch pf back to NAT using the address on the physical interface, I get my full 60Mbps. Any ideas as to what I could be doing wrong that would limit performance through the carp interface to around 12Mbps - 13Mbps ? You might want to try posting this to the pf mailing list: http://www.benzedrine.cx/mailinglist.html Maybe somebody there will have a suggestion?
Re: Poor CARP Interface Performance with NAT
Can't see anything immediatly wrong with this but their are a few
things you could change/improve;
# Why are you setting the tcpflags. The defaults are fine and usualy
better.. And keep state is the default
tcpflags = S/SA
# macros
ext_if = em0
carp_ip = removed
int_nets = { removed }
router = removed
# reject = black hole
table reject file /etc/pf/reject
# nat/rdr
match out on em0 from $int_nets to any nat-to $carp_ip
#**
#**
# Rules start here
#**
#**
set skip on { lo0, pfsync0, em1, em4 }
###
# All the IN rules go here.
# allow ssh/smtp/dns to router - Why are you addressing the physical
routers IP directly? Generaly it is better for all inbound traffic
(public DNS) to refer to CARP IPs so it doesn't matter which firewall
is live/if one is down etc.
pass in quick on $ext_if proto { tcp, udp } from any to { $router,
$carp_ip } port 53 set prio 6
pass in quick on $ext_if proto tcp from any to { $router, $carp_ip }
port { 25 } set prio 2
pass in quick on $ext_if proto tcp from any to { $router, $carp_ip }
port 22 synproxy state (max-src-conn-rate 5/60, overload reject flush
global) set prio (2,6)
###
# all the OUT rules go here.
# allow router and nat out
pass out quick on $if_ext all modulate state
# block the rest
block out log quick all
block in log quick all
Good luck, Andy.
On Wed 22 Jan 2014 17:20:52 GMT, Gabriel Kuri wrote:
Here it is.
em0 - external interface (facing ISP)
em1 - internal interface, layer3, (facing switch running OSPF)
em4 - pfsync interface
pf.conf:
tcpflags = S/SA
# macros
ext_if = em0
carp_ip = removed
int_nets = { removed }
router = removed
# reject = black hole
table reject file /etc/pf/reject
# nat/rdr
match out on em0 from $int_nets to any nat-to $carp_ip
#**
#**
# Rules start here
#**
#**
# anything goes on the loopback interface
pass in quick on lo0 all
pass out quick on lo0 all
# anything goes on em1
pass in quick on em1 all
pass out quick on em1 all
# anything goes on em4
pass in quick on em4 all
pass out quick on em4 all
###
# All the IN rules go here.
# allow ssh/smtp/dns to router
pass in quick on $ext_if proto udp from any to $router port 53 keep state
pass in quick on $ext_if proto tcp from any to $router port {25 53} flags
$tcpflags keep state
pass in quick on $ext_if proto tcp from any to $router port 22 flags
$tcpflags keep state (max-src-conn-rate 5/60)
###
# all the OUT rules go here.
# allow router out
pass out quick on $ext_if from ip_removed to any flags $tcpflags keep
state
# allow nat interface out
pass out quick on $ext_if from $carp_ip to any flags $tcpflags keep state
# block the rest
block out log quick all
block in log quick all
On Tue, Jan 21, 2014 at 11:21 PM, mxb m...@alumni.chalmers.se wrote:
You PF rules are needed too for this.
On 22 jan 2014, at 00:51, Gabriel Kuri gk...@ieee.org wrote:
I am running obsd 5.4 as my NAT router. I decided to setup a second obsd
box and run carp between the two for the external NATed interface (facing
the ISP). After I setup everything and switched pf to NAT using the
address
on the carp interface, I'm seeing about 12Mbps - 13Mbps on the download,
I
have a 60Mbps pipe (down). When I switch pf back to NAT using the address
on the physical interface, I get my full 60Mbps. Any ideas as to what I
could be doing wrong that would limit performance through the carp
interface to around 12Mbps - 13Mbps ?
Thanks ...
Re: Poor CARP Interface Performance with NAT
Here it is.
em0 - external interface (facing ISP)
em1 - internal interface, layer3, (facing switch running OSPF)
em4 - pfsync interface
pf.conf:
tcpflags = S/SA
# macros
ext_if = em0
carp_ip = removed
int_nets = { removed }
router = removed
# reject = black hole
table reject file /etc/pf/reject
# nat/rdr
match out on em0 from $int_nets to any nat-to $carp_ip
#**
#**
# Rules start here
#**
#**
# anything goes on the loopback interface
pass in quick on lo0 all
pass out quick on lo0 all
# anything goes on em1
pass in quick on em1 all
pass out quick on em1 all
# anything goes on em4
pass in quick on em4 all
pass out quick on em4 all
###
# All the IN rules go here.
# allow ssh/smtp/dns to router
pass in quick on $ext_if proto udp from any to $router port 53 keep state
pass in quick on $ext_if proto tcp from any to $router port {25 53} flags
$tcpflags keep state
pass in quick on $ext_if proto tcp from any to $router port 22 flags
$tcpflags keep state (max-src-conn-rate 5/60)
###
# all the OUT rules go here.
# allow router out
pass out quick on $ext_if from ip_removed to any flags $tcpflags keep
state
# allow nat interface out
pass out quick on $ext_if from $carp_ip to any flags $tcpflags keep state
# block the rest
block out log quick all
block in log quick all
On Tue, Jan 21, 2014 at 11:21 PM, mxb m...@alumni.chalmers.se wrote:
You PF rules are needed too for this.
On 22 jan 2014, at 00:51, Gabriel Kuri gk...@ieee.org wrote:
I am running obsd 5.4 as my NAT router. I decided to setup a second obsd
box and run carp between the two for the external NATed interface (facing
the ISP). After I setup everything and switched pf to NAT using the
address
on the carp interface, I'm seeing about 12Mbps - 13Mbps on the download,
I
have a 60Mbps pipe (down). When I switch pf back to NAT using the address
on the physical interface, I get my full 60Mbps. Any ideas as to what I
could be doing wrong that would limit performance through the carp
interface to around 12Mbps - 13Mbps ?
Thanks ...
Poor CARP Interface Performance with NAT
I am running obsd 5.4 as my NAT router. I decided to setup a second obsd box and run carp between the two for the external NATed interface (facing the ISP). After I setup everything and switched pf to NAT using the address on the carp interface, I'm seeing about 12Mbps - 13Mbps on the download, I have a 60Mbps pipe (down). When I switch pf back to NAT using the address on the physical interface, I get my full 60Mbps. Any ideas as to what I could be doing wrong that would limit performance through the carp interface to around 12Mbps - 13Mbps ? Thanks ...
Re: Poor CARP Interface Performance with NAT
Also, in case it might help, here's the dmesg output and appropriate hostname.* files from the primary (master): dmesg: OpenBSD 5.4 (GENERIC.MP) #41: Tue Jul 30 15:30:02 MDT 2013 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 2119761920 (2021MB) avail mem = 2055667712 (1960MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.6 @ 0x9f800 (22 entries) bios0: vendor American Megatrends Inc. version 080016 date 08/10/2012 acpi0 at bios0: rev 2 acpi0: sleep states S0 S1 S4 S5 acpi0: tables DSDT FACP APIC MCFG OEMB HPET GSCI acpi0: wakeup devices P0P1(S4) PS2K(S4) PS2M(S4) USB0(S4) USB1(S4) USB2(S4) USB5(S4) EUSB(S4) USB3(S4) USB4(S4) USB6(S4) USBE(S4) P0P4(S4) P0P5(S4) P0P6(S4) P0P7(S4) [...] acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1800.22 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF cpu0: 512KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 0 cpu0: apic clock running at 199MHz cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1799.99 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF cpu1: 512KB 64b/line 8-way L2 cache cpu1: smt 0, core 1, package 0 cpu2 at mainbus0: apid 1 (application processor) cpu2: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1799.98 MHz cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF cpu2: 512KB 64b/line 8-way L2 cache cpu2: smt 1, core 0, package 0 cpu3 at mainbus0: apid 3 (application processor) cpu3: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1799.99 MHz cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF cpu3: 512KB 64b/line 8-way L2 cache cpu3: smt 1, core 1, package 0 ioapic0 at mainbus0: apid 4 pa 0xfec0, version 20, 24 pins ioapic0: misconfigured as apic 1, remapped to apid 4 acpimcfg0 at acpi0 addr 0xe000, bus 0-255 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 5 (P0P1) acpiprt2 at acpi0: bus 1 (P0P4) acpiprt3 at acpi0: bus -1 (P0P5) acpiprt4 at acpi0: bus 2 (P0P6) acpiprt5 at acpi0: bus 3 (P0P7) acpiprt6 at acpi0: bus 4 (P0P8) acpiprt7 at acpi0: bus -1 (P0P9) acpicpu0 at acpi0 acpicpu1 at acpi0 acpicpu2 at acpi0 acpicpu3 at acpi0 acpibtn0 at acpi0: SLPB acpibtn1 at acpi0: PWRB pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 Intel Pineview DMI rev 0x02 vga1 at pci0 dev 2 function 0 Intel Pineview Video rev 0x02 intagp0 at vga1 agp0 at intagp0: aperture at 0xd000, size 0x1000 inteldrm0 at vga1 drm0 at inteldrm0 intel_overlay_map_regs partial stub inteldrm0: 1024x768 wsdisplay0 at vga1 mux 1: console (std, vt100 emulation) wsdisplay0: screen 1-5 added (std, vt100 emulation) Intel Pineview Video rev 0x02 at pci0 dev 2 function 1 not configured uhci0 at pci0 dev 26 function 0 Intel 82801I USB rev 0x02: apic 4 int 16 uhci1 at pci0 dev 26 function 1 Intel 82801I USB rev 0x02: apic 4 int 21 uhci2 at pci0 dev 26 function 2 Intel 82801I USB rev 0x02: apic 4 int 19 ehci0 at pci0 dev 26 function 7 Intel 82801I USB rev 0x02: apic 4 int 18 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb0 at pci0 dev 28 function 0 Intel 82801I PCIE rev 0x02: msi pci1 at ppb0 bus 1 ppb1 at pci0 dev 28 function 2 Intel 82801I PCIE rev 0x02: msi pci2 at ppb1 bus 2 em0 at pci2 dev 0 function 0 Intel 82574L rev 0x00: msi, address 00:30:18:a2:fd:ad ppb2 at pci0 dev 28 function 3 Intel 82801I PCIE rev 0x02: msi pci3 at ppb2 bus 3 em1 at pci3 dev 0 function 0 Intel 82574L rev 0x00: msi, address 00:30:18:a2:fd:ae ppb3 at pci0 dev 28 function 4 Intel 82801I PCIE rev 0x02: msi pci4 at ppb3 bus 4 ASMedia ASM1042 xHCI rev 0x00 at pci4 dev 0 function 0 not configured uhci3 at pci0 dev 29 function 0 Intel 82801I USB rev 0x02: apic 4 int 23 uhci4 at pci0 dev 29 function 1 Intel 82801I USB rev 0x02: apic 4 int 19 uhci5 at pci0 dev 29 function 2 Intel 82801I USB rev 0x02: apic 4 int 18 ehci1 at pci0 dev 29 function 7 Intel 82801I USB rev 0x02: apic 4 int 23 usb1 at ehci1: USB revision 2.0 uhub1 at usb1 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb4 at pci0 dev 30 function 0 Intel 82801BA Hub-to-PCI rev 0x92 pci5 at ppb4 bus 5 em2 at pci5 dev 4 function 0 Intel 82541GI rev 0x05: apic 4 int 18, address 00:30:18:a0:f5:ad em3 at pci5 dev 6 function 0 Intel 82541GI rev 0x05: apic 4 int 19, address 00:30:18:a0:f5:ae em4
Re: Poor CARP Interface Performance with NAT
You PF rules are needed too for this. On 22 jan 2014, at 00:51, Gabriel Kuri gk...@ieee.org wrote: I am running obsd 5.4 as my NAT router. I decided to setup a second obsd box and run carp between the two for the external NATed interface (facing the ISP). After I setup everything and switched pf to NAT using the address on the carp interface, I'm seeing about 12Mbps - 13Mbps on the download, I have a 60Mbps pipe (down). When I switch pf back to NAT using the address on the physical interface, I get my full 60Mbps. Any ideas as to what I could be doing wrong that would limit performance through the carp interface to around 12Mbps - 13Mbps ? Thanks ...
