I ran a few more tests on a local setup: * 2 x OpenBSD 5.1 (i386) w/ Gbit NICs connected on the same switch * `cat /etc/ipsec.conf`: "ike esp from 10.0.0.1 to 10.0.0.2" (and vice versa) * pf is disabled
Running `isakmpd -K ; ipsecctl -f /etc/ipsec.conf` "caps" tcpbench at ~50Mbit speeds, same as our production tunnel. Without isakmpd the speed ramps up to ~800Mbit or so, which is reasonable. Right now, I have no idea what else I can try. Any suggestions are appreciated. On Wed, Oct 17, 2012 at 10:05 AM, Michael Sideris <[email protected]> wrote: > `ping -c10` > > (L-VPN --> G-VPN) > > PING G.G.G.G (G.G.G.G): 56 data bytes > 64 bytes from G.G.G.G: icmp_seq=0 ttl=255 time=17.073 ms > 64 bytes from G.G.G.G: icmp_seq=1 ttl=255 time=3.604 ms > 64 bytes from G.G.G.G: icmp_seq=2 ttl=255 time=3.666 ms > 64 bytes from G.G.G.G: icmp_seq=3 ttl=255 time=3.716 ms > 64 bytes from G.G.G.G: icmp_seq=4 ttl=255 time=3.639 ms > 64 bytes from G.G.G.G: icmp_seq=5 ttl=255 time=3.685 ms > 64 bytes from G.G.G.G: icmp_seq=6 ttl=255 time=3.734 ms > 64 bytes from G.G.G.G: icmp_seq=7 ttl=255 time=3.658 ms > 64 bytes from G.G.G.G: icmp_seq=8 ttl=255 time=3.707 ms > 64 bytes from G.G.G.G: icmp_seq=9 ttl=255 time=3.755 ms > --- G.G.G.G ping statistics --- > 10 packets transmitted, 10 packets received, 0.0% packet loss > round-trip min/avg/max/std-dev = 3.604/5.023/17.073/4.017 ms > > > (G-VPN --> L-VPN) > > PING L.L.L.L (L.L.L.L): 56 data bytes > 64 bytes from L.L.L.L: icmp_seq=0 ttl=255 time=3.707 ms > 64 bytes from L.L.L.L: icmp_seq=1 ttl=255 time=3.746 ms > 64 bytes from L.L.L.L: icmp_seq=2 ttl=255 time=3.677 ms > 64 bytes from L.L.L.L: icmp_seq=3 ttl=255 time=3.717 ms > 64 bytes from L.L.L.L: icmp_seq=4 ttl=255 time=3.754 ms > 64 bytes from L.L.L.L: icmp_seq=5 ttl=255 time=3.670 ms > 64 bytes from L.L.L.L: icmp_seq=6 ttl=255 time=3.703 ms > 64 bytes from L.L.L.L: icmp_seq=7 ttl=255 time=3.742 ms > 64 bytes from L.L.L.L: icmp_seq=8 ttl=255 time=3.654 ms > 64 bytes from L.L.L.L: icmp_seq=9 ttl=255 time=3.693 ms > --- L.L.L.L ping statistics --- > 10 packets transmitted, 10 packets received, 0.0% packet loss > round-trip min/avg/max/std-dev = 3.654/3.706/3.754/0.057 ms > > > It is also worth mentioning that if I send anything from one endpoint > to the other, the speed is ~7.5MB/s. Better than a transfer between 2 > nodes from each site but still a bit slow for a 150Mbit/s <--> 1Gbit/s > link. > > On Wed, Oct 17, 2012 at 1:36 AM, Kent Fritz <[email protected]> wrote: >> I didn't see anyone reply to this yet, so let me ask a really dumb question: >> what's the round-trip-time between G.G.G.G and L.L.L.L? Are you running >> into the TCP limits due to this? >> >> >> On Tue, Oct 16, 2012 at 2:43 AM, Michael Sideris <[email protected]> wrote: >>> >>> Hey @misc, >>> >>> ----------- ENDPOINT INFO ----------- >>> >>> `dmesg` >>> >>> (G-VPN) >>> OpenBSD 5.1 (GENERIC.MP) #207: Sun Feb 12 09:42:14 MST 2012 >>> [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP >>> real mem = 2146172928 (2046MB) >>> avail mem = 2074935296 (1978MB) >>> mainbus0 at root >>> bios0 at mainbus0: SMBIOS rev. 2.3 @ 0xfa850 (75 entries) >>> bios0: vendor Dell Computer Corporation version "A03" date 01/04/2006 >>> bios0: Dell Computer Corporation PowerEdge SC1425 >>> acpi0 at bios0: rev 0 >>> acpi0: sleep states S0 S4 S5 >>> acpi0: tables DSDT FACP APIC SPCR HPET MCFG >>> acpi0: wakeup devices PCI0(S5) PALO(S5) PXH_(S5) PXHB(S5) PXHA(S5) >>> PICH(S5) >>> acpitimer0 at acpi0: 3579545 Hz, 24 bits >>> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat >>> cpu0 at mainbus0: apid 0 (boot processor) >>> cpu0: Intel(R) Xeon(TM) CPU 2.80GHz, 2800.48 MHz >>> cpu0: >>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR,NXE,LONG >>> cpu0: 1MB 64b/line 8-way L2 cache >>> cpu0: apic clock running at 200MHz >>> cpu1 at mainbus0: apid 1 (application processor) >>> cpu1: Intel(R) Xeon(TM) CPU 2.80GHz, 2800.11 MHz >>> cpu1: >>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR,NXE,LONG >>> cpu1: 1MB 64b/line 8-way L2 cache >>> ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins >>> ioapic0: misconfigured as apic 0, remapped to apid 2 >>> ioapic1 at mainbus0: apid 3 pa 0xfec80000, version 20, 24 pins >>> ioapic1: misconfigured as apic 0, remapped to apid 3 >>> ioapic2 at mainbus0: apid 4 pa 0xfec80800, version 20, 24 pins >>> ioapic2: misconfigured as apic 0, remapped to apid 4 >>> acpihpet0 at acpi0: 14318179 Hz >>> acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255 >>> acpiprt0 at acpi0: bus 0 (PCI0) >>> acpiprt1 at acpi0: bus 1 (PALO) >>> acpiprt2 at acpi0: bus 3 (PXHB) >>> acpiprt3 at acpi0: bus 2 (PXHA) >>> acpiprt4 at acpi0: bus 4 (PICH) >>> acpicpu0 at acpi0 >>> acpicpu1 at acpi0 >>> ipmi at mainbus0 not configured >>> pci0 at mainbus0 bus 0 >>> pchb0 at pci0 dev 0 function 0 "Intel E7520 Host" rev 0x09 >>> ppb0 at pci0 dev 2 function 0 "Intel E7520 PCIE" rev 0x09 >>> pci1 at ppb0 bus 1 >>> ppb1 at pci1 dev 0 function 0 "Intel 6700PXH PCIE-PCIX" rev 0x09 >>> pci2 at ppb1 bus 2 >>> em0 at pci2 dev 4 function 0 "Intel PRO/1000MT (82541GI)" rev 0x05: >>> apic 3 int 0, address 00:14:22:72:61:c6 >>> ppb2 at pci1 dev 0 function 2 "Intel 6700PXH PCIE-PCIX" rev 0x09 >>> pci3 at ppb2 bus 3 >>> isp0 at pci3 dev 7 function 0 "QLogic ISP2312" rev 0x02: apic 4 int 2 >>> isp0: board type 2312 rev 0x2, loaded firmware rev 3.3.19 >>> scsibus0 at isp0: 512 targets, WWPN 210000e08b1d3fc7, WWNN >>> 200000e08b1d3fc7 >>> uhci0 at pci0 dev 29 function 0 "Intel 82801EB/ER USB" rev 0x02: apic 2 >>> int 16 >>> uhci1 at pci0 dev 29 function 1 "Intel 82801EB/ER USB" rev 0x02: apic 2 >>> int 19 >>> ehci0 at pci0 dev 29 function 7 "Intel 82801EB/ER USB2" rev 0x02: apic 2 >>> int 23 >>> usb0 at ehci0: USB revision 2.0 >>> uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 >>> ppb3 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xc2 >>> pci4 at ppb3 bus 4 >>> em1 at pci4 dev 3 function 0 "Intel PRO/1000MT (82541GI)" rev 0x05: >>> apic 2 int 20, address 00:14:22:72:61:c7 >>> vga1 at pci4 dev 13 function 0 "ATI Radeon VE" rev 0x00 >>> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) >>> wsdisplay0: screen 1-5 added (80x25, vt100 emulation) >>> radeondrm0 at vga1: apic 2 int 17 >>> drm0 at radeondrm0 >>> pcib0 at pci0 dev 31 function 0 "Intel 82801EB/ER LPC" rev 0x02 >>> pciide0 at pci0 dev 31 function 1 "Intel 82801EB/ER IDE" rev 0x02: >>> DMA, channel 0 configured to compatibility, channel 1 configured to >>> compatibility >>> atapiscsi0 at pciide0 channel 0 drive 0 >>> scsibus1 at atapiscsi0: 2 targets >>> cd0 at scsibus1 targ 0 lun 0: <HL-DT-ST, CD-ROM GCR-8240N, 1.06> ATAPI >>> 5/cdrom removable >>> cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2 >>> pciide0: channel 1 ignored (disabled) >>> pciide1 at pci0 dev 31 function 2 "Intel 82801EB SATA" rev 0x02: DMA, >>> channel 0 configured to native-PCI, channel 1 configured to native-PCI >>> pciide1: using apic 2 int 18 for native-PCI interrupt >>> wd0 at pciide1 channel 0 drive 0: <Maxtor 7Y250M0> >>> wd0: 16-sector PIO, LBA48, 238418MB, 488281250 sectors >>> wd0(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 6 >>> usb1 at uhci0: USB revision 1.0 >>> uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1 >>> usb2 at uhci1: USB revision 1.0 >>> uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1 >>> isa0 at pcib0 >>> isadma0 at isa0 >>> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo >>> pckbc0 at isa0 port 0x60/5 >>> pckbd0 at pckbc0 (kbd slot) >>> pckbc0: using irq 1 for kbd slot >>> wskbd0 at pckbd0: console keyboard, using wsdisplay0 >>> pcppi0 at isa0 port 0x61 >>> spkr0 at pcppi0 >>> mtrr: Pentium Pro MTRR support >>> vscsi0 at root >>> scsibus2 at vscsi0: 256 targets >>> softraid0 at root >>> scsibus3 at softraid0: 256 targets >>> root on wd0a (a29928cba946c858.a) swap on wd0b dump on wd0b >>> >>> (L-VPN) >>> OpenBSD 5.1 (GENERIC.MP) #207: Sun Feb 12 09:42:14 MST 2012 >>> [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP >>> real mem = 3219914752 (3070MB) >>> avail mem = 3120099328 (2975MB) >>> mainbus0 at root >>> bios0 at mainbus0: SMBIOS rev. 2.3 @ 0xfa850 (75 entries) >>> bios0: vendor Dell Computer Corporation version "A03" date 01/04/2006 >>> bios0: Dell Computer Corporation PowerEdge SC1425 >>> acpi0 at bios0: rev 0 >>> acpi0: sleep states S0 S4 S5 >>> acpi0: tables DSDT FACP APIC SPCR HPET MCFG >>> acpi0: wakeup devices PCI0(S5) PALO(S5) PXH_(S5) PXHB(S5) PXHA(S5) >>> PICH(S5) >>> acpitimer0 at acpi0: 3579545 Hz, 24 bits >>> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat >>> cpu0 at mainbus0: apid 0 (boot processor) >>> cpu0: Intel(R) Xeon(TM) CPU 2.80GHz, 2800.45 MHz >>> cpu0: >>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR,NXE,LONG >>> cpu0: 1MB 64b/line 8-way L2 cache >>> cpu0: apic clock running at 200MHz >>> cpu1 at mainbus0: apid 1 (application processor) >>> cpu1: Intel(R) Xeon(TM) CPU 2.80GHz, 2800.11 MHz >>> cpu1: >>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR,NXE,LONG >>> cpu1: 1MB 64b/line 8-way L2 cache >>> ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins >>> ioapic0: misconfigured as apic 0, remapped to apid 2 >>> ioapic1 at mainbus0: apid 3 pa 0xfec80000, version 20, 24 pins >>> ioapic1: misconfigured as apic 0, remapped to apid 3 >>> ioapic2 at mainbus0: apid 4 pa 0xfec80800, version 20, 24 pins >>> ioapic2: misconfigured as apic 0, remapped to apid 4 >>> acpihpet0 at acpi0: 14318179 Hz >>> acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255 >>> acpiprt0 at acpi0: bus 0 (PCI0) >>> acpiprt1 at acpi0: bus 1 (PALO) >>> acpiprt2 at acpi0: bus 3 (PXHB) >>> acpiprt3 at acpi0: bus 2 (PXHA) >>> acpiprt4 at acpi0: bus 4 (PICH) >>> acpicpu0 at acpi0 >>> acpicpu1 at acpi0 >>> ipmi at mainbus0 not configured >>> pci0 at mainbus0 bus 0 >>> pchb0 at pci0 dev 0 function 0 "Intel E7520 Host" rev 0x09 >>> ppb0 at pci0 dev 2 function 0 "Intel E7520 PCIE" rev 0x09 >>> pci1 at ppb0 bus 1 >>> ppb1 at pci1 dev 0 function 0 "Intel 6700PXH PCIE-PCIX" rev 0x09 >>> pci2 at ppb1 bus 2 >>> em0 at pci2 dev 4 function 0 "Intel PRO/1000MT (82541GI)" rev 0x05: >>> apic 3 int 0, address 00:14:22:72:5e:bd >>> ppb2 at pci1 dev 0 function 2 "Intel 6700PXH PCIE-PCIX" rev 0x09 >>> pci3 at ppb2 bus 3 >>> em1 at pci3 dev 7 function 0 "Intel PRO/1000MT (82546GB)" rev 0x03: >>> apic 4 int 2, address 00:04:23:ce:d0:0c >>> em2 at pci3 dev 7 function 1 "Intel PRO/1000MT (82546GB)" rev 0x03: >>> apic 4 int 3, address 00:04:23:ce:d0:0d >>> uhci0 at pci0 dev 29 function 0 "Intel 82801EB/ER USB" rev 0x02: apic 2 >>> int 16 >>> uhci1 at pci0 dev 29 function 1 "Intel 82801EB/ER USB" rev 0x02: apic 2 >>> int 19 >>> ehci0 at pci0 dev 29 function 7 "Intel 82801EB/ER USB2" rev 0x02: apic 2 >>> int 23 >>> usb0 at ehci0: USB revision 2.0 >>> uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 >>> ppb3 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xc2 >>> pci4 at ppb3 bus 4 >>> em3 at pci4 dev 3 function 0 "Intel PRO/1000MT (82541GI)" rev 0x05: >>> apic 2 int 20, address 00:14:22:72:5e:be >>> vga1 at pci4 dev 13 function 0 "ATI Radeon VE" rev 0x00 >>> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) >>> wsdisplay0: screen 1-5 added (80x25, vt100 emulation) >>> radeondrm0 at vga1: apic 2 int 17 >>> drm0 at radeondrm0 >>> pcib0 at pci0 dev 31 function 0 "Intel 82801EB/ER LPC" rev 0x02 >>> pciide0 at pci0 dev 31 function 1 "Intel 82801EB/ER IDE" rev 0x02: >>> DMA, channel 0 configured to compatibility, channel 1 configured to >>> compatibility >>> atapiscsi0 at pciide0 channel 0 drive 0 >>> scsibus0 at atapiscsi0: 2 targets >>> cd0 at scsibus0 targ 0 lun 0: <HL-DT-ST, CD-ROM GCR-8240N, 1.06> ATAPI >>> 5/cdrom removable >>> cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2 >>> pciide0: channel 1 ignored (disabled) >>> pciide1 at pci0 dev 31 function 2 "Intel 82801EB SATA" rev 0x02: DMA, >>> channel 0 configured to native-PCI, channel 1 configured to native-PCI >>> pciide1: using apic 2 int 18 for native-PCI interrupt >>> wd0 at pciide1 channel 0 drive 0: <WDC WD400BD-75LRA0> >>> wd0: 16-sector PIO, LBA48, 38146MB, 78125000 sectors >>> wd0(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 6 >>> wd1 at pciide1 channel 1 drive 0: <Maxtor 7Y250M0> >>> wd1: 16-sector PIO, LBA48, 238418MB, 488281250 sectors >>> wd1(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 6 >>> usb1 at uhci0: USB revision 1.0 >>> uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1 >>> usb2 at uhci1: USB revision 1.0 >>> uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1 >>> isa0 at pcib0 >>> isadma0 at isa0 >>> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo >>> pckbc0 at isa0 port 0x60/5 >>> pckbd0 at pckbc0 (kbd slot) >>> pckbc0: using irq 1 for kbd slot >>> wskbd0 at pckbd0: console keyboard, using wsdisplay0 >>> pcppi0 at isa0 port 0x61 >>> spkr0 at pcppi0 >>> mtrr: Pentium Pro MTRR support >>> vscsi0 at root >>> scsibus1 at vscsi0: 256 targets >>> softraid0 at root >>> scsibus2 at softraid0: 256 targets >>> root on wd0a (c66c13b9ce71dcfc.a) swap on wd0b dump on wd0b >>> >>> >>> `ifconfig` (for the sake of security, G.G.G.G is the public IP for >>> G-VPN where L.L.L.L is the public IP for L-VPN) >>> >>> (G-VPN) >>> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33152 >>> priority: 0 >>> groups: lo >>> inet6 ::1 prefixlen 128 >>> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 >>> inet 127.0.0.1 netmask 0xff000000 >>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 >>> lladdr 00:14:22:72:61:c6 >>> priority: 0 >>> media: Ethernet autoselect (1000baseT full-duplex) >>> status: active >>> inet 10.1.50.181 netmask 0xffffff00 broadcast 10.1.50.255 >>> inet6 fe80::214:22ff:fe72:61c6%em0 prefixlen 64 scopeid 0x1 >>> em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 >>> lladdr 00:14:22:72:61:c7 >>> priority: 0 >>> groups: egress >>> media: Ethernet autoselect (1000baseT full-duplex) >>> status: active >>> inet G.G.G.G netmask 0xfffffff0 broadcast G.G.G.X >>> inet6 fe80::214:22ff:fe72:61c7%em1 prefixlen 64 scopeid 0x2 >>> enc0: flags=0<> >>> priority: 0 >>> groups: enc >>> status: active >>> pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33152 >>> priority: 0 >>> groups: pflog >>> >>> (L-VPN) >>> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33152 >>> priority: 0 >>> groups: lo >>> inet6 ::1 prefixlen 128 >>> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6 >>> inet 127.0.0.1 netmask 0xff000000 >>> em0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> >>> mtu 1500 >>> lladdr 00:14:22:72:5e:bd >>> priority: 0 >>> trunk: trunkdev trunk0 >>> media: Ethernet autoselect (1000baseT full-duplex) >>> status: active >>> inet6 fe80::204:23ff:fece:d00c%em0 prefixlen 64 scopeid 0x1 >>> em1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> >>> mtu 1500 >>> lladdr 00:14:22:72:5e:bd >>> priority: 0 >>> trunk: trunkdev trunk0 >>> media: Ethernet autoselect (1000baseT full-duplex) >>> status: active >>> inet6 fe80::204:23ff:fece:d00d%em1 prefixlen 64 scopeid 0x2 >>> em2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> >>> mtu 1500 >>> lladdr 00:04:23:ce:d0:0d >>> priority: 0 >>> trunk: trunkdev trunk1 >>> media: Ethernet autoselect (1000baseT full-duplex) >>> status: active >>> inet6 fe80::214:22ff:fe72:5ebe%em2 prefixlen 64 scopeid 0x3 >>> em3: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> >>> mtu 1500 >>> lladdr 00:04:23:ce:d0:0d >>> priority: 0 >>> trunk: trunkdev trunk1 >>> media: Ethernet autoselect (1000baseT full-duplex) >>> status: active >>> inet6 fe80::214:22ff:fe72:5ebd%em3 prefixlen 64 scopeid 0x4 >>> enc0: flags=0<> >>> priority: 0 >>> groups: enc >>> status: active >>> trunk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 >>> lladdr 00:14:22:72:5e:bd >>> priority: 0 >>> trunk: trunkproto lacp >>> trunk id: [(8000,00:14:22:72:5e:bd,403C,0000,0000), >>> (8000,00:23:05:1d:fb:80,000C,0000,0000)] >>> trunkport em1 active,collecting,distributing >>> trunkport em0 collecting,distributing >>> groups: trunk >>> media: Ethernet autoselect >>> status: active >>> inet6 fe80::214:22ff:fe72:5ebd%trunk0 prefixlen 64 scopeid 0x7 >>> trunk1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 >>> lladdr 00:04:23:ce:d0:0d >>> priority: 0 >>> trunk: trunkproto lacp >>> trunk id: [(8000,00:04:23:ce:d0:0d,4044,0000,0000), >>> (8000,00:23:05:3f:19:80,0010,0000,0000)] >>> trunkport em3 active,collecting,distributing >>> trunkport em2 collecting,distributing >>> groups: trunk >>> media: Ethernet autoselect >>> status: active >>> inet6 fe80::204:23ff:fece:d00d%trunk1 prefixlen 64 scopeid 0x8 >>> vlan10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 >>> lladdr 00:14:22:72:5e:bd >>> priority: 0 >>> vlan: 10 parent interface: trunk0 >>> groups: vlan egress >>> status: active >>> inet6 fe80::214:22ff:fe72:5ebd%vlan10 prefixlen 64 scopeid 0x9 >>> inet L.L.L.L netmask 0xfffffff8 broadcast L.L.L.X >>> vlan20: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 >>> lladdr 00:04:23:ce:d0:0d >>> priority: 0 >>> vlan: 20 parent interface: trunk1 >>> groups: vlan >>> status: active >>> inet6 fe80::204:23ff:fece:d00d%vlan20 prefixlen 64 scopeid 0xa >>> inet 10.240.2.169 netmask 0xffffff00 broadcast 10.240.2.255 >>> vlan30: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 >>> lladdr 00:04:23:ce:d0:0d >>> priority: 0 >>> vlan: 30 parent interface: trunk1 >>> groups: vlan >>> status: active >>> inet6 fe80::204:23ff:fece:d00d%vlan30 prefixlen 64 scopeid 0xb >>> inet 10.240.3.169 netmask 0xffffff00 broadcast 10.240.3.255 >>> vlan40: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 >>> lladdr 00:14:22:72:5e:bd >>> priority: 0 >>> vlan: 40 parent interface: trunk0 >>> groups: vlan >>> status: active >>> inet6 fe80::214:22ff:fe72:5ebd%vlan40 prefixlen 64 scopeid 0xc >>> inet 10.240.4.169 netmask 0xffffff00 broadcast 10.240.4.255 >>> pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33152 >>> priority: 0 >>> groups: pflog >>> >>> >>> `cat /etc/pf.conf` >>> >>> (G-VPN) >>> >>> int_if="em0" >>> ext_if="em1" >>> >>> remote_gw="L.L.L.L" >>> >>> admins_net="{ 10.17.6.0/24, 10.32.24.0/24 }" >>> devs_net="{ 10.1.2.0/24, 10.17.8.0/24 }" >>> >>> L_databases="{ 10.240.4.111, 10.240.4.112, 10.240.4.121, 10.240.4.122, >>> 10.240.4.131, 10.240.4.132 }" >>> G_databases="{ 10.1.50.121, 10.1.50.122 }" >>> >>> set skip on { lo enc0 } >>> >>> table <authpf_users> persist >>> >>> block >>> >>> # VPN >>> pass in quick on $ext_if proto esp from $remote_gw to $ext_if >>> pass out quick on $ext_if proto esp from $ext_if to $remote_gw >>> >>> pass in quick on $ext_if proto udp from $remote_gw to $ext_if port { >>> isakmp, ipsec-nat-t } >>> pass out quick on $ext_if proto udp from $ext_if to $remote_gw port { >>> isakmp, ipsec-nat-t } >>> >>> # DNS/NTP/SSH >>> pass out quick on $int_if proto udp to port domain >>> pass out quick on $int_if proto udp to port ntp >>> pass in quick on $int_if proto tcp to 10.1.50.181 port ssh >>> >>> # TRAFFIC >>> pass in on $int_if proto tcp from { 10.1.50.11, $devs_net } to >>> 10.240.4.21 port ssh >>> pass out on $ext_if proto tcp from { 10.1.50.11, $devs_net } to >>> 10.240.4.21 port ssh >>> >>> pass in on $int_if proto tcp from { $devs_net, $G_databases } to >>> $L_databases port 1521 >>> pass out on $int_if proto tcp from { $devs_net, $G_databases } to >>> $L_databases port 1521 >>> >>> pass in on $ext_if proto tcp from $L_databases to $G_databases port 1521 >>> pass out on $int_if proto tcp from $L_databases to $G_databases port 1521 >>> >>> pass in on $int_if from <authpf_users> >>> pass out on $ext_if from <authpf_users> >>> >>> (L-VPN) >>> ext_if="vlan10" >>> >>> remote_gw="G.G.G.G" >>> >>> admins_net="{ 10.17.6.0/24, 10.32.24.0/24 }" >>> devs_net="{ 10.1.2.0/24, 10.17.8.0/24 }" >>> >>> L_databases="{ 10.240.4.111, 10.240.4.112, 10.240.4.121, 10.240.4.122, >>> 10.240.4.131, 10.240.4.132 }" >>> G_databases="{ 10.1.50.121, 10.1.50.122 }" >>> >>> set skip on { lo enc0 } >>> >>> block >>> >>> # VPN >>> pass in quick on $ext_if proto esp from $remote_gw to $ext_if >>> pass out quick on $ext_if proto esp from $ext_if to $remote_gw >>> >>> pass in quick on $ext_if proto udp from $remote_gw to $ext_if port { >>> isakmp, ipsec-nat-t } >>> pass out quick on $ext_if proto udp from $ext_if to $remote_gw port { >>> isakmp, ipsec-nat-t } >>> >>> # DNS/NTP/SSH >>> pass out quick on $ext_if proto udp to port domain >>> pass out quick on $ext_if proto udp to port ntp >>> pass in quick on vlan20 proto tcp to 10.240.2.169 port ssh >>> >>> # TRAFFIC >>> pass in on vlan10 from $admins_net >>> pass out on { vlan20, vlan30, vlan40 } from $admins_net >>> >>> pass in on vlan10 proto tcp from { 10.1.50.11, $devs_net } to >>> 10.240.4.21 port 22 >>> pass out on vlan40 proto tcp from { 10.1.50.11, $devs_net } to >>> 10.240.4.21 port 22 >>> >>> pass in on vlan10 proto tcp from { $devs_net, $G_databases } to >>> $L_databases port 1521 >>> pass out on vlan40 proto tcp from { $devs_net, $G_databases } to >>> $L_databases port 1521 >>> >>> pass in on vlan40 proto tcp from $L_databases to $G_databases port 1521 >>> pass out on vlan10 proto tcp from $L_databases to $G_databases port 1521 >>> >>> pass in on vlan40 proto tcp from 10.1.50.181 to 10.240.2.169 >>> pass out on vlan20 proto tcp from 10.1.50.181 to 10.240.2.169 >>> >>> >>> `cat /etc/ipsec.conf` >>> >>> (G-VPN) >>> local_ip="G.G.G.G" >>> local_net="{ 10.1.2.0/24, 10.1.50.0/24, 10.17.6.0/24, 10.17.8.0/24, >>> 10.32.24.0/24 }" >>> remote_ip="L.L.L.L" >>> remote_net="{ 10.240.2.0/24, 10.240.3.0/24, 10.240.4.0/24 }" >>> >>> ike esp from $local_net to $remote_net peer $remote_ip >>> ike esp from $local_ip to $remote_net peer $remote_ip >>> ike esp from $local_ip to $remote_ip >>> >>> >>> (L-VPN) >>> local_ip="L.L.L.L" >>> local_net="{ 10.240.2.0/24, 10.240.3.0/24, 10.240.4.0/24 }" >>> remote_ip="G.G.G.G" >>> remote_net="{ 10.1.2.0/24, 10.1.50.0/24, 10.17.6.0/24, 10.17.8.0/24, >>> 10.32.24.0/24 }" >>> >>> ike esp from $local_net to $remote_net peer $remote_ip >>> ike esp from $local_ip to $remote_net peer $remote_ip >>> ike esp from $local_ip to $remote_ip >>> >>> ----------- ENDPOINT INFO ----------- >>> >>> >>> Both endpoints run stock OpenBSD 5.1 (amd64). We use the VPN link to >>> manage our platform remotely and perform daily backups. G-VPN runs on >>> a 150Mbit/s link while L-VPN on a 1Gbit/s link. On one hand, our VPN >>> setup runs really nicely. The connections are routed properly, pf is >>> godsent and authpf works wonders. On the other hand, network >>> throughput over the VPN tunnel never exceeds 3.4MB/s (ftp, scp, rsync, >>> etc...) >>> >>> I welcome any suggestions. Keep in mind that this is our production >>> VPN tunnel, so I cannot shut it down at will. Thanks in advance. >>> >>> --- >>> Mike
