Re: Xen PV DomU with OpenBSD?
On 21 February 2015 at 10:31, Markus Kolb open...@tower-net.de wrote: there isn't any support for Xen PV DomU in OpenBSD, isn't it? No, there is not such support. But you can run it in HVM mode without effort. Well, may be some effort in XenServer, where there is no easy way to chose the type of emulated hardware. Another problem when using Xen: the shutdown. Every OS that can not communicate with xenstore will suffer from that. You will have to edit some scripts in your environment to make it work with ACPI. Best regards, Raimundo Santos
Re: OpenBSD Tablet-ish
Hello, Lenovo Thinkpad x201 works well for me. On 19 February 2015 at 17:15, Jack Woehr jwo...@softwoehr.com wrote: What's the smallest, most tablet-ish device I can put OpenBSD on? Want to travel and stay connected. -- Jack Woehr # There's too much emphasis on things Box 51, Golden CO 80402 # like pawn structure in modern chess. http://www.softwoehr.com # Checkmate ends the game. - N. Shor
Re: OpenBSD Tablet-ish
On 19 February 2015 at 21:19, Carl Trachte ctrac...@gmail.com wrote: It's definitely not a tablet, but it's way more portable than a desktop or full sized laptop. Surprisingly, for it's size, it's easy to take apart and deal with. It is my choice because ot its little weight: most of it is my 5 hours battery. But it is not that new: a first generation core i5, AFAIK. Besides that, it is a very good hardware and interacts well with OpenBSD. Best regards. Raimundo Santos
Re: unbound problem in 5.6
Thank you very much, Otto! Almost one day of test after configuring _unbound class, and no more Too many open files. Once more, thank you for your time, and happy new year. Raimundo Santos On 30 December 2014 at 11:14, Otto Moerbeek o...@drijf.net wrote: On Tue, Dec 30, 2014 at 11:09:44AM -0200, Raimundo Santos wrote: Hello misc@! I have a router (peaking at 70Mbps of aggregated traffic) that acts as a recursive internal DNS server too (this configuration will die soon, as my traffic is growing), but Unbound keep saying, in /var/log/messages: Dec 30 09:57:07 myhost unbound: [3873:0] error: can't create socket: Too many open files Dec 30 09:57:08 myhost last message repeated 20284 times Dec 30 10:26:48 myhost unbound: [3873:0] error: can't create socket: Too many open files Dec 30 10:26:50 myhost last message repeated 24896 times Sometimes it says: Dec 27 21:49:19 myhost unbound: [2565:0] notice: sendto failed: No buffer space available I have: kern.maxfiles=16384 kern.somaxconn=16384 And in login.conf: daemon:\ :ignorenologin:\ :datasize=infinity:\ :maxproc=infinity:\ :openfiles-cur=4096:\ :openfiles-max=8192:\ :stacksize-cur=8M:\ :localcipher=blowfish,9:\ :tc=default: unbound:\ :ignorenologin:\ :datasize=infinity:\ :maxproc=infinity:\ :openfiles-cur=8192:\ :openfiles-max=16384:\ :stacksize-cur=32M:\ :localcipher=blowfish,9:\ :tc=default: With many resources just for Unbound, how can it keep complaining? There's an undocumented feature with unbound: it (only) sets its resource limits based on the class of its user (_unbound by default). So set the class of the _unbound user to unbound and you're all set. -Otto Thandk you in advance, and happy new year! Raimundo Santos -- Here are some more info... # systat -B mbufs 1 usersLoad 0.16 0.12 0.09 Tue Dec 30 11:02:00 2014 IFACE LIVELOCKS SIZE ALIVE LWM HWM CWM System0 256 194 93 2048 184 442 lo0 em0 20489010 25690 xl0 re0 re1 enc0 pflog0 ...and the dmesg: OpenBSD 5.6 (GENERIC) #310: Fri Aug 8 00:14:24 MDT 2014 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC real mem = 8502595584 (8108MB) avail mem = 8267517952 (7884MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xe84b0 (59 entries) bios0: vendor Intel Corp. version NBG4310H.86A.0051.2008.0804.1738 date 08/04/2008 bios0: Intel Corporation DP43TF acpi0 at bios0: rev 2couldn't map rsdt , can't load tables cpu0 at mainbus0: (uniprocessor) cpu0: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 2394.30 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF cpu0: 4MB 64b/line 16-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 7 var ranges, 88 fixed ranges pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 Intel G45 Host rev 0x03 ppb0 at pci0 dev 1 function 0 Intel G45 PCIE rev 0x03: irq 11 pci1 at ppb0 bus 1 vga1 at pci1 dev 0 function 0 NVIDIA GeForce 8600 GT rev 0xa1 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) vendor Intel, unknown product 0x2e24 (class communications subclass miscellaneous, rev 0x03) at pci0 dev 3 function 0 not configured em0 at pci0 dev 25 function 0 Intel ICH10 R BM V rev 0x00: irq 10, address 00:1c:c0:91:9f:c3 uhci0 at pci0 dev 26 function 0 Intel 82801JI USB rev 0x00: irq 11 uhci1 at pci0 dev 26 function 1 Intel 82801JI USB rev 0x00: irq 5 uhci2 at pci0 dev 26 function 2 Intel 82801JI USB rev 0x00: irq 3 ehci0 at pci0 dev 26 function 7 Intel 82801JI USB rev 0x00: irq 3 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb1 at pci0 dev 28 function 0 Intel 82801JI PCIE rev 0x00: irq 7 pci2 at ppb1 bus 2 ppb2 at pci0 dev 28 function 3 Intel 82801JI PCIE rev 0x00: irq 4 pci3 at ppb2 bus 3 jmb0 at pci3 dev 0 function 0 JMicron JMB368 IDE rev 0x00 pciide0 at jmb0: DMA, channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide0: using irq 4 for native-PCI interrupt pciide0: channel 0 disabled (no drives) pciide0: channel 1 disabled (no drives) uhci3 at pci0 dev 29 function 0 Intel 82801JI USB rev 0x00: irq 11 uhci4 at pci0 dev 29 function 1 Intel 82801JI USB rev 0x00: irq 4 uhci5 at pci0 dev 29 function 2 Intel 82801JI USB rev 0x00: irq 3 ehci1 at pci0 dev 29
unbound problem in 5.6
Hello misc@! I have a router (peaking at 70Mbps of aggregated traffic) that acts as a recursive internal DNS server too (this configuration will die soon, as my traffic is growing), but Unbound keep saying, in /var/log/messages: Dec 30 09:57:07 myhost unbound: [3873:0] error: can't create socket: Too many open files Dec 30 09:57:08 myhost last message repeated 20284 times Dec 30 10:26:48 myhost unbound: [3873:0] error: can't create socket: Too many open files Dec 30 10:26:50 myhost last message repeated 24896 times Sometimes it says: Dec 27 21:49:19 myhost unbound: [2565:0] notice: sendto failed: No buffer space available I have: kern.maxfiles=16384 kern.somaxconn=16384 And in login.conf: daemon:\ :ignorenologin:\ :datasize=infinity:\ :maxproc=infinity:\ :openfiles-cur=4096:\ :openfiles-max=8192:\ :stacksize-cur=8M:\ :localcipher=blowfish,9:\ :tc=default: unbound:\ :ignorenologin:\ :datasize=infinity:\ :maxproc=infinity:\ :openfiles-cur=8192:\ :openfiles-max=16384:\ :stacksize-cur=32M:\ :localcipher=blowfish,9:\ :tc=default: With many resources just for Unbound, how can it keep complaining? Thandk you in advance, and happy new year! Raimundo Santos -- Here are some more info... # systat -B mbufs 1 usersLoad 0.16 0.12 0.09 Tue Dec 30 11:02:00 2014 IFACE LIVELOCKS SIZE ALIVE LWM HWM CWM System0 256 194 93 2048 184 442 lo0 em0 20489010 25690 xl0 re0 re1 enc0 pflog0 ...and the dmesg: OpenBSD 5.6 (GENERIC) #310: Fri Aug 8 00:14:24 MDT 2014 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC real mem = 8502595584 (8108MB) avail mem = 8267517952 (7884MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xe84b0 (59 entries) bios0: vendor Intel Corp. version NBG4310H.86A.0051.2008.0804.1738 date 08/04/2008 bios0: Intel Corporation DP43TF acpi0 at bios0: rev 2couldn't map rsdt , can't load tables cpu0 at mainbus0: (uniprocessor) cpu0: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 2394.30 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF cpu0: 4MB 64b/line 16-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 7 var ranges, 88 fixed ranges pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 Intel G45 Host rev 0x03 ppb0 at pci0 dev 1 function 0 Intel G45 PCIE rev 0x03: irq 11 pci1 at ppb0 bus 1 vga1 at pci1 dev 0 function 0 NVIDIA GeForce 8600 GT rev 0xa1 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) vendor Intel, unknown product 0x2e24 (class communications subclass miscellaneous, rev 0x03) at pci0 dev 3 function 0 not configured em0 at pci0 dev 25 function 0 Intel ICH10 R BM V rev 0x00: irq 10, address 00:1c:c0:91:9f:c3 uhci0 at pci0 dev 26 function 0 Intel 82801JI USB rev 0x00: irq 11 uhci1 at pci0 dev 26 function 1 Intel 82801JI USB rev 0x00: irq 5 uhci2 at pci0 dev 26 function 2 Intel 82801JI USB rev 0x00: irq 3 ehci0 at pci0 dev 26 function 7 Intel 82801JI USB rev 0x00: irq 3 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb1 at pci0 dev 28 function 0 Intel 82801JI PCIE rev 0x00: irq 7 pci2 at ppb1 bus 2 ppb2 at pci0 dev 28 function 3 Intel 82801JI PCIE rev 0x00: irq 4 pci3 at ppb2 bus 3 jmb0 at pci3 dev 0 function 0 JMicron JMB368 IDE rev 0x00 pciide0 at jmb0: DMA, channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide0: using irq 4 for native-PCI interrupt pciide0: channel 0 disabled (no drives) pciide0: channel 1 disabled (no drives) uhci3 at pci0 dev 29 function 0 Intel 82801JI USB rev 0x00: irq 11 uhci4 at pci0 dev 29 function 1 Intel 82801JI USB rev 0x00: irq 4 uhci5 at pci0 dev 29 function 2 Intel 82801JI USB rev 0x00: irq 3 ehci1 at pci0 dev 29 function 7 Intel 82801JI USB rev 0x00: irq 11 ehci1: timed out waiting for BIOS usb1 at ehci1: USB revision 2.0 uhub1 at usb1 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb3 at pci0 dev 30 function 0 Intel 82801BA Hub-to-PCI rev 0x90 pci4 at ppb3 bus 4 xl0 at pci4 dev 0 function 0 3Com 3c905C 100Base-TX rev 0x74: irq 10, address 00:50:da:0d:d8:b9 bmtphy0 at xl0 phy 24: 3C905C internal PHY, rev. 6 re0 at pci4 dev 2 function 0 Realtek 8169 rev 0x10: RTL8169/8110SB (0x1000), irq 5, address 00:0c:76:86:02:37 rgephy0 at re0 phy 7: RTL8169S/8110S PHY, rev. 3 re1 at pci4 dev 4 function 0 Realtek 8169 rev 0x10: RTL8169/8110SB (0x1000), irq 10, address 00:0c:76:87:03:23 rgephy1 at re1 phy 7: RTL8169S/8110S PHY, rev. 3 pcib0 at pci0 dev 31 function 0 Intel 82801JIB LPC rev 0x00 ahci0 at pci0 dev 31
Re: NetMap in OpenBSD
Sorry, replied to fast and to OP only. Below is one use case and a lot o things that Henning have said, put from my point of view. -- Forwarded message -- From: Raimundo Santos rait...@gmail.com Date: 14 October 2014 15:02 Subject: Re: NetMap in OpenBSD To: Mikael mikael.tr...@gmail.com On 14 October 2014 11:33, Mikael mikael.tr...@gmail.com wrote: userland reimplementing the stack[...] I didn't necessarily/specifically suggest that. The only case I can see to not reimplement full stack is working on pure Ethernet. All other really nice one can do with TCP/IP are sadly going to be reimplemented. This is how netmap works, barely: put packets in ring buffers, bypassing all the neat work of years in the OS network stack. How do you route a packet within netmap logic? How do you check for source or destiny addresses or TCP/UDP ports? You need to reimplement it on your own program, and do that for EVERY program using netmap. There is a whole world of need of network monitoring and manipulation and other specialized networking software. I read a collection of buzzwords with nothing specific. A solution in dire need of a problem. Here I see the limit of a general purpose OS. Well, lets deal with all the corner cases, and all the possibilities, and lets create a general purpose OS that is a specific purpose for everyone who uses it. Makes no sense to me. Specific needs that are not covered by the general facilities of such an OS must be covered by specific work of who needs it. You can even make a profitable product of this work. :) Bypass years of correct and conscious work to make all the stack more secure just because the needs of a few are for speed? It is a bad choice. netmap have one thing that really interests me: the ability to enforce specific per-ip bandwidth with dummynet, but at the cost of doing this with netmap-ipfw, reimplementing all the needed stack parts. Why, my sacred believes, WHY?! So, instead of improving that stack to do a free for all, correct and conscious speed up, lets do it by reimplementing the needed parts in every application. sosplice(9) may serve us with a starting point to that really fast things of zero-copy hype. http://www.openbsd.org/papers/eurobsdcon_2013_sosplice-slides.pdf Summarizing: netmap bypasses ALL the OS network stack. Period. Therefore, you must reimplement such things. Best regards, Raimundo Santos
OT: SiLK, libfixbuf and GPLR - Government Purpose License Rights
Hello, I was to begin tests with FlowViewer ( http://sourceforge.net/projects/flowviewer/), which needs SiLK, which, in turns, needs libfixbuf, both from NetSA/CERT: http://tools.netsa.cert.org Are there anyone using these softwares? I am able to download only if I accept GPLv2 (or LGPLv2) and GPLR, but I do not understand this line: Government Purpose License Rights (GPLR) pursuant to DFARS 252.227.7013 I could not find good information sources about it. So, I ask: 1 - These kind of licences are meaningful by a Brazilian (i.e., USA's outsiders) point of view? 2 - What is the OpenBSD Project judgment about GPLR/DFARS? (Do not know exactly how to name it.) 3 - Where can I find more information about GPLR (DFARS?)? Thank you very much for your time on this, Raimundo Santos
Re: OT: SiLK, libfixbuf and GPLR - Government Purpose License Rights
On 28 September 2014 04:13, Ted Unangst t...@tedunangst.com wrote: You are not the government, so instead the software is available to you under the terms of the LGPL. Thank you for the clarification. I got that it has nothing to do with GPL or FSF at all just reading the name: it is pretty clear, and scary in the first sight. Once again: thank you for your time, Raimundo Santos
Re: Anyone running Zabbix server/agent with success?
Hello again.
Problem solved!
Sorry to not depict better the situation at first mail, but here are the
solution: just follow closely the indications under
/usr/local/share/doc/pkg-readmes/{postgres*,zabbix*}
And that is it. In my case, a very important information was about running
PostgreSQL as a backend data base system, which needs to play a little with
IPC via shared memory. Specifically, here are my changes...
...regarding sysctl.conf:
kern.shminfo.shmall=524288
kern.seminfo.semmni=240
kern.seminfo.semmns=4096
(Caution: this machine have lots of RAM to play with!)
...regarding login.conf:
#
# From zabbix-server package indications
#
zabbix_server:\
:openfiles-cur=1024:\
:openfiles-max=2048:\
:tc=daemon:
#
# From PostgreSQL package indications
#
postgresql:\
:openfiles=2048:\
:tc=daemon:
Patrick, thank you for point out that log files needs, I just let it log to
/tmp and the problem became more clear.
Best regards,
Raimundo Santos
Re: Anyone running Zabbix server/agent with success?
On 21 September 2014 05:26, Patrick Ditzel patr...@central-computer.de wrote: Hello Reimundo, I tried do install zabbix from the ports on Sparc64 but this does not work. Yet I have not try to figure out why (maybe next week ...). I'm on amd64, and nothing working at all. If you just need to monitor your OpenBSD machine you can also use snmp. Thank you Patrick, but I need to monitor an entire network. For documentation: I also tried to indicate the configuration files manually: zabbix_server -c /etc/zabbix/zabbix_server.conf and got an error about permission denied over /var/log/zabbix_server.log, but just when I change the line to this file, otherwise the behaviour is the same, ungracefully exits. In the time of this writing, Zabbix packages are 2.2.1 for OpenBSD 5.5. Best regards, Raimundo Santos
Anyone running Zabbix server/agent with success?
Hello, I have installed Zabbix agent, server and web interface via pkg_add, and the only one that is working is web interface (nginx + php-fpm +postgres), reading data from backup. Anyone here is running zabbix server with success on OpenBSD 5.5? I am on a clean install. More details as needed. I try to /etc/rc.d/zabbix_agentd start /etc/rc.d/zabbix_server start and nothing happens, besides printing the famous (ok) for both tries. Manually trying does not work either, failing silently. I am in position to change zabbix for any other network monitoring tool that works well with OpenBSD. Best regards, Raimundo Santos
Re: pf queuing not limiting bandwidth
HI Loïc,
just setting max does not work for me. I reached my intent with
queue root on alc0 bandwidth 600M, min 100M, max 100M default
pass out on alc0 inet from any to 192.168.2.2 flags S/SA set ( queue root )
Thank you for that insight!
On 12 August 2014 04:10, Loïc Blot loic.b...@unix-experience.fr wrote:
Hi Raimundo,
please use max directive:
queue root on alc0 bandwidth 600M, max 500M
--
Best regards,
Loïc BLOT, Engineering
UNIX Systems, Security and Network Engineer
http://www.unix-experience.fr
Le mardi 12 août 2014 à 02:11 -0300, Raimundo Santos a écrit :
Hello misc!
I am with a very non expected behaviour. With this simple pf.conf
# pfctl -vnf /etc/pf.conf
set skip on { lo }
queue root on alc0 bandwidth 600M default
pass out on alc0 all flags S/SA set ( queue root )
I got this queue output when running tcpbench in client mode
# pfctl -vvvsq
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes:
0
]
[ qlength: 0/ 50 ]
queue root on alc0 bandwidth 600M default qlimit 50
[ pkts:6099167 bytes: 9233990662 dropped pkts: 0 bytes:
0
]
[ qlength: 0/ 50 ]
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes:
0
]
[ qlength: 0/ 50 ]
[ measured: 0.0 packets/s, 0 b/s ]
queue root on alc0 bandwidth 600M default qlimit 50
[ pkts:6500911 bytes: 9842225822 dropped pkts: 0 bytes:
0
]
[ qlength: 0/ 50 ]
[ measured: 80348.8 packets/s, 973.18Mb/s ]
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes:
0
]
[ qlength: 0/ 50 ]
[ measured: 0.0 packets/s, 0 b/s ]
queue root on alc0 bandwidth 600M default qlimit 50
[ pkts:6902593 bytes: 10450369962 dropped pkts: 0 bytes:
0 ]
[ qlength: 0/ 50 ]
[ measured: 80342.6 packets/s, 973.10Mb/s ]
# pfctl -vsr
pass out on alc0 all flags S/SA set ( queue root )
[ Evaluations: 493 Packets: 14082601 Bytes: 13949048492
States: 1
]
[ Inserted: uid 0 pid 3493 State Creations: 1 ]
I've tried with 100M, 200M and 400M, all not shaping.
I've also tried to setup a root queue with 200M and two child: a default
with 1M and the other, referred in the rule, with 100M, also not
working.
I am playing with tcpbench and this is the only traffic I really care
about
on this machine. I restarted the tcpbench client on this machine every
time
I reloaded the testing rule and queue, and even deleted the related
states
(or states, in cases that I run tcpbench -b some alias), but nothing
leads me to the desired bandwidth shaping.
I am experiencing the same behaviour in a virtual machine under KVM with
PCI Passthrough of an Intel NIC. These are the conf and results from the
virtual machine:
# pfctl -vf /etc/pf.conf
set skip on { lo }
queue std on em0 bandwidth 100M default
pass out on em0 all flags S/SA set ( queue std )
# pfctl -vvvsq
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes:
0
]
[ qlength: 0/ 50 ]
queue std on em0 bandwidth 100M default qlimit 50
[ pkts: 1195513815 bytes: 87858084628 dropped pkts: 0 bytes:
0 ]
[ qlength: 0/ 50 ]
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes:
0
]
[ qlength: 0/ 50 ]
[ measured: 0.0 packets/s, 0 b/s ]
queue std on em0 bandwidth 100M default qlimit 50
[ pkts: 1195734870 bytes: 88192747866 dropped pkts: 0 bytes:
0 ]
[ qlength: 0/ 50 ]
[ measured: 44211.0 packets/s, 535.46Mb/s ]
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes:
0
]
[ qlength: 0/ 50 ]
[ measured: 0.0 packets/s, 0 b/s ]
queue std on em0 bandwidth 100M default qlimit 50
[ pkts: 1195960995 bytes: 88535089028 dropped pkts: 0 bytes:
0 ]
[ qlength: 0/ 50 ]
[ measured: 44718.0 packets/s, 541.60Mb/s ]
# pfctl -vsr
pass out on em0 all flags S/SA set ( queue std )
[ Evaluations: 2 Packets: 1853414 Bytes: 1708817040
States: 2
]
[ Inserted: uid 0 pid 19622 State Creations: 2 ]
The traffic passes through a Linux box where I have per ip bandwitdh
control (justifying tcpbench -b alias), an in house bandwidth
controller
(poor man's 'net equalizer'). My intent was to not put a very high load
over this machine by getting close to my real pps and bps and so make my
capacity planing.
What am I doing wrong with these queues?
Thank you all,
Raimundo Santos
Here is my dmesgs, first from the physical machine and after from the
virtual machine:
OpenBSD 5.5 (GENERIC.MP) #315: Wed Mar 5 09:37:46 MST 2014
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8538095616 (8142MB)
avail mem = 8302202880 (7917MB)
mainbus0 at root
pf queuing not limiting bandwidth
Hello misc!
I am with a very non expected behaviour. With this simple pf.conf
# pfctl -vnf /etc/pf.conf
set skip on { lo }
queue root on alc0 bandwidth 600M default
pass out on alc0 all flags S/SA set ( queue root )
I got this queue output when running tcpbench in client mode
# pfctl -vvvsq
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0
]
[ qlength: 0/ 50 ]
queue root on alc0 bandwidth 600M default qlimit 50
[ pkts:6099167 bytes: 9233990662 dropped pkts: 0 bytes: 0
]
[ qlength: 0/ 50 ]
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0
]
[ qlength: 0/ 50 ]
[ measured: 0.0 packets/s, 0 b/s ]
queue root on alc0 bandwidth 600M default qlimit 50
[ pkts:6500911 bytes: 9842225822 dropped pkts: 0 bytes: 0
]
[ qlength: 0/ 50 ]
[ measured: 80348.8 packets/s, 973.18Mb/s ]
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0
]
[ qlength: 0/ 50 ]
[ measured: 0.0 packets/s, 0 b/s ]
queue root on alc0 bandwidth 600M default qlimit 50
[ pkts:6902593 bytes: 10450369962 dropped pkts: 0 bytes:
0 ]
[ qlength: 0/ 50 ]
[ measured: 80342.6 packets/s, 973.10Mb/s ]
# pfctl -vsr
pass out on alc0 all flags S/SA set ( queue root )
[ Evaluations: 493 Packets: 14082601 Bytes: 13949048492 States: 1
]
[ Inserted: uid 0 pid 3493 State Creations: 1 ]
I've tried with 100M, 200M and 400M, all not shaping.
I've also tried to setup a root queue with 200M and two child: a default
with 1M and the other, referred in the rule, with 100M, also not working.
I am playing with tcpbench and this is the only traffic I really care about
on this machine. I restarted the tcpbench client on this machine every time
I reloaded the testing rule and queue, and even deleted the related states
(or states, in cases that I run tcpbench -b some alias), but nothing
leads me to the desired bandwidth shaping.
I am experiencing the same behaviour in a virtual machine under KVM with
PCI Passthrough of an Intel NIC. These are the conf and results from the
virtual machine:
# pfctl -vf /etc/pf.conf
set skip on { lo }
queue std on em0 bandwidth 100M default
pass out on em0 all flags S/SA set ( queue std )
# pfctl -vvvsq
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0
]
[ qlength: 0/ 50 ]
queue std on em0 bandwidth 100M default qlimit 50
[ pkts: 1195513815 bytes: 87858084628 dropped pkts: 0 bytes:
0 ]
[ qlength: 0/ 50 ]
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0
]
[ qlength: 0/ 50 ]
[ measured: 0.0 packets/s, 0 b/s ]
queue std on em0 bandwidth 100M default qlimit 50
[ pkts: 1195734870 bytes: 88192747866 dropped pkts: 0 bytes:
0 ]
[ qlength: 0/ 50 ]
[ measured: 44211.0 packets/s, 535.46Mb/s ]
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0
]
[ qlength: 0/ 50 ]
[ measured: 0.0 packets/s, 0 b/s ]
queue std on em0 bandwidth 100M default qlimit 50
[ pkts: 1195960995 bytes: 88535089028 dropped pkts: 0 bytes:
0 ]
[ qlength: 0/ 50 ]
[ measured: 44718.0 packets/s, 541.60Mb/s ]
# pfctl -vsr
pass out on em0 all flags S/SA set ( queue std )
[ Evaluations: 2 Packets: 1853414 Bytes: 1708817040 States: 2
]
[ Inserted: uid 0 pid 19622 State Creations: 2 ]
The traffic passes through a Linux box where I have per ip bandwitdh
control (justifying tcpbench -b alias), an in house bandwidth controller
(poor man's 'net equalizer'). My intent was to not put a very high load
over this machine by getting close to my real pps and bps and so make my
capacity planing.
What am I doing wrong with these queues?
Thank you all,
Raimundo Santos
Here is my dmesgs, first from the physical machine and after from the
virtual machine:
OpenBSD 5.5 (GENERIC.MP) #315: Wed Mar 5 09:37:46 MST 2014
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8538095616 (8142MB)
avail mem = 8302202880 (7917MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0x9f400 (53 entries)
bios0: vendor American Megatrends Inc. version 0803 date 07/23/2012
bios0: ASUSTeK Computer INC. M4A78LT-M-LE
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S3 S4 S5
acpi0: tables DSDT FACP APIC MCFG OEMB SRAT HPET SSDT
acpi0: wakeup devices PCE2(S4) PCE3(S4) PCE4(S4) PCE5(S4) PCE6(S4) ALAN(S4)
PCE7(S4) PCE9(S4) PCEA(S4) PCEB(S4) PCEC(S4) SBAZ(S4) PS2M(S4) PS2K(S4)
UAR1(S4) P0PC(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Phenom(tm) II X2 565 Processor, 3415.72 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP
'ifconfigre bridge add' adding more than 256 ports
Hello, I am testing (playing with it, too) bridge capabilities in OpenBSD and with this sequence of commands: ifconfig bridge0 add et1 for i in `jot - 0 500 1`; do ifconfig vether$i create; ifconfig vether$i up; ifconfig bridge0 add vether$i; done ifconfig vether500 192.168.1.1/24 ifconfig bridge0 up from other machine connected to this one via et1, which has 192.168.1.2 as it's address, I can not ping 192.168.1.1 nor from .1 ping .2. If I change ifconfig vether500 192.168.1.1/24 to ifconfig vether200 192.168.1.1/24 ping works like a charm. I can see an overflow, counting from et1 to vether247, on bridge port number: ... 254 255 0 1 ... My dmesg (the last message repeated a lot of times), followed by 'ifconfig bridge0' output are at the end of message. Why ifconfig do not complain about adding more than 256 ports (as you can see from the ifconfig bridge0 output, some ports are not filled, like port number 6, in bridge0)? Thank you for your time! Raimundo Santos OpenBSD 5.5 (GENERIC.MP) #315: Wed Mar 5 09:37:46 MST 2014 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 8538095616 (8142MB) avail mem = 8302202880 (7917MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.5 @ 0x9f400 (53 entries) bios0: vendor American Megatrends Inc. version 0803 date 07/23/2012 bios0: ASUSTeK Computer INC. M4A78LT-M-LE acpi0 at bios0: rev 2 acpi0: sleep states S0 S1 S3 S4 S5 acpi0: tables DSDT FACP APIC MCFG OEMB SRAT HPET SSDT acpi0: wakeup devices PCE2(S4) PCE3(S4) PCE4(S4) PCE5(S4) PCE6(S4) ALAN(S4) PCE7(S4) PCE9(S4) PCEA(S4) PCEB(S4) PCEC(S4) SBAZ(S4) PS2M(S4) PS2K(S4) UAR1(S4) P0PC(S4) [...] acpitimer0 at acpi0: 3579545 Hz, 32 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: AMD Phenom(tm) II X2 565 Processor, 3415.62 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line 16-way L2 cache, 6MB 64b/line 48-way L3 cache cpu0: ITLB 32 4KB entries fully associative, 16 4MB entries fully associative cpu0: DTLB 48 4KB entries fully associative, 48 4MB entries fully associative cpu0: AMD erratum 721 detected and fixed cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 200MHz cpu0: mwait min=64, max=64, C-substates=0.0.0.0.0, IBE cpu1 at mainbus0: apid 1 (application processor) cpu1: AMD Phenom(tm) II X2 565 Processor, 3415.24 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC cpu1: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line 16-way L2 cache, 6MB 64b/line 48-way L3 cache cpu1: ITLB 32 4KB entries fully associative, 16 4MB entries fully associative cpu1: DTLB 48 4KB entries fully associative, 48 4MB entries fully associative cpu1: AMD erratum 721 detected and fixed cpu1: smt 0, core 1, package 0 ioapic0 at mainbus0: apid 2 pa 0xfec0, version 21, 24 pins acpimcfg0 at acpi0 addr 0xe000, bus 0-255 acpihpet0 at acpi0: 14318180 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (P0P1) acpiprt2 at acpi0: bus 2 (PCE2) acpiprt3 at acpi0: bus -1 (PCE3) acpiprt4 at acpi0: bus 3 (PCE4) acpiprt5 at acpi0: bus 4 (PCE6) acpiprt6 at acpi0: bus 5 (P0PC) acpicpu0 at acpi0: PSS acpicpu1 at acpi0: PSS aibs0 at acpi0 RTMP RVLT RFAN GGRP GITM SITM acpibtn0 at acpi0: PWRB cpu0: 3415 MHz: speeds: 3400 2700 2300 800 MHz pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 AMD RS780 Host rev 0x00 ppb0 at pci0 dev 1 function 0 vendor Asustek, unknown product 0x9602 rev 0x00 pci1 at ppb0 bus 1 radeondrm0 at pci1 dev 5 function 0 ATI Radeon HD 3000 rev 0x00 drm0 at radeondrm0 radeondrm0: apic 2 int 18 ppb1 at pci0 dev 2 function 0 AMD RS780 PCIE rev 0x00: msi pci2 at ppb1 bus 2 et0 at pci2 dev 0 function 0 ATT/Lucent ET1310 rev 0x02: apic 2 int 18, address 00:13:3b:03:07:1a etphy0 at et0 phy 0: ET1011 10/100/1000baseT PHY, rev. 2 ppb2 at pci0 dev 4 function 0 AMD RS780 PCIE rev 0x00: msi pci3 at ppb2 bus 3 et1 at pci3 dev 0 function 0 ATT/Lucent ET1310 rev 0x02: apic 2 int 16, address 00:13:3b:03:02:b9 etphy1 at et1 phy 0: ET1011 10/100/1000baseT PHY, rev. 2 ppb3 at pci0 dev 6 function 0 AMD RS780 PCIE rev 0x00: msi pci4 at ppb3 bus 4 alc0 at pci4 dev 0 function 0 Attansic Technology L1C rev 0xc0: msi, address f4:6d:04:00:8f:a1 atphy0 at alc0 phy 0: F1 10/100/1000 PHY, rev. 11 ahci0 at pci0 dev 17 function 0 ATI SBx00 SATA rev 0x00: apic 2 int 22, AHCI 1.1 scsibus0 at ahci0: 32 targets sd0 at scsibus0 targ 0 lun 0: ATA, ST3500418AS, CC34 SCSI3 0/direct fixed naa.5000c500120cd875 sd0: 476940MB
Re: Are nc -lu /dev/zero /dev/null a good throughput test?
On 21 July 2014 18:17, Giancarlo Razzolini grazzol...@gmail.com wrote: I've noticed similar performance and, in some cases, better than vio(4) when using the host's pci passthrough and assigning a real hardware to the VM. But Hello Giancarlo, thank you for your time. I am at a very bleeding edge (or awkward) project of putting almost all machines of a little WISP into a virtualized system. My concern mainly touches packets and bits flows, storage is not one. XenServer has very nice facilities, but is a pain to tailor it in network area (well, almost in all areas: lots of long commands which are hard to remember, tricks that could vanish with updates, ...). The amount of work to tune it is equal or more than to use libvirt, so I am dropping it. Ubuntu Server 14.04 came out with qemu-kvm 2.0.0, with newer host VirtIO implementations in many areas. I am on my way to test it. I dislike Ubuntu as a Server, but I am not in that project to take much pain to manage the hosts, compile that sadly GNU-crafted things and so on, therefore if Ubuntu give me good performance, I will take it. Can you tell me where are you using qemu-kvm 2.0.0 and how you manage it (upgrades, etc.)? you shouldn't expected very great performance between VM's hosted in the same host, unless you're using linux's macvtap with a switch that supports VEPA. Using bridge is slow. I suggest you create a virtual network and assign an interface for each of your VM's that need communicating, and also use vio(4) on the guest OS. As you stated before, I expect a lot more performance from PCI passthrough, and things like clients bandwidth enforcement will depend on it. I will try as match as possible to let that main traffic outside host internal networks. Have you played with Open vSwitch as a bridging facility? My client (the WISP) is very excited about turning off that old machines, but, while I am enjoying the challenge, am I too with three foot behind the line of excitement when the subject are reliability and scalability of the solution. Nonetheless, it is an experimental. And someone could think: why OpenBSD? Well, have you ever tried setting RIPv2 in other OSes? The more general answer: it Just Works for almost all things I need to setup. The only thing that I can not figure out how to do is the WISP's clients contracted bandwidth enforcement. Cheers, Raimundo Santos
Re: Are nc -lu /dev/zero /dev/null a good throughput test?
On 19 July 2014 21:22, Sean Kamath kam...@moltingpenguin.com wrote: Are you counting all those zeros to make sure they all came through? 'cause TCP is guaranteed delivery, in order. UDP guarantees nothing. Hello Sean! Why counting? My guess, and therefore the start of my reasoning and later questioning here, is that all those zeroes inside and UDP could flood the virtual network structure. May be you are confusing nc(1) with wc(1).
Re: Are nc -lu /dev/zero /dev/null a good throughput test?
On 19 July 2014 21:28, Philip Guenther guent...@gmail.com wrote: tcpbench(1) - TCP/UDP benchmarking and measurement tool Oh, just beneath my eyes, in the base install. Thank you, Philip. May I loose time comparing tcpbench(1) with iperf?
Re: Are nc -lu /dev/zero /dev/null a good throughput test?
On 20 July 2014 19:44, Adam Thompson athom...@athompso.net wrote: No, what he meant was that using nc -u can produce false results. Thank you Adam to point out my misinterpretation. Now I understand that Sean asked about how am I sure that all those zeroes generated in one host are really going to the other. The sender can send as many packets as its CPU can possibly send, even if 99.9% of those packets are getting dropped by the receiver; the sender still thinks it successfully send a bazillion bytes per second even though it's a meaningless number. Good point, as this: FWIW, you're almost certainly going to be CPU-bound. I can't get more than ~200Mbps on an emulated em(4) interface under ProxmoxVE (KVM 1.7.1) between two VMs running on the same host. Granted, the CPUs are slowish (2.2GHz Xeon L5520). I get better throughput using vio(4) but then I have to reboot the VMs once every 2 or 3 days to prevent them from locking up hard. What version of ProxmoxVE? I am considering this as a counterpart to XenServer, but I have some kind of faith in hypervisors in Xen and VMWare style, but in this project I can not afford VMWare prices. Thank you again, Adam!
Are nc -lu /dev/zero /dev/null a good throughput test?
Hello all! I am testing OpenBSD 5.5 Release over XenServer 6.2 with HVM and qemu-dm wrapper to change the default r8139 to virtio, adapted from [1]. So, to test the server private network throughput and other things related, I am using netcat. In this fashion: nc -lu 9000 /dev/zero /dev/null nc -u 192.168.1.10 9000 /dev/zero /dev/null Despite of pings showing 18ms of average time, it reached near 1Gbps of cross traffic (600Mbps in to and 300Mbps out from virtual router, at average) in the following configuration: . two virtual networks (int0 and int1 - internal networks) . one router between them . two vms for each network In int0, vms are servers (nc -l, as described before). In int1, vms are clients. Of course, there are no such terms when the connection starts, both ends are server and client at same time. Trying to start the same netcat idea, but in TCP mode, it only generate a few Mbps (mostly seem: 10Mbps of cross traffic, 5 in and 5 out) for each client/server. What could it be? No clues here, as a similar test with em on bare metal gave few Mbits less than UDP. And the main question: are this a good method to stress the virtual structure, or there are other good methods? Thank you for your time, Raimundo Santos [1] http://marc.info/?l=openbsd-miscm=135336071024634w=2
Re: Problem with icmp state creation on 5.3 PF
On 20 June 2013 16:53, Stuart Henderson s...@spacehopper.org wrote: On 2013-06-18, WiesÅaw Herr hers...@makhleb.net wrote: I suspect you may have an issue where state is not being created where you expect it. It's now recommended (and we've changed the sample pf.conf to match) to start your ruleset with an explicit block (or block log) rule to ensure that you don't accidentally allow any traffic to pass without keeping state. In case of a tproxy, which does no-evil and necessary IP spoofing, how will states be treated? My PF is in production, so I can not test now, but I had same issue (packets that bypass nat) with route-to from an interface to another and nat-to in the later. I have disabled states to test, and well... nat-to does not work without it... so I leave everything without states, only nat-to, but the same problem ocurred. By now in our ISP we have made a choice for the felxibility of FreeBSD IPFW, but I really like OpenBSD correctness and the shinny match PF rules, and ALTQ being removed/reconstructed in a new way.
Re: Problem with icmp state creation on 5.3 PF
Hello Wieslaw, hello misc@!
I run into a similar problem with my 'litle border' gateway here at my ISP.
We was experimenting with a regular ADSL connection to put what we call low
traffic priority, but our ADSL provider is diferent from our 2x fibber. All
of our IPs are from the fibber connection, so we can not send traffic
generated under these addresses to pass over the regular ADSL wires.
So I put this traffic behind NAT: routing with route-to and NATing out the
PPPoE interface. But it worked for little number of packets, a lot of
packets came in via fibber, what makes no sense in using an
emergencial/regular ADSL if we can not receive data over the line.
Did you know if it works under OpenBSD 5.2? I really like the simplicity
of OpenBSD, but we are migrating to FreeBSD due to this and the more
flexibility of IPFW - as we need to control bandwidth via IP addresses,
which lead to a lot of queues associated with one interface.
There are my thread: http://marc.info/?l=openbsd-miscm=136978016717969w=2
Thank you!
On 18 June 2013 13:38, WiesÅaw Herr hers...@makhleb.net wrote:
Hi misc@!
After deploying a new OpenBSD 5.3 firewall today I ran into a strange
problem. The first rule in my ruleset is one NAT-ing ICMP packets from my
host to Google's DNS IP (8.8.8.8):
fw1a-spt # pfctl -sr -R0
pass out log quick inet proto icmp from 192.168.5.96 to 8.8.8.8 nat-to
195.182.23.4
195.182.23.4 is my public IP address.
The problem is that only one in every ~20 packets gets NAT-ed. Other ones
get passed as-is. A tcpdump is available here:
http://hpaste.org/90099
I managed to increase the number of NAT-ed packets to about one-in-five
adding the following line to my PF:
set timeout { icmp.first 0, icmp.error 0 }
I would like not to paste my whole pf.conf file, since it's a bit large
(~1k lines) and has been recently migrated from a FreeBSD machine. There
are no arcane things inside (like limits or any other 'set' directives),
just other pass and block rules.
The trunk0 interface configuration:
trunkproto lacp
trunkport bge0
trunkport bge1
up
Disabling one of the bge interfaces didn't help either, which makes me
think LACP is unrelated to this.
A carp interface with the 195.182.23.4 IP address sits on top of the trunk
interface:
vhid 1 advskew 10 carpdev trunk0 pass *snip!*
inet 195.182.23.4 255.255.255.0 NONE
other aliases here...
If any other information is needed please ask and I'll happily provide it.
Does anybody have an idea what might be causing this? I'll try testing this
on CURRENT later, since I'm running out of ideas here...
And a mandatory dmesg follows:
OpenBSD 5.3 (GENERIC.MP) #62: Tue Mar 12 18:21:20 MDT 2013
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 2145452032 (2046MB)
avail mem = 2065903616 (1970MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.31 @ 0xdc010 (57 entries)
bios0: vendor HP version O08 date 08/13/2007
bios0: HP ProLiant DL140 G3
acpi0 at bios0: rev 0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP SPMI APIC MCFG BOOT SPCR SSDT
acpi0: wakeup devices BPD0(S5) BMF3(S5) P0P4(S5) P0P6(S5) PEX0(S5) PEX1(S5)
PEX2(S5) PEX3(S5) USB1(S5) USB2(S5) USB3(S5) EUSB(S5) PCIB(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU 5130 @ 2.00GHz, 1995.29 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,TM2,SS
SE3,CX16,xTPR,PDCM,DCA,NXE,LONG,LAHF,PERF
cpu0: 4MB 64b/line 16-way L2 cache
cpu0: apic clock running at 332MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Xeon(R) CPU 5130 @ 2.00GHz, 1995.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,TM2,SS
SE3,CX16,xTPR,PDCM,DCA,NXE,LONG,LAHF,PERF
cpu1: 4MB 64b/line 16-way L2 cache
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins
ioapic1 at mainbus0: apid 3 pa 0xfec8, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xe000, bus 0-23
acpiprt0 at acpi0: bus 1 (P0P2)
acpiprt1 at acpi0: bus 2 (BMD0)
acpiprt2 at acpi0: bus 3 (BPD0)
acpiprt3 at acpi0: bus -1 (BPD1)
acpiprt4 at acpi0: bus -1 (BPD2)
acpiprt5 at acpi0: bus 7 (BMF3)
acpiprt6 at acpi0: bus 12 (P0P4)
acpiprt7 at acpi0: bus 14 (P0P6)
acpiprt8 at acpi0: bus 0 (PCI0)
acpiprt9 at acpi0: bus 22 (PEX0)
acpiprt10 at acpi0: bus 23 (PEX1)
acpiprt11 at acpi0: bus -1 (PEX2)
acpiprt12 at acpi0: bus -1 (PEX3)
acpiprt13 at acpi0: bus 24 (PCIB)
acpicpu0 at acpi0
acpicpu1 at acpi0
acpibtn0 at acpi0: PWRB
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 Intel 5000X Host rev 0x31
ppb0 at pci0 dev 2 function 0 Intel 5000 PCIE x8 rev 0x31
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 Intel
Re: PF policy routing route-to rules don’t catch any packet
I've got the issue solved by disabling states on all rules which deal with
the tproxy.
On 4 June 2013 11:28, Raimundo Santos rait...@gmail.com wrote:
I am guessing that the problem lies with flags S/SA.
Changing all rules to flags any, and the packets hits the rules, but
things go worse: no web navigation... this is driving me mad!
On 3 June 2013 13:09, Raimundo Santos rait...@gmail.com wrote:
Hi there!
I asked, without an answer, something about nat-to and real IPs. Well, I
really need an answer there, so if someone get a clue, I will be glad tho
hear :)
Now, to the new issue!
Here in our WiFi ISP we are have contracted a tproxy service from FreeBSD
Brasil. It is somehow working, but I can not figure out exactly how. Here
is a diagram of the desired paths:
http://devio.us/~raitech/Obsd53PfTproxy.png
These are my rules by now:
RFC1918 = { 172.16/12, 192.168/16, 10/8, 127/8 }
table INT_NET persist { internal nets, all valid IPs }
ext_if_1 = em0
ext_gw_1 = 187.72.X.X
ext_ip_1 = 187.72.X.X
ext_if_2 = em1
ext_gw_2 = 187.72.X.X
ext_ip_2 = 187.72.X.X
ext_if_3 = alc0
ext_gw_3 = 187.72.X.X
ext_ip_3 = 187.72.X.X
int_if_1 = em2
int_gw_1 = 187.72.X.X
int_ip_1 = 187.72.X.X
squid_master_if = em3
squid_master_gw = 187.72.X.X
squid_master_ip = 187.72.X.X
set limit states 6304000
set limit tables 5000
set limit src-nodes 20
set limit frags 3000
set optimization aggressive
set state-defaults pflow, no-sync
set skip on lo
block in log quick on { \
$ext_if_1,\
$ext_if_2,\
$ext_if_3,\
$squid_master_if, \
$int_if_1 } from $RFC1918 label blocking RFC1918
# trying to prioritizing ACKs...
match set prio (3,5)
# ... and all traffic http. https over the others
match proto tcp to port { http, https } set prio (5,6)
match proto tcp from port { http, https } set prio (5,6)
match proto tcp to port { ssh, 9876 } set prio(5,7)
pass in on $int_if_1 proto tcp from { INT_NET, $int_gw_1 } to port http
\
route-to ($squid_master_if $squid_master_gw)
pass in on { $ext_if_1, $ext_if_2, $ext_if_3 } proto tcp from port http \
to { INT_NET, $int_gw_1 } \
route-to ($squid_master_if $squid_master_gw)
pass in on $squid_master_if proto tcp from { INT_NET, $int_gw_1 } to \
port http no state route-to \
{ \
($ext_if_1 $ext_gw_1) , \
($ext_if_2 $ext_gw_2) \
} least-states label cahce external outbound balancing
pass in on $squid_master_if proto tcp from port http\
to { INT_NET, $int_gw_1 } route-to ($int_if_1 $int_gw_1) \
label cahce internal outbound routing
An here are a pfctl -vsr output:
block drop in log quick on em0 inet from 172.16.0.0/12 to any label
blocking RFC1918
[ Evaluations: 61764339 Packets: 332 Bytes: 32854 States:
0 ]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em0 inet from 192.168.0.0/16 to any label
blocking RFC1918
[ Evaluations: 5883927 Packets: 114 Bytes: 28621 States:
0 ]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em0 inet from 10.0.0.0/8 to any label
blocking RFC1918
[ Evaluations: 5883813 Packets: 170 Bytes: 18354 States:
0 ]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em0 inet from 127.0.0.0/8 to any label
blocking RFC1918
[ Evaluations: 5883643 Packets: 0 Bytes: 0 States:
0 ]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em1 inet from 172.16.0.0/12 to any label
blocking RFC1918
[ Evaluations: 60684174 Packets: 305 Bytes: 30912 States:
0 ]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em1 inet from 192.168.0.0/16 to any label
blocking RFC1918
[ Evaluations: 6862827 Packets: 93Bytes: 9232States:
0 ]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em1 inet from 10.0.0.0/8 to any label
blocking RFC1918
[ Evaluations: 6862734 Packets: 196 Bytes: 19396 States:
0 ]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em1 inet from 127.0.0.0/8 to any label
blocking RFC1918
[ Evaluations: 6862538 Packets: 0 Bytes: 0 States:
0 ]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on alc0 inet from 172.16.0.0/12 to any label
blocking RFC1918
[ Evaluations: 50726925 Packets: 304 Bytes: 30856 States:
0 ]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on alc0 inet from 192.168.0.0/16 to any label
blocking RFC1918
[ Evaluations: 1251 Packets: 79Bytes: 8268States:
0 ]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on alc0 inet from 10.0.0.0/8 to any label
blocking RFC1918
[ Evaluations: 1172
Re: PF policy routing route-to rules don’t catch any packet
On 5 June 2013 17:50, Ville Valkonen weezeld...@gmail.com wrote:
Hi,
just confirming one thing: did you flush the pf states between the
tests? I must admit, I mainly glanced the problem, so sorry if this is
an old tip. This was the first thing that popped into my mind when
reading about your solution.
Hi Ville!
I am glad to hear some response :) - make me feel more part of misc@.
Yes, in almost all tests done I have flushed the states. Like this:
pfctl -F all pfctl -d pfct -ef test.conf
I am starting to guess that the beautiful PF state machine really does
not work well with this kind of IP spoofing, to name it better, tproxy.
But this is only a guess.
Oh, and sorry, I forgot to put the new and fresh and working config, here
they are:
# all macros are now living outside, for future anchors files
include /etc/pf.d/macros.conf
table AKAMAI persist file /etc/pf.d/akamai.table
table INT_NET persist file /etc/pf.d/int_net.table
# okey, I really do not think we will need all that mutch,
# but the machine has a lot of resources, why not spare?
set limit states 6304000
set limit tables 5000
set limit src-nodes 20
set limit frags 3000
set optimization aggressive
#set state-policy if-bound
# future is netflow/openflow to network analisys, isn't it?
set state-defaults pflow, no-sync
set skip on lo
#block private nets
block in log quick on { \
$ext_if_1,\
$ext_if_2,\
$ext_if_3,\
$squid_master_if, \
$int_if_1 } from $RFC1918 label blocking RFC1918
# trying to prioritizing things
#match in all scrub (no-df max-mss 1440)
match proto tcp flags any no state set prio (3,5)
match proto tcp to port { ssh, 9876 } flags any no state set prio(5,7)
match proto tcp to port { http, https } flags any no state set prio (5,6)
match proto tcp from port { http, https } flags any no state set prio (5,6)
match proto udp no state set prio 4
# test NAT IP_REAL - IP_REAL:
# $ext_if_4 is a PPPoE pseudo-interface
# testing how to route/nat to a emergency link traffic to/from Akamai
pass out quick on $ext_if_4 to AKAMAI flags any nat-to ($ext_if_4)
pass in quick on $squid_master_if proto tcp to AKAMAI port http flags
any \
route-to ($ext_if_4 $ext_gw_4)
pass in quick on $int_if_1 proto tcp to AKAMAI port != http flags any \
route-to ($ext_if_4 $ext_gw_4)
pass in quick on $int_if_1 proto udp to AKAMAI port != http \
route-to ($ext_if_4 $ext_gw_4)
pass out quick on { $ext_if_1, $ext_if_2, $ext_if_3 } to AKAMAI flags any
\
route-to ($ext_if_4 $ext_gw_4)
# here the magic begins
# just to see how many packets are passing
pass out on $squid_master_if from {INT_NET, $int_gw_1 } flags any no state
pass out on $squid_master_if to {INT_NET, $int_gw_1 } flags any no state
pass in on $squid_master_if from {INT_NET, $int_gw_1 } flags any no state
pass in on $squid_master_if to {INT_NET, $int_gw_1 } flags any no state
# route to squid_master
# ...from int_net
pass in quick on $int_if_1 proto tcp from { INT_NET, $int_gw_1 } to port
http \
route-to ($squid_master_if $squid_master_gw) flags any no state set prio
(5,6)
# ...from external_net
pass in quick on { $ext_if_1, $ext_if_2, $ext_if_3 } proto tcp from port
http \
to { INT_NET, $int_gw_1 } \
route-to ($squid_master_if $squid_master_gw) flags any no state set prio
(5,6)
Ville, if you have some idea about keeping states with tproxy in mind, it
will be very welcomed! Thank you :)
Raimundo Santos
Re: PF policy routing route-to rules don’t catch any packet
I am guessing that the problem lies with flags S/SA.
Changing all rules to flags any, and the packets hits the rules, but things
go worse: no web navigation... this is driving me mad!
On 3 June 2013 13:09, Raimundo Santos rait...@gmail.com wrote:
Hi there!
I asked, without an answer, something about nat-to and real IPs. Well, I
really need an answer there, so if someone get a clue, I will be glad tho
hear :)
Now, to the new issue!
Here in our WiFi ISP we are have contracted a tproxy service from FreeBSD
Brasil. It is somehow working, but I can not figure out exactly how. Here
is a diagram of the desired paths:
http://devio.us/~raitech/Obsd53PfTproxy.png
These are my rules by now:
RFC1918 = { 172.16/12, 192.168/16, 10/8, 127/8 }
table INT_NET persist { internal nets, all valid IPs }
ext_if_1 = em0
ext_gw_1 = 187.72.X.X
ext_ip_1 = 187.72.X.X
ext_if_2 = em1
ext_gw_2 = 187.72.X.X
ext_ip_2 = 187.72.X.X
ext_if_3 = alc0
ext_gw_3 = 187.72.X.X
ext_ip_3 = 187.72.X.X
int_if_1 = em2
int_gw_1 = 187.72.X.X
int_ip_1 = 187.72.X.X
squid_master_if = em3
squid_master_gw = 187.72.X.X
squid_master_ip = 187.72.X.X
set limit states 6304000
set limit tables 5000
set limit src-nodes 20
set limit frags 3000
set optimization aggressive
set state-defaults pflow, no-sync
set skip on lo
block in log quick on { \
$ext_if_1,\
$ext_if_2,\
$ext_if_3,\
$squid_master_if, \
$int_if_1 } from $RFC1918 label blocking RFC1918
# trying to prioritizing ACKs...
match set prio (3,5)
# ... and all traffic http. https over the others
match proto tcp to port { http, https } set prio (5,6)
match proto tcp from port { http, https } set prio (5,6)
match proto tcp to port { ssh, 9876 } set prio(5,7)
pass in on $int_if_1 proto tcp from { INT_NET, $int_gw_1 } to port http \
route-to ($squid_master_if $squid_master_gw)
pass in on { $ext_if_1, $ext_if_2, $ext_if_3 } proto tcp from port http \
to { INT_NET, $int_gw_1 } \
route-to ($squid_master_if $squid_master_gw)
pass in on $squid_master_if proto tcp from { INT_NET, $int_gw_1 } to \
port http no state route-to \
{ \
($ext_if_1 $ext_gw_1) , \
($ext_if_2 $ext_gw_2) \
} least-states label cahce external outbound balancing
pass in on $squid_master_if proto tcp from port http\
to { INT_NET, $int_gw_1 } route-to ($int_if_1 $int_gw_1) \
label cahce internal outbound routing
An here are a pfctl -vsr output:
block drop in log quick on em0 inet from 172.16.0.0/12 to any label
blocking RFC1918
[ Evaluations: 61764339 Packets: 332 Bytes: 32854 States: 0
]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em0 inet from 192.168.0.0/16 to any label
blocking RFC1918
[ Evaluations: 5883927 Packets: 114 Bytes: 28621 States: 0
]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em0 inet from 10.0.0.0/8 to any label
blocking RFC1918
[ Evaluations: 5883813 Packets: 170 Bytes: 18354 States: 0
]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em0 inet from 127.0.0.0/8 to any label
blocking RFC1918
[ Evaluations: 5883643 Packets: 0 Bytes: 0 States: 0
]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em1 inet from 172.16.0.0/12 to any label
blocking RFC1918
[ Evaluations: 60684174 Packets: 305 Bytes: 30912 States: 0
]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em1 inet from 192.168.0.0/16 to any label
blocking RFC1918
[ Evaluations: 6862827 Packets: 93Bytes: 9232States: 0
]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em1 inet from 10.0.0.0/8 to any label
blocking RFC1918
[ Evaluations: 6862734 Packets: 196 Bytes: 19396 States: 0
]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on em1 inet from 127.0.0.0/8 to any label
blocking RFC1918
[ Evaluations: 6862538 Packets: 0 Bytes: 0 States: 0
]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on alc0 inet from 172.16.0.0/12 to any label
blocking RFC1918
[ Evaluations: 50726925 Packets: 304 Bytes: 30856 States: 0
]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on alc0 inet from 192.168.0.0/16 to any label
blocking RFC1918
[ Evaluations: 1251 Packets: 79Bytes: 8268States: 0
]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on alc0 inet from 10.0.0.0/8 to any label
blocking RFC1918
[ Evaluations: 1172 Packets: 152 Bytes: 16948 States: 0
]
[ Inserted: uid 0 pid 19584 State Creations: 0 ]
block drop in log quick on alc0 inet
PF policy routing route-to rules don’t catch any packet
(no-sync, pflow) route-to 187.72.X.X@em3 [ Evaluations: 3694317 Packets: 12437474 Bytes: 9381159120 States: 3396 ] [ Inserted: uid 0 pid 19584 State Creations: 128206] pass in on em3 inet proto tcp from INT_NET to any port = 80 no state label cahce external outbound balancing route-to __automatic_9ca6f8d9_0 least-states [ Evaluations: 38420511 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 19584 State Creations: 0 ] pass in on em3 inet proto tcp from 187.72.X.X to any port = 80 no state label cahce external outbound balancing route-to __automatic_9ca6f8d9_1 least-states [ Evaluations: 13586403 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 19584 State Creations: 0 ] pass in on em3 inet proto tcp from any port = 80 to INT_NET flags S/SA keep state (no-sync, pflow) label cahce internal outbound routing route-to 187.72.X.X@em2 [ Evaluations: 13731058 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 19584 State Creations: 0 ] pass in on em3 inet proto tcp from any port = 80 to 187.72.X.X flags S/SA keep state (no-sync, pflow) label cahce internal outbound routing route-to 187.72.X.X@em2 [ Evaluations: 13586403 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 19584 State Creations: 0 ] This is the same behavior with or without multipath routing. What bahavior? Well, only rules for in on em3 that are destineted to internal network are working, the others barelly catches a few thousands of packets. Very strange... But, as said before: more strange is the fact that the cache solution is almost working, just some delays to load a page here, youtube gasps there, but overall it seems to work! Tested without multipath routing, without keep state, and the behavior are the same. Will apreciate any kind of help on this, thank you in advance. Raimundo Santos
PANIC when loading pf rules
Hello!
If you are following my debut here in misc@ (if not, please help me to put
our OpenBSD to rock this network!), you are somehow familiar with my
problems. I was trying to reproduce the panic in another context, but
unsuccessful... it only happens in production. Well, this is the ruleset:
RFC1918 = { 172.16/12, 192.168/16, 10/8, 127/8 }
table INT_NET persist { internal valid IPs }
ext_if_1 = em0
ext_gw_1 = 187.72.X.X
ext_ip_1 = 187.72.
ext_if_2 = em1
ext_gw_2 = 187.72.X.X
ext_ip_2 = 187.72.X.X
ext_if_3 = alc0
ext_gw_3 = 187.72.X.X
ext_ip_3 = 187.72.X.X
int_if_1 = em2
int_gw_1 = 187.72.X.X
int_ip_1 = 187.72.X.X
squid_master_if = em3
squid_master_gw = 187.72.X.X
squid_master_ip = 187.72.X.X
# increase default state limit from 10'000 states on busy systems
set limit states 6304000
set limit tables 5000
set limit src-nodes 20
set limit frags 3000
set optimization aggressive
set state-defaults pflow, no-sync
set skip on lo
#block private nets
block in log quick on { \
$ext_if_1,\
$ext_if_2,\
$ext_if_3,\
$squid_master_if, \
$int_if_1 } from $RFC1918 label blocking RFC1918
match on { $ext_if_1, $ext_if_2, $ext_if_3 } set prio (3,5)
match on $int_if_1 set prio (3,5)
match on $squid_master_if set prio (3,5)
match proto tcp to port { ssh, 9876 } set prio(5,7)
## outbound balancing
pass in on $int_if_1 from $int_gw_1 route-to \
{ \
($ext_if_1 $ext_gw_1) , \
($ext_if_2 $ext_gw_2) weight 10, \
($ext_if_3 $ext_gw_3) \
} least-states set prio (4,6) label outbound balancing NATed
pass in on $int_if_1 from INT_NET route-to \
{ \
($ext_if_1 $ext_gw_1) , \
($ext_if_2 $ext_gw_2) weight 10, \
($ext_if_3 $ext_gw_3) \
} least-states set prio (4,6) label outbound balancing all but NATed
And the only thing I could save was:
May 29 19:38:18 monster /bsd: fatal integer divide fault in supervisor mode
May 29 19:38:18 monster /bsd: trap type 8 code 0 rip 80272252 cs 8
rflags 10246 cr2 208444010 cpl 5 rsp 8000330cd920
May 29 19:38:18 monster /bsd: panic: trap type 8, code=0,
pc=80272252
May 29 19:38:18 monster /bsd: Starting stack trace...
May 29 19:38:18 monster /bsd: panic() at panic+0xf5
May 29 19:38:18 monster /bsd: trap() at trap+0x7f1
May 29 19:38:18 monster /bsd: --- trap (number 8) ---
May 29 19:38:18 monster /bsd: pf_map_addr() at pf_map_addr+0x8c2
May 29 19:38:18 monster /bsd: pf_set_rt_ifp() at pf_set_rt_ifp+0xf9
May 29 19:38:18 monster /bsd: pf_test_rule() at pf_test_rule+0xe3d
May 29 19:38:18 monster /bsd: pf_test() at pf_test+0xd15
May 29 19:38:18 monster /bsd: ipv4_input() at ipv4_input+0x230
May 29 19:38:18 monster /bsd: ipintr() at ipintr+0x7f
May 29 19:38:18 monster /bsd: netintr() at netintr+0xd5
May 29 19:38:18 monster /bsd: softintr_dispatch() at softintr_dispatch+0x5d
May 29 19:38:18 monster /bsd: Xsoftnet() at Xsoftnet+0x2d
May 29 19:38:18 monster /bsd: --- interrupt ---
May 29 19:38:18 monster /bsd: end trace frame: 0x0, count: 246
May 29 19:38:18 monster /bsd: 0x8:
May 29 19:38:18 monster /bsd: End of stack trace.
May 29 19:38:18 monster /bsd: syncing disks... splassert: assertwaitok:
want -1 have 1
May 29 19:38:18 monster /bsd: splassert: assertwaitok: want -1 have 1
May 29 19:38:18 monster last message repeated 21 times
May 29 19:38:18 monster /bsd: done
May 29 19:38:18 monster /bsd: done
May 29 19:38:18 monster /bsd: dump to dev 4,1 not possible
May 29 19:38:18 monster /bsd: rebooting...
Doing the load in boot time, the same problem. Doing the load after another
working ruleset, the same problem. This is just annoying, cos I can not do
the balancing with PF in this way. The problematic rules, in my tests (on
4a.m., lowest traffic over the network - I guess some pr0n and torrents),
are these for load balancing outbound traffic that arrives in on $int_if_1
(em2).
My other needs are:
put traffic from/to Akamai and another CDNs over an emergenial link - by
nat-to.
put port 80 traffic to the web over $squid_master, a proprietary cache
solution from FreeBSD Brasil - this is almost working, but I notice some
problems.
But all these are over another threads, just citating here.
A fresh dmesg:
OpenBSD 5.3 (GENERIC.MP) #62: Tue Mar 12 18:21:20 MDT 2013
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17101266944 (16309MB)
avail mem = 16623542272 (15853MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xeb420 (75 entries)
bios0: vendor American Megatrends Inc. version F6 date 03/23/2012
bios0: Gigabyte Technology Co., Ltd. Z77X-D3H
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC MCFG HPET SSDT SSDT SSDT DMAR
acpi0: wakeup devices PS2K(S3) PS2M(S3) P0P1(S4) USB1(S3) USB2(S3) USB3(S3)
USB4(S3) USB5(S3) USB6(S3) USB7(S3) PXSX(S4) RP01(S4) PXSX(S4) RP02(S4)
PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) PXSX(S4) RP05(S4) PXSX(S4) RP06(S4)
PXSX(S4) RP07(S4) PXSX(S4) RP08(S4) PEGP(S4) PEG0(S4) PEG1(S4) PEG2(S4)
PEG3(S4) GLAN(S4) EHC1(S4) EHC2(S4) XHC_(S4)
PF: nat-to from real IP to real IP is possible?
Hello folks!
I have this PF config (for whom could not see Web things, this config is
also at the end of the message):
http://pastebin.com/KZgzRJ6B
running well in OpenBSD 5.3 over a Core i5 Ivy Bridge, 16GB of RAM, 120GB
SSD, one 3Com 10/100 (driver xl), two Agere (driver et) 10/100/1000, one
Atthansic (alc) on-board Gigabit, and one Quad Port Intel (em). All things
going fine! :) - but there is Akamai...
My needs are: put through an internet emergencial link all Akamai CDN
traffic (and all the like we can track). This link is an ADSL, not so
reliable as our other two links (2x10Mbps, opticals, symmetric). And is not
intended to do routing for us, so I can not just drop my packets as
src:187.72.K.L over that ISP's line and expect them coming back through it.
In other words, there is no RIPv2 as we have in the other links.
So my thought was: why not do NAT through this emergencial link? Put a lot
of known IPs from Akamai and their friends in a PF table, and every packet
with destiny to any IP from that table go through this emergencial link.
How can I solve this? Our two best links are from one ISP, this emergencial
is from another.
Thank you all for the spent time!
An as promised, the configuration (sorry about formatation, I dunno how
GMail will treat this):
RFC1918 = { 172.16/12, 192.168/16, 10/8, 127/8 }
INT_NET = { internal real IPs }
ext_if_1 = em0
ext_gw_1 = 187.72.A.X
ext_ip_1 = 187.72.A.Y
ext_if_2 = em1
ext_gw_2 = 187.72.B.X
ext_ip_2 = 187.72.B.Y
ext_if_3 = alc0
ext_gw_3 = 187.72.C.X
ext_ip_3 = 187.72.C.Y
int_if_1 = em2
int_gw_1 = 187.72.D.X
int_ip_1 = 187.72.D.Y
squid_master_if = em3
squid_master_gw = 187.72.E.X
squid_master_ip = 187.72.E.Y
#all_ifs = { $ext_if_1, $ext_if_2, $ext_if_3, $int_if_1, $squid_master_if
}
# increase default state limit from 10'000 states on busy systems
set limit states 6304000
set limit tables 5000
set limit src-nodes 20
set limit frags 3000
set optimization normal
set state-defaults pflow, no-sync
set skip on lo
#block private nets
block in log quick on { \
$ext_if_1,\
$ext_if_2,\
$ext_if_3,\
$squid_master_if, \
$int_if_1 } from $RFC1918 label blocking RFC1918
# test nat-to IP_REAL - IP_REAL:
pass in on $int_if_1 from 187.72.W.A route-to pppoe0# can these...
pass out quick on pppoe0 from 187.72.W.A nat-to (pppoe0) # two rules work?
there is a way?
#pass on lo0 all flags S/SA
pass all flags any allow-opts # establish keep-state
# route to squid_master
pass in quick on $int_if_1 proto tcp from { $INT_NET, $int_gw_1 } to port
http \
route-to ($squid_master_if $squid_master_gw)
pass in quick on $ext_if_1 proto tcp from port http to { $INT_NET,
$int_gw_1 } \
route-to ($squid_master_if $squid_master_gw)
pass in quick on $ext_if_2 proto tcp from port http to { $INT_NET,
$int_gw_1 } \
route-to ($squid_master_if $squid_master_gw)
pass in quick on $ext_if_3 proto tcp from port http to { $INT_NET,
$int_gw_1 } \
route-to ($squid_master_if $squid_master_gw)
# route from squid_master
pass in quick on $squid_master_if proto tcp from { $INT_NET, $int_gw_1 } to
\
port http route-to \
{ \
($ext_if_1 $ext_gw_1) weight 1, \
($ext_if_2 $ext_gw_2) weight 50 \
} least-states label cahce outbound balancing
pass in quick on $squid_master_if proto tcp from port http to { $INT_NET,
$int_gw_1 } route-to ($int_if_1 $int_gw_1)
# let traffic in!
#pass in quick on $int_if_1 from { $INT_NET, $int_gw_1 } to {\
#$ext_if_1:network, \
#$ext_if_2:network, \
#$ext_if_3:network, \
#$squid_master_if:network }
pass in quick to { \
$ext_if_1:network, \
$ext_if_2:network, \
$ext_if_3:network, \
$squid_master_if:network } label passing in to myself nets
# outbound balancing
pass in quick on $int_if_1 from $int_gw_1 route-to \
{ \
($ext_if_1 $ext_gw_1) weight 1, \
($ext_if_2 $ext_gw_2) weight 10 \
} least-states label outbound balancing NATed
pass in quick on $int_if_1 from $INT_NET route-to \
{ \
($ext_if_1 $ext_gw_1) weight 10, \
($ext_if_2 $ext_gw_2) weight 1 \
} least-states label outbound balancing all but NATed
#pass in quick on $int_if_1 from $int_gw_1 route-to ($ext_if_2 $ext_gw_2) \
# label outbinding NATed to the best link
# symetric routing? may be not... ask someone else
pass out on $ext_if_1 from $ext_if_2 route-to ($ext_if_2 $ext_gw_2)
pass out on $ext_if_1 from $ext_if_3 route-to ($ext_if_3 $ext_gw_3)
pass out on $ext_if_2 from $ext_if_1 route-to ($ext_if_1 $ext_gw_1)
pass out on $ext_if_2 from $ext_if_3 route-to ($ext_if_3 $ext_gw_3)
pass out on $ext_if_3 from $ext_if_1 route-to ($ext_if_1 $ext_gw_1)
pass out on $ext_if_3 from $ext_if_2 route-to ($ext_if_2 $ext_gw_2)
