Re: OpenBSD as a router on Oracle T5120
On Tue, Jan 21, 2014 at 5:32 AM, Patrick Lamaiziere patf...@davenulle.orgwrote: Le Mon, 20 Jan 2014 18:59:02 -0200, Eduardo Meyer dudu.me...@gmail.com a écrit : hello, I am doing some basic testings on the above mentioned scenario and I am stuck on some limits which I consider to be very low: I cannot get more than 27Kpps and 200Mbit/s routing performance without starting to loose packets. ... All 6 network cards are Intel 82571EB which support MSI-X and should, in theory support IRQ balance. MSI are disabled on this chipset since OpenBSD 5.2... You can try to renabled MSI in em(4), here this helps a lot (on amd64). Check the thread (5.3) load problem on em(4) MSI / interrupt ? on misc@ https://www.mail-archive.com/misc@openbsd.org/msg123743.html Regards, patch on 5.3: --- /usr/src/sys/dev/pci/if_em.c.orig Tue Oct 1 14:45:36 2013 +++ /usr/src/sys/dev/pci/if_em.cTue Oct 1 14:48:52 2013 @@ -337,7 +337,7 @@ * Only use MSI on the newer PCIe parts, with the exception * of 82571/82572 due to Byte Enables 2 and 3 Are Not Set errata */ - if (sc-hw.mac_type = em_82572) + if (sc-hw.mac_type em_82571) sc-osdep.em_pa.pa_flags = ~PCI_FLAGS_MSI_ENABLED; /* Parameters (to be read from user) */ Thank you everyone, here we go with the results. Disabling PF caused a 15% performance improvement. OpenBSD 5.5 made the system more responsive under this load but made no real difference on pps or bps results. MSIX did not cause any trouble and it helped to raise pps up to 30K and bps up to 240M which is better but still very low :-( Thank you all :-)
Re: Power consumption of various architectures
Juan Francisco Cantero Hurtado i...@juanfra.info wrote: How can i find out this numbers? From power block sticker? I use a (good) power meter. Don't buy the cheapest one. From time to time, c't magazine reviews the cheap wattmeters available on the German (~ European) market. Their latest test, in the September 23 issue, revealed a number of models that were eerily in agreement with our 3,500-euro reference unit, including the cheapest one (10 euros). But really, any unit you buy is probably good enough. -- Christian naddy Weisgerber na...@mips.inka.de
Poor CARP Interface Performance with NAT
I am running obsd 5.4 as my NAT router. I decided to setup a second obsd box and run carp between the two for the external NATed interface (facing the ISP). After I setup everything and switched pf to NAT using the address on the carp interface, I'm seeing about 12Mbps - 13Mbps on the download, I have a 60Mbps pipe (down). When I switch pf back to NAT using the address on the physical interface, I get my full 60Mbps. Any ideas as to what I could be doing wrong that would limit performance through the carp interface to around 12Mbps - 13Mbps ? Thanks ...
my PF ICMP Issues
I have two Firewalls running OBSD 5.4 x64 that are both live and working fine except that they are unable to ping each others IP address or the gateway address while PF is enabled. If I quickly disable PF on the FW-D=Backup then I am able to ping everything from that machine. I've gone over everything I can think of but haven’t been able to figure this out so thought I'd ask here. FW-C = 192.168.xx.67 255.255.252.0 = Carp Master FW-D = 192.168.xx.65 255.255.252.0 = Carp Backup carp1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:00:5e:00:01:03 description: Carp 1 - Outside Iface priority: 0 carp: BACKUP carpdev vlanxx vhid 3 advbase 1 advskew 10 groups: carp status: backup inet6 fe80::200:5eff:fe00:103%carp1 prefixlen 64 scopeid 0xa inet 192.168.xx.62 netmask 0xfc00 broadcast 192.168.23.255 inet 192.168.xx.63 netmask 0xfc00 broadcast 192.168.23.255 = alias inet 192.168.xx.64 netmask 0xfc00 broadcast 192.168.23.255 = alias inet 192.168.xx.66 netmask 0xfc00 broadcast 192.168.23.255 = alias inet 192.168.xx.70 netmask 0xfc00 broadcast 192.168.23.255 = alias inet 192.168.xx.52 netmask 0xfc00 broadcast 192.168.23.255 = alias Gateway = 192.168.xx.1 FW-C is active I can't disable PF on this server. Neither FW-C or FW-D can ping the gateway when PF is enabled... If I disable PF on FW-D then I can ping the gateway from FW-D. Neither FW-C or FW-D can ping each others main IP (.67 or .65), If I disable PF on FW-D then I can ping .65 .67 from FW-D !!! Neither firewall can ping main carp IP .62 but can ping all the aliases, unless PF is disabled then it is ping able. There are other machines on the 192.168.xx.x network and they can ping all the IP's that FW-C D have all the time... Both firewalls have three nic's, one is dedicated for pfsync, the other two are trunked and then there are two vlans on top of the trunk. I stripped the PF.conf file on the down to as little as possible on the backup firewall this afternoon figuring that it must be the PF file that was wrong but I couldn't get it so that ping was replying. I've run tcpdump on all the interfaces and have checked pflog0 for blocked packets to no-avail :( If I am on FW-C and run ping 192.168.xx.65 then all I see on FW-D is the echo request over and over again tcpdump -n -e -ttt -i vlan40 Jan 22 00:31:49.334032 00:0a:f7:3a:44:c4 00:0a:f7:3a:45:0c 0800 98: 192.168.xx.67 192.168.xx.65: icmp: echo request If anyone can help then it would really be appreciated. Thanks Keith.
Re: my PF ICMP Issues
keith wrote: I have two Firewalls running OBSD 5.4 x64 that are both live and working fine except that they are unable to ping each others IP address or the gateway address while PF is enabled. If I quickly disable PF on the FW-D=Backup then I am able to ping everything from that machine. I've gone over everything I can think of but haven’t been able to figure this out so thought I'd ask here. FW-C = 192.168.xx.67 255.255.252.0 = Carp Master FW-D = 192.168.xx.65 255.255.252.0 = Carp Backup carp1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:00:5e:00:01:03 description: Carp 1 - Outside Iface priority: 0 carp: BACKUP carpdev vlanxx vhid 3 advbase 1 advskew 10 groups: carp status: backup inet6 fe80::200:5eff:fe00:103%carp1 prefixlen 64 scopeid 0xa inet 192.168.xx.62 netmask 0xfc00 broadcast 192.168.23.255 inet 192.168.xx.63 netmask 0xfc00 broadcast 192.168.23.255 = alias inet 192.168.xx.64 netmask 0xfc00 broadcast 192.168.23.255 = alias inet 192.168.xx.66 netmask 0xfc00 broadcast 192.168.23.255 = alias inet 192.168.xx.70 netmask 0xfc00 broadcast 192.168.23.255 = alias inet 192.168.xx.52 netmask 0xfc00 broadcast 192.168.23.255 = alias Gateway = 192.168.xx.1 FW-C is active I can't disable PF on this server. Neither FW-C or FW-D can ping the gateway when PF is enabled... If I disable PF on FW-D then I can ping the gateway from FW-D. Neither FW-C or FW-D can ping each others main IP (.67 or .65), If I disable PF on FW-D then I can ping .65 .67 from FW-D !!! Neither firewall can ping main carp IP .62 but can ping all the aliases, unless PF is disabled then it is ping able. There are other machines on the 192.168.xx.x network and they can ping all the IP's that FW-C D have all the time... Both firewalls have three nic's, one is dedicated for pfsync, the other two are trunked and then there are two vlans on top of the trunk. I stripped the PF.conf file on the down to as little as possible on the backup firewall this afternoon figuring that it must be the PF file that was wrong but I couldn't get it so that ping was replying. I've run tcpdump on all the interfaces and have checked pflog0 for blocked packets to no-avail :( If I am on FW-C and run ping 192.168.xx.65 then all I see on FW-D is the echo request over and over again tcpdump -n -e -ttt -i vlan40 Jan 22 00:31:49.334032 00:0a:f7:3a:44:c4 00:0a:f7:3a:45:0c 0800 98: 192.168.xx.67 192.168.xx.65: icmp: echo request If anyone can help then it would really be appreciated. Thanks Keith. Please post your pf.conf file, otherwise we can't help you.
Re: Apache able to open tty
Ted, Thank you so much for responding. I understand all of the words you used. However, this definitely goes beyond what I have done yet. I will need for the apache server to instigate the request. I imagine I would want a vast majority of the scripting to be run via CGI as normal calling out to the daemon when the connection is necessary. The part where I get very fuzzy is having the CGI script call out to a daemon which would be perhaps a “wrapper” for my PERL scripting that manages the process of making connections and retrieving data from my network devices. Would you be able to provide any links or verbiage I could search to head me in the right direction to figuring out this process? On Jan 20, 2014, at 7:38 PM, Ted Unangst t...@tedunangst.com wrote: On Wed, Jan 15, 2014 at 14:25, David Sticht wrote: Understanding the risks I am wanting to either allow the www user right to open tty or change the user running the apache daemon. I am developing a suite of intranet tools with perl to perform some network diagnostics. Does anybody have a suggestion to move me in the right direction? Build a small daemon that does whatever it is needs doing, run it as a user with the correct privileges, then have the www user talk to that via a socket.
Re: Power consumption of various architectures
On Tue, Jan 21, 2014 at 07:31:14PM +, Christian Weisgerber wrote: Juan Francisco Cantero Hurtado i...@juanfra.info wrote: How can i find out this numbers? From power block sticker? I use a (good) power meter. Don't buy the cheapest one. From time to time, c't magazine reviews the cheap wattmeters available on the German (~ European) market. Their latest test, in the September 23 issue, revealed a number of models that were eerily in agreement with our 3,500-euro reference unit, including the cheapest one (10 euros). But really, any unit you buy is probably good enough. The problem isn't the precision of the numbers, the real problem is the quality of the build and especially the quality of the circuit. I don't want to set fire to my house :P -- Juan Francisco Cantero Hurtado http://juanfra.info
Re: Poor CARP Interface Performance with NAT
Also, in case it might help, here's the dmesg output and appropriate hostname.* files from the primary (master): dmesg: OpenBSD 5.4 (GENERIC.MP) #41: Tue Jul 30 15:30:02 MDT 2013 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 2119761920 (2021MB) avail mem = 2055667712 (1960MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.6 @ 0x9f800 (22 entries) bios0: vendor American Megatrends Inc. version 080016 date 08/10/2012 acpi0 at bios0: rev 2 acpi0: sleep states S0 S1 S4 S5 acpi0: tables DSDT FACP APIC MCFG OEMB HPET GSCI acpi0: wakeup devices P0P1(S4) PS2K(S4) PS2M(S4) USB0(S4) USB1(S4) USB2(S4) USB5(S4) EUSB(S4) USB3(S4) USB4(S4) USB6(S4) USBE(S4) P0P4(S4) P0P5(S4) P0P6(S4) P0P7(S4) [...] acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1800.22 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF cpu0: 512KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 0 cpu0: apic clock running at 199MHz cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1799.99 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF cpu1: 512KB 64b/line 8-way L2 cache cpu1: smt 0, core 1, package 0 cpu2 at mainbus0: apid 1 (application processor) cpu2: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1799.98 MHz cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF cpu2: 512KB 64b/line 8-way L2 cache cpu2: smt 1, core 0, package 0 cpu3 at mainbus0: apid 3 (application processor) cpu3: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1799.99 MHz cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF cpu3: 512KB 64b/line 8-way L2 cache cpu3: smt 1, core 1, package 0 ioapic0 at mainbus0: apid 4 pa 0xfec0, version 20, 24 pins ioapic0: misconfigured as apic 1, remapped to apid 4 acpimcfg0 at acpi0 addr 0xe000, bus 0-255 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 5 (P0P1) acpiprt2 at acpi0: bus 1 (P0P4) acpiprt3 at acpi0: bus -1 (P0P5) acpiprt4 at acpi0: bus 2 (P0P6) acpiprt5 at acpi0: bus 3 (P0P7) acpiprt6 at acpi0: bus 4 (P0P8) acpiprt7 at acpi0: bus -1 (P0P9) acpicpu0 at acpi0 acpicpu1 at acpi0 acpicpu2 at acpi0 acpicpu3 at acpi0 acpibtn0 at acpi0: SLPB acpibtn1 at acpi0: PWRB pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 Intel Pineview DMI rev 0x02 vga1 at pci0 dev 2 function 0 Intel Pineview Video rev 0x02 intagp0 at vga1 agp0 at intagp0: aperture at 0xd000, size 0x1000 inteldrm0 at vga1 drm0 at inteldrm0 intel_overlay_map_regs partial stub inteldrm0: 1024x768 wsdisplay0 at vga1 mux 1: console (std, vt100 emulation) wsdisplay0: screen 1-5 added (std, vt100 emulation) Intel Pineview Video rev 0x02 at pci0 dev 2 function 1 not configured uhci0 at pci0 dev 26 function 0 Intel 82801I USB rev 0x02: apic 4 int 16 uhci1 at pci0 dev 26 function 1 Intel 82801I USB rev 0x02: apic 4 int 21 uhci2 at pci0 dev 26 function 2 Intel 82801I USB rev 0x02: apic 4 int 19 ehci0 at pci0 dev 26 function 7 Intel 82801I USB rev 0x02: apic 4 int 18 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb0 at pci0 dev 28 function 0 Intel 82801I PCIE rev 0x02: msi pci1 at ppb0 bus 1 ppb1 at pci0 dev 28 function 2 Intel 82801I PCIE rev 0x02: msi pci2 at ppb1 bus 2 em0 at pci2 dev 0 function 0 Intel 82574L rev 0x00: msi, address 00:30:18:a2:fd:ad ppb2 at pci0 dev 28 function 3 Intel 82801I PCIE rev 0x02: msi pci3 at ppb2 bus 3 em1 at pci3 dev 0 function 0 Intel 82574L rev 0x00: msi, address 00:30:18:a2:fd:ae ppb3 at pci0 dev 28 function 4 Intel 82801I PCIE rev 0x02: msi pci4 at ppb3 bus 4 ASMedia ASM1042 xHCI rev 0x00 at pci4 dev 0 function 0 not configured uhci3 at pci0 dev 29 function 0 Intel 82801I USB rev 0x02: apic 4 int 23 uhci4 at pci0 dev 29 function 1 Intel 82801I USB rev 0x02: apic 4 int 19 uhci5 at pci0 dev 29 function 2 Intel 82801I USB rev 0x02: apic 4 int 18 ehci1 at pci0 dev 29 function 7 Intel 82801I USB rev 0x02: apic 4 int 23 usb1 at ehci1: USB revision 2.0 uhub1 at usb1 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb4 at pci0 dev 30 function 0 Intel 82801BA Hub-to-PCI rev 0x92 pci5 at ppb4 bus 5 em2 at pci5 dev 4 function 0 Intel 82541GI rev 0x05: apic 4 int 18, address 00:30:18:a0:f5:ad em3 at pci5 dev 6 function 0 Intel 82541GI rev 0x05: apic 4 int 19, address 00:30:18:a0:f5:ae em4
Re: NAT reliability in light of recent checksum changes
On 2014-01-15, Stuart Henderson s...@spacehopper.org wrote: On 2014-01-14, Richard Procter richard.n.proc...@gmail.com wrote: I've a question about the new checksum changes. [...] My understanding is that checksums are now always recalculated when a header is altered, never updated. Is that right and if so has this affected NAT reliability? Recalculation here would compromise reliable end-to-end transport as the payload checksum no longer covers the entire network path, and so break a basic transport layer design principle. That is exactly what slides 30-33 talk about. PF now checks the incoming packets before it rewrites the checksum, so it can reject them if they are broken. Right -- so NAT now replaces the existing transport checksum with one newly computed from the payload [0]. This fundamentally weakens its usefulness, though: a correct checksum now implies only that the payload likely matches what the last NAT router happened to have in its memory, whereas the receiver wants to know whether what it got is what was originally transmitted. In the worst case of NAT on every intermediate node the transport checksum is effectively reduced to an adjunct of the link layer checksum. This means transport layer payload integrity is no longer reliant on the quality of the checksum algorithm alone but now depends too on the reliability of the path the packet took through the network. I think it's great to see someone working hard to simplify crucial code but in light of the above I believe pf should always update the checksum, as it did in versions prior to 5.4, as the alternative fundamentally undermines TCP by making the undetected error rate of its streams unknown and unbounded. One might argue networks these days are reliable; I think it better to avoid the need to make the argument. In any case the work I've found on that question is not reassuring [1]. best, Richard. [0] pf.c 1.863 On initial rule match: pf_test_rule() 3445: pf_translate() 3707: pf_change_ap() 1677: PF_ACPY [= pf_addrcpy()] 3461: pf_cksum() 6775: pd-hdr.tcp-th_sum = 0; m-m_pkthdr.csum_flags |= M_TCP_CSUM_OUT (if orig checksum good) On subsequent state matching: pf_test_state() ~4445: pf_change_ap() etc 4471: pf_cksum() etc [1] Probably the strongest message of this study is that the networking hardware is often trashing the packets which are entrusted to it http://conferences.sigcomm.org/sigcomm/2000/conf/paper/sigcomm2000-9-1.pdf Jonathan Stone and Craig Partridge. 2000. When the CRC and TCP checksum disagree. In Proceedings of the conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM '00). ACM, New York, NY, USA, 309-319. DOI=10.1145/347059.347561 http://doi.acm.org/10.1145/347059.347561
Re: NAT reliability in light of recent checksum changes
* Richard Procter richard.n.proc...@gmail.com [2014-01-22 06:44]: That is exactly what slides 30-33 talk about. PF now checks the incoming packets before it rewrites the checksum, so it can reject them if they are broken. Right -- so NAT now replaces the existing transport checksum with one newly computed from the payload [0]. correct - IF the original cksum was right. This fundamentally weakens its usefulness, though: a correct checksum now implies only that the payload likely matches what the last NAT router happened to have in its memory, whereas the receiver wants to know whether what it got is what was originally transmitted. In the worst case of NAT on every intermediate node the transport checksum is effectively reduced to an adjunct of the link layer checksum. huh? we receive a packet with correct cksum - NAT - packet goes out with correct cksum. we receive a packet with broken cksum - NAT - we leave the cksum alone, i. e. leave it broken. I think it's great to see someone working hard to simplify crucial code but in light of the above I believe pf should always update the checksum, as it did in versions prior to 5.4, as the alternative fundamentally undermines TCP by making the undetected error rate of its streams unknown and unbounded. One might argue networks these days are reliable; I think it better to avoid the need to make the argument. In any case the work I've found on that question is not reassuring [1]. It doesn't seem you know what you are talking about. the cksum is dead simple, if we had bugs in claculating or verifying it, we really had a LOT of other problems. There is no undetected error rate, nothing really changes there. -- Henning Brauer, h...@bsws.de, henn...@openbsd.org BS Web Services GmbH, http://bsws.de, Full-Service ISP Secure Hosting, Mail and DNS Services. Dedicated Servers, Root to Fully Managed Henning Brauer Consulting, http://henningbrauer.com/
How can limit per ip bandwidth in pf ?
Hi all,
I'm using pf as my firewall and authpf as my authenticating gateway.I think
it works well. But I have two problem about it:
1. I want to limit per ip(user) bandwidth using ALTQ. I google it and
haven't found any command or conf in pf.conf which can do it. My solution
is create queue per ip(user), for example:
altq on bnx1 cbq bandwideth 10Mb queue { user1, user2, user3 userN }
queue user1 bandwidth 1% cbq(default)
queue user2 bandwidth 1% cbq
...
queue userN bandwidth 1% cbq
in the autfpf.rules
pass in quick on bnx1 from $user_ip queue $user_id
it can meet my needs, but I think it is not dynamic, I need rewrite queue
if new user added. And I don't know how many queue can be supported in ALTQ.
2. Can I restruct one user_id only ssh login once ?
--
Blog Site: livemoon.org
Twitter: mwjpiero
·Çµ±¡ÎÞÒÔÃ÷Ö¾£¬·ÇÄþ¾²ÎÞÒÔÖÂÔ¶
Re: Poor CARP Interface Performance with NAT
You PF rules are needed too for this. On 22 jan 2014, at 00:51, Gabriel Kuri gk...@ieee.org wrote: I am running obsd 5.4 as my NAT router. I decided to setup a second obsd box and run carp between the two for the external NATed interface (facing the ISP). After I setup everything and switched pf to NAT using the address on the carp interface, I'm seeing about 12Mbps - 13Mbps on the download, I have a 60Mbps pipe (down). When I switch pf back to NAT using the address on the physical interface, I get my full 60Mbps. Any ideas as to what I could be doing wrong that would limit performance through the carp interface to around 12Mbps - 13Mbps ? Thanks ...
