Dell Poweedge 750 Mellanox ConnectX-6 LX with 1G SFP SX
Hi misc, I am trying to connect a 1GE SFP of type 1G SFP SX (Flexoptix S.8512.02.D) to a Power Edge R750 with a Connect Mellanox ConnectX-6 Lx. The mellanox driver supports the corresponding mode. I think it should be "1000base-SGMII": mcx5: flags=8843 mtu 1500 lladdr a0:88:c2:33:d1:b7 index 8 priority 0 llprio 3 media: Ethernet autoselect status: no carrier supported media: media 1000base-SGMII media 1000base-KX media 10GbaseKR media 10GSFP+Cu media 10GbaseSR media 10GbaseLR media 25GbaseCR media 25GbaseKR media 25GbaseSR media autoselect The SFP is recognized correctly: fw# ifconfig mcx5 transceiver mcx5: flags=8843 mtu 1500 lladdr a0:88:c2:33:d1:b7 index 8 priority 0 llprio 3 media: Ethernet autoselect status: no carrier transceiver: SFP LC, 850 nm, 270m OM1, 550m OM2 model: FLEXOPTIX S.8512.02.D rev A serial: F7AM3CB, date: 2023-06-13 voltage: 3.30 V, bias current: 9.70 mA temp: 40.60 C (low -10.00 C, high 85.00 C) tx: -6.30 dBm (low -12.00 dBm, high -1.00 dBm) rx: -8.40 dBm (low -20.00 dBm, high 1.00 dBm) However, the status remains in state "no carrier". It is interesting to note that the interface can obviously receive network packets: tcpdump: listening on mcx5, link-type EN10MB Mar 14 09:04:16.675476 802.1Q vid 1004 pri 6 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 demote=0 (DF) [tos 0x10] Mar 14 09:04:16.675476 802.1Q vid 1003 pri 6 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 demote=0 (DF) [tos 0x10] ... These network packets are sent from another system. This means that things seem to work in the receiving direction. In the same system there are mellanox connectX-5 nics. The 1GE-SFP works flawlessly with it. Regards, Joerg -- Dipl.-Ing. (FH) Joerg Streckfuss M.Sc. (Senior IT-Specialist) DFN-CERT Services GmbH, https://www.dfn-cert.de/, Phone +49 40 808077-555 Sitz / Register: Hamburg, AG Hamburg, HRB 88805, Ust-IdNr.: DE 232129737 Nagelsweg 41, 20097 Hamburg, Germany. CEO: Dr. Klaus-Peter Kossakowski smime.p7s Description: S/MIME Cryptographic Signature
Re: OpenBSD 7.2 fw stack trace on Dell R740
Hi Stuart, Am 25.09.23 um 19:08 schrieb Stuart Henderson: That might possibly be the one fixed by 7.2 errata 008, so if you don't already have that you at least want to syspatch. That was my guess as well. However, the systems were patched up to 7.2 errata-016. I applied the remaining patches. So far the systems are running stable. Are there any changes between the 7.2 and 7.3 releases that could indicate a bug? Many regards, Joerg On 2023-09-25, Joerg Streckfuss wrote: This is a cryptographically signed message in MIME format. --ms030306090501000403020005 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Dear list, today two of our firewalls crashed. after i was able to bring the first firewall back online, this one crashed again within a few minutes. this time i was able to take a stack-trace from the console: OpenBSD/amd64 (fw1) (tty00) login: uvm_fault(0x823237a0, 0x0, 0, 1) -> e fatal page fault in supervisor mode trap type 6 code 0 rip 81c38d68 cs 8 rflags 10246 cr2 0 cpl 0 rsp 80002b853590 gsbase 0x80001d3dcff0 kgsbase 0x0 panic: trap type 6, code=0, pc=81c38d68 Starting stack trace... panic(81f22fcf) at panic+0x12c kerntrap(80002b8534e0) at kerntrap+0x114 alltraps_kern_meltdown() at alltraps_kern_meltdown+0x7b pf_state_export(fd80513f0bd4,fd877600a2a0) at pf_state_export+0x38 pfsync_sendout() at pfsync_sendout+0x5e4 pfsync_update_state(fd887b0ef6c0) at pfsync_update_state+0x15b pf_test(18,1,82eed000,80002b853a08) at pf_test+0x117a ip6_input_if(80002b853a08,80002b853a14,29,0,82eed000) at ip6_input_if+0x1ae ipv6_input(82eed000,fd8050cb7c00) at ipv6_input+0x39 ether_input(82eed000,fd8050cb7c00) at ether_input+0x3b1 carp_input(8193d050,fd8050cb7c00,5e000102) at carp_input+0x196 ether_input(8193d050,fd8050cb7c00) at ether_input+0x1d9 if_input_process(8193d050,80002b853be8) at if_input_process+0x6f ifiq_process(8193aa00) at ifiq_process+0x69 taskq_thread(80037180) at taskq_thread+0x100 end trace frame: 0x0, count: 242 End of stack trace. Both Systems are OpenBSD 7.2 running on Dell PowerEdge R740 Is anyone able to interpret the stack trace? Regards, Joerg --ms030306090501000403020005 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCC EP8wggUSMIID+qADAgECAgkA4wvV+K8l2YEwDQYJKoZIhvcNAQELBQAwgYIxCzAJBgNVBAYT AkRFMSswKQYDVQQKDCJULVN5c3RlbXMgRW50ZXJwcmlzZSBTZXJ2aWNlcyBHbWJIMR8wHQYD VQQLDBZULVN5c3RlbXMgVHJ1c3QgQ2VudGVyMSUwIwYDVQQDDBxULVRlbGVTZWMgR2xvYmFs Um9vdCBDbGFzcyAyMB4XDTE2MDIyMjEzMzgyMloXDTMxMDIyMjIzNTk1OVowgZUxCzAJBgNV BAYTAkRFMUUwQwYDVQQKEzxWZXJlaW4genVyIEZvZXJkZXJ1bmcgZWluZXMgRGV1dHNjaGVu IEZvcnNjaHVuZ3NuZXR6ZXMgZS4gVi4xEDAOBgNVBAsTB0RGTi1QS0kxLTArBgNVBAMTJERG Ti1WZXJlaW4gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkgMjCCASIwDQYJKoZIhvcNAQEBBQAD ggEPADCCAQoCggEBAMtg1/9moUHN0vqHl4pzq5lN6mc5WqFggEcVToyVsuXPztNXS43O+FZs FVV2B+pG/cgDRWM+cNSrVICxI5y+NyipCf8FXRgPxJiZN7Mg9mZ4F4fCnQ7MSjLnFp2uDo0p eQcAIFTcFV9Kltd4tjTTwXS1nem/wHdN6r1ZB+BaL2w8pQDcNb1lDY9/Mm3yWmpLYgHurDg0 WUU2SQXaeMpqbVvAgWsRzNI8qIv4cRrKO+KA3Ra0Z3qLNupOkSk9s1FcragMvp0049ENF4N1 xDkesJQLEvHVaY4l9Lg9K7/AjsMeO6W/VRCrKq4Xl14zzsjz9AkH4wKGMUZrAcUQDBHHWekC AwEAAaOCAXQwggFwMA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUk+PYMiba1fFKpZFK4OpL 4qIMz+EwHwYDVR0jBBgwFoAUv1kgNgB5oKAia4zV8mHSuCzLgkowEgYDVR0TAQH/BAgwBgEB /wIBAjAzBgNVHSAELDAqMA8GDSsGAQQBga0hgiwBAQQwDQYLKwYBBAGBrSGCLB4wCAYGZ4EM AQICMEwGA1UdHwRFMEMwQaA/oD2GO2h0dHA6Ly9wa2kwMzM2LnRlbGVzZWMuZGUvcmwvVGVs ZVNlY19HbG9iYWxSb290X0NsYXNzXzIuY3JsMIGGBggrBgEFBQcBAQR6MHgwLAYIKwYBBQUH MAGGIGh0dHA6Ly9vY3NwMDMzNi50ZWxlc2VjLmRlL29jc3ByMEgGCCsGAQUFBzAChjxodHRw Oi8vcGtpMDMzNi50ZWxlc2VjLmRlL2NydC9UZWxlU2VjX0dsb2JhbFJvb3RfQ2xhc3NfMi5j ZXIwDQYJKoZIhvcNAQELBQADggEBAIcL/z4Cm2XIVi3WO5qYi3FP2ropqiH5Ri71sqQPrhE4 eTizDnS6dl2e6BiClmLbTDPo3flq3zK9LExHYFV/53RrtCyD2HlrtrdNUAtmB7Xts5et6u5/ MOaZ/SLick0+hFvu+c+Z6n/XUjkurJgARH5pO7917tALOxrN5fcPImxHhPalR6D90Bo0fa3S PXez7vTXTf/D6OWST1k+kEcQSrCFWMBvf/iu7QhCnh7U3xQuTY+8npTD5+32GPg8SecmqKc2 2CzeIs2LgtjZeOJVEqM7h0S2EQvVDFKvaYwPBt/QolOLV5h7z/0HJPT8vcP9SpIClxvyt7bP ZYoaorVyGTkwggWsMIIElKADAgECAgcbY7rQHiw9MA0GCSqGSIb3DQEBCwUAMIGVMQswCQYD VQQGEwJERTFFMEMGA1UEChM8VmVyZWluIHp1ciBGb2VyZGVydW5nIGVpbmVzIERldXRzY2hl biBGb3JzY2h1bmdzbmV0emVzIGUuIFYuMRAwDgYDVQQLEwdERk4tUEtJMS0wKwYDVQQDEyRE Rk4tVmVyZWluIENlcnRpZmljYXRpb24gQXV0aG9yaXR5IDIwHhcNMTYwNTI0MTEzODQwWhcN MzEwMjIyMjM1OTU5WjCBjTELMAkGA1UEBhMCREUxRTBDBgNVBAoMPFZlcmVpbiB6dXIgRm9l cmRlcnVuZyBlaW5lcyBEZXV0c2NoZW4gRm9yc2NodW5nc25ldHplcyBlLiBWLjEQMA4GA1UE CwwHREZOLVBLSTElMCMGA1UEAwwcREZOLVZlcmVpbiBHbG9iYWwgSXNzdWluZyBDQTCCASIw DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJ07eRxH3h+Gy
OpenBSD 7.2 fw stack trace on Dell R740
Dear list, today two of our firewalls crashed. after i was able to bring the first firewall back online, this one crashed again within a few minutes. this time i was able to take a stack-trace from the console: OpenBSD/amd64 (fw1) (tty00) login: uvm_fault(0x823237a0, 0x0, 0, 1) -> e fatal page fault in supervisor mode trap type 6 code 0 rip 81c38d68 cs 8 rflags 10246 cr2 0 cpl 0 rsp 80002b853590 gsbase 0x80001d3dcff0 kgsbase 0x0 panic: trap type 6, code=0, pc=81c38d68 Starting stack trace... panic(81f22fcf) at panic+0x12c kerntrap(80002b8534e0) at kerntrap+0x114 alltraps_kern_meltdown() at alltraps_kern_meltdown+0x7b pf_state_export(fd80513f0bd4,fd877600a2a0) at pf_state_export+0x38 pfsync_sendout() at pfsync_sendout+0x5e4 pfsync_update_state(fd887b0ef6c0) at pfsync_update_state+0x15b pf_test(18,1,82eed000,80002b853a08) at pf_test+0x117a ip6_input_if(80002b853a08,80002b853a14,29,0,82eed000) at ip6_input_if+0x1ae ipv6_input(82eed000,fd8050cb7c00) at ipv6_input+0x39 ether_input(82eed000,fd8050cb7c00) at ether_input+0x3b1 carp_input(8193d050,fd8050cb7c00,5e000102) at carp_input+0x196 ether_input(8193d050,fd8050cb7c00) at ether_input+0x1d9 if_input_process(8193d050,80002b853be8) at if_input_process+0x6f ifiq_process(8193aa00) at ifiq_process+0x69 taskq_thread(80037180) at taskq_thread+0x100 end trace frame: 0x0, count: 242 End of stack trace. Both Systems are OpenBSD 7.2 running on Dell PowerEdge R740 Is anyone able to interpret the stack trace? Regards, Joerg smime.p7s Description: S/MIME Cryptographic Signature
relayd: pfe_route: failed to add gateway 22 Invalid argument
Hello,
I'm trying to use the relayd router function to add host routes to the routing
table with a route label for further processing by bgpd. The host ist directly
connected to the firewall.
relayd.conf:
table { 2001:::::4 }
router "service_v6" {
route 2001:::::4/128
forward to port 80 check tcp
rtlabel geo_service
}
fw1# relayd -vvvd
startup
socket_rlimit: max open files 1024
pfe: filter init done
socket_rlimit: max open files 1024
socket_rlimit: max open files 1024
socket_rlimit: max open files 1024
parent_tls_ticket_rekey: rekeying tickets
hce_notify_done: 2001:::::4 (tcp connect ok)
host 2001:::::4, check tcp (0ms,tcp connect ok), state unknown ->
up, availability 100.00%
pfe_dispatch_hce: state 1 for host 1 2001:::::4
sync_routes: router service_v6 route 2001:::::4/128 gateway
2001:::::4 up priority 0
hce_notify_done: 2001:::::4 (tcp connect ok)
pfe_route: failed to add gateway 2001:::::4: 22 Invalid argument
hce_notify_done: 2001:::::4 (tcp connect ok)
hce_notify_done: 2001:::::4 (tcp connect ok)
hce_notify_done: 2001:::::4 (tcp connect ok)
The route with the route label never pops up in the routing table. With IPv4
addresses the setup works as expected.
Any suggestions?
smime.p7s
Description: S/MIME Cryptographic Signature
Intel nic on Dell R710: failed to allocate interrupt slot for PIC msix
Dear List, we have problems with Intel nics of type Intel X710 (10 GbE) on a Dell R740. In total we have three nics with four ports each. With the uprade to OpenBSD 6.8 we lost two ports (ixl11 and ixl12). Now we upraded iteratively to OpenBSD 7.1 an we lost another port (ixl10). The update to OpenBSD 7.2 is pending, but I don't want to risk losing another port. Cause seems to be a problem with the interrupt assignment. The relevant dmesg part is as fallows: ixl11 at pci12 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 2, FW 7.83.59945 API 1.9, ms1 failed to allocate interrupt slot for PIC msix pin -2135686911 ixl11: unable to establish interrupt 1 ixl12 at pci12 dev 0 function 2 "Intel X710 SFP+" rev 0x02: port 0, FW 7.83.59945 API 1.9, ms2 failed to allocate interrupt slot for PIC msix pin -2135686655 ixl12: unable to establish interrupt 1 ixl13 at pci12 dev 0 function 3 "Intel X710 SFP+" rev 0x02: port 1, FW 7.83.59945 API 1.9, ms3 failed to allocate interrupt slot for PIC msix pin -2135686399 ixl13: unable to establish interrupt 1 full dmesg: Booting from Hard drive C: Using drive 0, partition 3. Loading.. probing: pc0 com0 mem[624K 1266M 2M 398M 30720M a20=on] disk: hd0+ >> OpenBSD/amd64 BOOT 3.53 switching console to com0 >> OpenBSD/amd64 BOOT 3.53 booting hd0a:/bsd: 15639832+3699728+348192+0+1175552 [1126995+128+1220904+924861]=0x17074c0 entry point at 0x81001000 �[ using 3273920 bytes of bsd ELF symbol table ] Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. Copyright (c) 1995-2022 OpenBSD. All rights reserved. https://www.OpenBSD.org OpenBSD 7.1 (GENERIC.MP) #2: Fri Jan 20 13:16:22 MST 2023 t...@syspatch-71-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 33941528576 (32369MB) avail mem = 32895590400 (31371MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 3.2 @ 0x68e36000 (75 entries) bios0: vendor Dell Inc. version "2.10.2" date 02/24/2021 bios0: Dell Inc. PowerEdge R740 acpi0 at bios0: ACPI 6.1 acpi0: sleep states S0 S5 acpi0: tables DSDT FACP SSDT TPM2 SSDT MCEJ WDAT SLIC HPET APIC MCFG MIGT MSCT PCAT PCCT RASFJ acpi0: wakeup devices XHCI(S4) RP17(S4) PXSX(S4) RP18(S4) PXSX(S4) RP19(S4) PXSX(S4) RP20(S4)] acpitimer0 at acpi0: 3579545 Hz, 24 bits acpihpet0 at acpi0: 2399 Hz acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 2 (boot processor) cpu0: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3692.05 MHz, 06-55-04 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N cpu0: 256KB 64b/line 8-way L2 cache cpu0: cannot disable silicon debug cpu0: smt 0, core 1, package 0 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 24MHz cpu0: mwait min=64, max=64, C-substates=0.2.0.2, IBE cpu1 at mainbus0: apid 10 (application processor) cpu1: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.34 MHz, 06-55-04 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N cpu1: 256KB 64b/line 8-way L2 cache cpu1: cannot disable silicon debug cpu1: smt 0, core 5, package 0 cpu2 at mainbus0: apid 4 (application processor) cpu2: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.33 MHz, 06-55-04 cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N cpu2: 256KB 64b/line 8-way L2 cache cpu2: cannot disable silicon debug cpu2: smt 0, core 2, package 0 cpu3 at mainbus0: apid 22 (application processor) cpu3: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.33 MHz, 06-55-04 cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N cpu3: 256KB 64b/line 8-way L2 cache cpu3: cannot disable silicon debug cpu3: smt 0, core 11, package 0 cpu4 at mainbus0: apid 3 (application processor) cpu4: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.32 MHz, 06-55-04 cpu4: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N cpu4: 256KB 64b/line 8-way L2 cache cpu4: cannot disable silicon debug cpu4: smt 1, core 1, package 0 cpu5 at mainbus0: apid 11 (application processor) cpu5: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.32 MHz, 06-55-04 cpu5: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N cpu5: 256KB 64b/line 8-way L2 cache cpu5: cannot disable silicon debug cpu5: smt 1, core 5, package 0 cpu6 at mainbus0: apid 5 (application processor) cpu6: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.32 MHz, 06-55-04 cpu6: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N cpu6: 256KB 64b/line 8-way L2 cache cpu6: cannot disable silicon debug cpu6: smt 1, core 2, package 0 cpu7 at mainbus0: apid 23 (application processor) cpu7: Intel(R) Xeon(R) Gold 5122 CPU @
smtp relay host with two mx entries
Hi misc, i am trying to create a simple smtp client configuration, where the client should only send local mails to a relay host. the key point is that the relay host hides a redundant MX record with different priorities in the DNS. A DNS A-record (or quad A) on the other hand does not exist. As I understand it, this is not possible with a relay statement, because no MX lookup is done. How must a corresponding configuration look like to be able to use a MX lookup? this is my (still broken) configuration: table aliases file:/etc/mail/aliases listen on socket listen on lo0 action "local_mail" mbox alias action "outbound" relay smtp+tls:// match from local for local action "local_mail" match from local for any action "outbound" Many thanks!
Dell PE 6515 with Intel DP XXV710 SFP28
Hello list, I am trying to get Intel XXV710 SFP28 dual port nics to work under OpenBSD 7.0-beta on a PE 6515 with AMD Milan CPU. There are two cards in the server. The behavior is such that only one port works on one card at a time. Occasionally two ports distributed on two cards work but never two ports on one card. The behavior can be influenced by subjecting the server to a power drain. Thus, for example, ixl0 can be activated if ixl2 was active before. We use DAC cables from Flexoptix with 25Gb SFPs and corresponding Intel firmware. Dmesg shows that the nics were detected correctly: ixl0 at pci6 dev 0 function 0 "Intel XXV710 SFP28" rev 0x02: port 0, FW 8.815.63341 API 1.12, msix, 8 queues, address 40:a6:b7:70:3a:70 ixl1 at pci6 dev 0 function 1 "Intel XXV710 SFP28" rev 0x02: port 1, FW 8.815.63341 API 1.12, msix, 8 queues, address 40:a6:b7:70:3a:71 ixl2 at pci14 dev 0 function 0 "Intel XXV710 SFP28" rev 0x02: port 0, FW 8.815.63341 API 1.12, msix, 8 queues, address 40:a6:b7:70:03:b0 ixl3 at pci14 dev 0 function 1 "Intel XXV710 SFP28" rev 0x02: port 1, FW 8.815.63341 API 1.12, msix, 8 queues, address 40:a6:b7:70:03:b1 The following output shows a dual port card with an optical sfp and a dac. only ixl0 has a carrier: fw1# ifconfig ixl0 transceiver ixl0: flags=8802 mtu 1500 lladdr 40:a6:b7:70:3a:70 index 3 priority 0 llprio 3 media: Ethernet autoselect (10GbaseSR full-duplex) status: active transceiver: SFP LC, 850 nm, 30m OM1, 80m OM2, 600m OM3 model: Intel Corp P.8596.02 rev A serial: F79HNJM, date: 2020-06-29 voltage: 3.29 V, bias current: 5.96 mA temp: 39.16 C (low -25.00 C, high 90.00 C) tx: -3.14 dBm (low -9.30 dBm, high 1.00 dBm) rx: -2.99 dBm (low -13.10 dBm, high 1.00 dBm) fw1# ifconfig ixl1 transceiver ixl1: flags=8802 mtu 1500 lladdr 40:a6:b7:70:3a:71 index 4 priority 0 llprio 3 media: Ethernet autoselect status: no carrier transceiver: SFP LC, 2m model: Intel Corp P.C3025G.2 rev 1.0 serial: F79SBRB-A, date: 2021-03-19 The complete dmesg: OpenBSD 7.0 (GENERIC.MP) #215: Tue Sep 14 15:25:03 MDT 2021 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 33933246464 (32361MB) avail mem = 3211520 (31365MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 3.3 @ 0x698a2000 (58 entries) bios0: vendor Dell Inc. version "2.2.4" date 04/12/2021 bios0: Dell Inc. PowerEdge R6515 acpi0 at bios0: ACPI 6.0 acpi0: sleep states S0 S5 acpi0: tables DSDT FACP BERT HEST HPET APIC MCFG WSMT SLIC SSDT SSDT EINJ PCCT SSDT CRAT CDIT IVRS SSDT SSDT acpi0: wakeup devices PC00(S5) XHCI(S3) PC01(S5) XHCI(S3) PC02(S5) XHCI(S3) PC03(S5) XHCI(S3) acpitimer0 at acpi0: 3579545 Hz, 32 bits acpihpet0 at acpi0: 14318180 Hz acpimadt0 at acpi0 addr 0xfee0: PC-AT compat ioapic0 at mainbus0: apid 240 pa 0xfec0, version 21, 24 pins, can't remap ioapic1 at mainbus0: apid 241 pa 0xe010, version 21, 32 pins, can't remap ioapic2 at mainbus0: apid 242 pa 0xc510, version 21, 32 pins, can't remap ioapic3 at mainbus0: apid 243 pa 0xaa10, version 21, 32 pins, can't remap ioapic4 at mainbus0: apid 244 pa 0xfd10, version 21, 32 pins, can't remap cpu0 at mainbus0: apid 0 (boot processor) cpu0: AMD EPYC 74F3 24-Core Processor, 3194.44 MHz, 19-01-01 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,INVPCID,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,PKU,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu0: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache cpu0: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu0: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 99MHz cpu0: mwait min=64, max=64, C-substates=1.1, IBE cpu1 at mainbus0: apid 2 (application processor) cpu1: AMD EPYC 74F3 24-Core Processor, 3194.01 MHz, 19-01-01 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,INVPCID,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,PKU,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu1: 32KB 64b/line
OpenBSD on Dell R6515 with AMD Milan 7313P
Hi list, We orderd some Dell machines PE R6515 with AMD EPYC 7302P 3GHz but surprise the CPUs are not available. An alternative suggested by dell would be the Epyc 7402p 24C/48T 2,8Ghz CPU. But I'm thinking of going for the Milan CPU right away. Specifically, it would be the AMD Milan 7313P 3Ghz 16C/32T CPU. Do you have any experience with the cpus and do you know if they work with openbsd? Many Regards, Joerg
OpenBSD on Dell PE R6515
Hello folks, in the past we used Dell servers like PE 1850, PE 2850, PE R730 and PE R740. We had good experiences running Openbsd on these systems. These models are all Intel based but for another project i'm considering giving AMD a chance. I'm very interested in the Dell PE R6515 with AMD EPYC 7302P 3GHz, 16C/32T CPU and with a mix of NICs (Intel XXV710 10/25 GbE SFP28, Broadcom 57416 Dual Port 10 GbE SFP+, Intel i350 Quad Port 1GbE BASE-T). The purpose is a Mix of PF firewall and bgp router. In the first stage of expansion, the system should be able to handle 10Gbits of traffic. Possibly more later. Does anyone have experience running OpenBSD on this platform? Thanks in advance for feedback, Joerg
Re: handling snapshot installation in production environment
Am 02.09.19 um 19:58 schrieb Stuart Henderson:
Use sysupgrade -n and monitor the OS version number ("what
/home/_sysupgrade/bsd"). If you see 6.6-current it is post-release and
you should not install it ("rm /bsd.upgrade"), you can then wait until
actual release day and update to be sure you're running on the proper
release binaries.
This is exactly what I was searching for.
Thanks!
smime.p7s
Description: S/MIME Cryptographic Signature
handling snapshot installation in production environment
Hi Misc, we have to run 6.6 snapshot on one of our firewall clusters to get in touch with the new aggr(4) driver. This driver seems to work great whith 6.6 snapshot on a dell pe 470 with intel X710 based quadport sfp+ nics doing LACP. We had serious problems with the trunk(4) driver on OpenBSD 6.5 stable which I discribed in the thread "Dell PE R740, Intel X710 QuadPort & LACP not working". I am new in running snapshots in production environments. Our goal is to run the 6.6 snapshot till 6.6 will be released. The sysupgrade tool is a nice way to install the newest snapshot, never had a problem. But what is the correct way to install a stable release on snapshot? Using the standard bsd.rd upgrade way? Furthermore I'm not sure which snapshot should I run. Almost every day there will be a fresh one. Perhaps is there a moment/date where a freeze of the code base will be done which reflects the 6.6 release? Thanks.
Re: Dell PE R740, Intel X710 QuadPort & LACP not working
Am 01.08.19 um 14:55 schrieb Joerg Streckfuss: Hi Misc, we bought two new Dell PowerEdges R740. Each System has 3 intel X770 based quadport sfp+ nics. Onboard are two further intel i350 based sfp+ ports. Correction - Of course I mean 3 intel X710 based quadport sfp+ nics and two intel x520 based sfp+ ports. Sorry smime.p7s Description: S/MIME Cryptographic Signature
Dell PE R740, Intel X710 QuadPort & LACP not working
Hi Misc, we bought two new Dell PowerEdges R740. Each System has 3 intel X770 based quadport sfp+ nics. Onboard are two further intel i350 based sfp+ ports. The firewalls are running OpenBSD 6.5 stable. To test lacp 802.3ad with ix and ixl based interfaces I build two trunks which directly connect the two systems: +-+ trunk1 +-+ | ix0||ix0 | | ix1||ix1 | | openbsd1 || openbsd2 | | ixl1||ixl1 | | ixl6||ixl6 | +-+ trunk2 +-+ fw1: /etc/hostname.trunk1 up trunkport ix0 trunkport ix1 trunkproto lacp lacpmode active lacptimeout fast inet 10.0.0.1/30 /etc/hostname.trunk2 up trunkport ixl6 trunkport ixl1 trunkproto lacp lacpmode active lacptimeout fast inet 10.1.0.1/3 fw2: /etc/hostname.trunk1 up trunkport ix0 trunkport ix1 trunkproto lacp lacpmode active lacptimeout fast inet 10.0.0.2/30 /etc/hostname.trunk2 up trunkport ixl6 trunkport ixl1 trunkproto lacp lacpmode active lacptimeout fast inet 10.1.0.2/3 Trunk1 with ix based ports behaved as expected. Disconnecting one of the fibers to simulate a broken one or doing ifconfig ix0|ix1 down has not disturbed the ping between the two firewalls. Futhermore doing an ifconfig ix0|ix1 up brought that interface back to the trunk correctly. The first impression with testing ixl based ports looked good. Doing ifconfig ixl1|ixl6 down let trunk switching to the only one active interface. But then checking the deactivated interface with ifconfig ixl6 transceiver showed the following: # ifconfig xl6 transceiver ixl6: flags=8902 mtu 1500 lladdr f8:f2:1e:65:30:e0 index 11 priority 0 llprio 3 trunk: trunkdev trunk2 media: Ethernet autoselect (10GbaseSR full-duplex) status: active transceiver: SFP LC, 850 nm, 80m OM2, 30m OM1, 300m OM3 model: Intel Corp P.8596.02 rev A serial: F78Y3VQ, date: 2019-06-12 voltage: 3.33 V, bias current: 5.64 mA temp: 29.80 C (low -10.00 C, high 90.00 C) tx: -3.88 dBm (low -9.30 dBm, high 1.00 dBm) rx: -3.40 dBm (low -13.10 dBm, high 1.00 dBm) Hmm, okey the status is still active. But tcpdump didnt recognized any packets on that device. Then i tried to reactivate ixl6 with ifconfig ixl6 up: # ifconfig ixl6 transceiver ixl6: flags=8943 mtu 1500 lladdr f8:f2:1e:65:30:e0 index 11 priority 0 llprio 3 trunk: trunkdev trunk2 media: Ethernet autoselect (10GbaseSR full-duplex) status: active transceiver: SFP LC, 850 nm, 80m OM2, 30m OM1, 300m OM3 model: Intel Corp P.8596.02 rev A serial: F78Y3VQ, date: 2019-06-12 voltage: 3.33 V, bias current: 5.56 mA temp: 28.34 C (low -10.00 C, high 90.00 C) tx: -3.95 dBm (low -9.30 dBm, high 1.00 dBm) rx: -3.39 dBm (low -13.10 dBm, high 1.00 dBm) The UP flag is set but trunk2 had some problems. The lacp_state actor, partner and status was switching beween different states: trunkport ixl6 lacp_state actor activity,timeout,aggregation,defaulted trunkport ixl6 lacp_state partner aggregation,sync,collecting, distributing trunkport ixl6 ... trunkport ixl6 lacp_state actor activity,timeout,aggregation,expired trunkport ixl6 lacp_state partner activity,timeout,aggregation, collecting,distributing,defaulted trunkport ixl6 active ... trunkport ixl6 lacp_state actor activity,timeout,aggregation,sync, collecting,distributing,defaulted trunkport ixl6 lacp_state partner aggregation,sync,collecting, distributing trunkport ixl6 collecting,distributing ... trunkport ixl6 lacp_state actor activity,timeout,aggregation,sync, collecting,distributing,defaulted trunkport ixl6 lacp_state partner aggregation,sync,collecting, distributing trunkport ixl6 collecting,distributing I was not able get the trunk fully functional. Only a reboot could solved this issue. Furthermore simulating a broken fiber by pulling it out showed a different behavior. By plugging out the fiber of ixl6 the interface status changed correctly to status: no carrier. By plugging it back the interface status change back to status: active. And the trunk uses both trunkports correctly, good! I also tested this setup with two switches, which are configured as a mlag (multi chassis link aggregation) running Cumulus Linux. We want to use mlag to do lacp without the need for stacking. +-+ +-- + | openbsd ixl6|-| cumulus linux | | 1 ixl1|\ /| switch 1| +-+ \ / +---++--+ / || mlag +-+ / \ +---++--+ | openbsd ixl6|/ \| cumulus linux | | 2 ixl1|-| switch 2| +-+ +---+ Trunks configured with ix ports behaved stable. Switch reboots, plugging out fibers etc. didnt harm anything. Switching to ixl based trunks changed the behavior. Here we ran into serious
Block udp fragments to a single host while reassembling is on
Dear list, i want to block udp fragments to a specific host while the reassembling is turned on for all other traffic: In pf I would write something like this: # reassemble fragmented packets (default yes) set reassemble yes # scrub all traffic match all scrub (random-id no-df) # block fragments to host 10.0.0.10 block log quick from any to 10.0.0.10 fragment For me, it sounds like this is not possible, because reassembling happens before pf and it is only possible to turn it on or off as a whole, right? Is there an other way to achieve this challenge. Any advice ? Thanks, Joerg
Using /32 resp. /128 netmask for carp ips
Dear list, i want to know why it is good practice to use /32 netmask for ipv4 respectively /128 netmask for ipv6 addresses on carp interfaces, while using the "real" netmask for example /24 for a dedicated address on an interface. Any advice ? Thanks, Joerg
Re: routing traffic to transparent squid cluster
Am 15.08.2018 um 18:26 schrieb Stuart Henderson:
On 2018-08-15, George wrote:
I believe you may be looking for a redirect not a relay. It all really
depends on your network topology and what you are trying to do but in
general something like this is what you are looking at:
For directing traffic from a PF box to a separate Squid box setup
as an interception ("transparent") proxy, you want "route-to" rather
than "rdr-to" (see squid pkg-readme).
I haven't tried this with relayd but it looks like redirection with
"route to" is what's needed here.
An associated "divert-to" is also needed on the box running Squid
(again see the pkg-readme).
I tried to get the relayd part with redirects and the route-to option
running. The Problem here in my opinion is the listen option of redirects.
You could say something like: listen on ip X port 80 route-to y. So the
redirect will listen for the _destination_ ip X with port 80 and will route
this packets to Y.
But how could you say: listen for _all_ packets coming from this special ip
to any port 80 route to Y?
My first thought was to divert the relevant packets with pf to localhost and
then a redirect should do the job. The pf part is not the problem. But I didn't
find a running redirect configuration. I think grabing diverted packets from
localhost are the job of relays not redirects, right?
routing traffic to transparent squid cluster
Dear list,
i'm playing around with a squid setup, where the http traffic from a client is
transparently routed from the gateway (openbsd 6.3) to two squid caches (squid
3.5.28). This means the caches are _not_ placed on the gateway.
With PF this is very easy to achieve:
pass in quick on $INT_IF inet proto tcp from $CLIENT to any port 80 \
route-to { ( $DMZ_IF $SQUID_1), (trunk2 SQUID_2) } least-states
So far, so good. My next goal is redundancy. In other words the gateway should
stop routing traffic to an unreachable cache. Imho I thought this is very easy
to achieve with the help of relayd.
To map the upper PF rule to a fully redundant setup, I tried something like
this:
PF:
pass in quick on $INT_IF inet proto tcp from $CLIENT to any port http \
divert-to 127.0.0.1 port 3130
Relayd:
relay webproxy_3130 {
listen on 127.0.0.1 port 3130
transparent forward to port 80 check tcp mode loadbalance
}
But of course this doesn't work because the relay translates the destination
address which it should not. I didn't found any options like a pf route-to for
relays and think it wouldn't make much sense in the context of relays.
Relayd supports a route-to option for redirects but I dind't found a working
configuration.
Perhaps this is all broken by design. If so could somebody point me out a better
solution (haproxy in front of the caches)?
Any help would be greatly appreciated.
Thanks
Re: sendsyslog: dropped 4 messages, error 55
Hi, Am 30.01.2017 um 18:17 schrieb Peter Fraser: > My /var/log/messages is filling up with messages like the following: > > Jan 30 10:28:06 gateway sendsyslog: dropped 4 messages, error 55 > Jan 30 10:28:06 gateway sendsyslog: dropped 2 messages, error 55 > Jan 30 10:28:06 gateway sendsyslog: dropped 2 messages, error 55 > Jan 30 10:28:06 gateway sendsyslog: dropped 1 message, error 55 > Jan 30 10:28:06 gateway sendsyslog: dropped 2 messages, error 55 > Jan 30 10:28:06 gateway last message repeated 2 times > Jan 30 10:28:06 gateway sendsyslog: dropped 4 messages, error 55 > Jan 30 10:28:06 gateway sendsyslog: dropped 2 messages, error 55 > Jan 30 10:28:06 gateway last message repeated 2 times > Jan 30 10:28:06 gateway sendsyslog: dropped 1 message, error 55 > Jan 30 10:28:06 gateway sendsyslog: dropped 1 message, error 55 > > The messages occur in bursts with several hundred messages per burst, > and here may be several seconds or hours between the bursts. > > I am quite willing to believe that I have done something stupid, but I have no > idea what. > Any hints to find out what is generating these messages. > We observe the same problem. Our system is logging blocked packets to a remote system using logger and syslog like documented in the faqs (http://www.openbsd.org/faq/pf/logging.html). We got this messages since the upgrade to 5.9 (amd64) stable. After the upgrade to 6.0 the problem remains. I ran some test on a VM running 6.0 stable amd64. I could reproduce it with a pcap which produces around 1000 lines when I piped it through tcpdump: # tcpdump -n -e -s 160 -ttt -r /var/log/pflog2syslog | logger -t pf -p local0.info Feb 8 11:55:02 ares sendsyslog: dropped 8 messages, error 55 Feb 8 11:55:02 ares sendsyslog: dropped 4 messages, error 55 Feb 8 11:55:02 ares sendsyslog: dropped 3 messages, error 55 Feb 8 11:55:02 ares sendsyslog: dropped 8 messages, error 55 Feb 8 11:55:02 ares sendsyslog: dropped 8 messages, error 55 Feb 8 11:55:02 ares sendsyslog: dropped 9 messages, error 55 Feb 8 11:55:02 ares last message repeated 4 times Feb 8 11:55:02 ares sendsyslog: dropped 8 messages, error 55 Feb 8 11:55:02 ares last message repeated 2 times Feb 8 11:55:02 ares sendsyslog: dropped 5 messages, error 55 Feb 8 11:55:02 ares sendsyslog: dropped 1 message, error 55 Feb 8 11:55:02 ares sendsyslog: dropped 8 messages, error 55 Feb 8 11:55:02 ares sendsyslog: dropped 9 messages, error 55 Feb 8 11:55:02 ares last message repeated 5 times Feb 8 11:55:02 ares sendsyslog: dropped 8 messages, error 55 Feb 8 11:55:02 ares last message repeated 2 times Feb 8 11:55:02 ares sendsyslog: dropped 4 messages, error 55 Feb 8 11:55:02 ares sendsyslog: dropped 2 messages, error 55 Feb 8 11:55:02 ares sendsyslog: dropped 8 messages, error 55 Feb 8 11:55:02 ares sendsyslog: dropped 9 messages, error 55 Feb 8 11:55:02 ares last message repeated 5 times Feb 8 11:55:02 ares sendsyslog: dropped 8 messages, error 55 dmesg: OpenBSD 6.0 (GENERIC.MP) #2: Mon Oct 17 10:22:47 CEST 2016 r...@stable-60-amd64.mtier.org:/binpatchng/work-binpatch60-amd64/src/sys/arch /amd64/compile/GENERIC.MP real mem = 4265054208 (4067MB) avail mem = 4131319808 (3939MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xbf49c000 (84 entries) bios0: vendor Dell Inc. version "3.0.0" date 01/31/2011 bios0: Dell Inc. PowerEdge R710 acpi0 at bios0: rev 2 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP APIC SPCR HPET DM__ MCFG WD__ SLIC ERST HEST BERT EINJ SRAT TCPA SSDT acpi0: wakeup devices PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 32 (boot processor) cpu0: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 2926.41 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1 GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT cpu0: 256KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 1 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 132MHz cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE cpu1 at mainbus0: apid 34 (application processor) cpu1: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 2926.00 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1 GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT cpu1: 256KB 64b/line 8-way L2 cache cpu1: smt 0, core 1, package 1 cpu2 at mainbus0: apid 50 (application processor) cpu2: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 2926.00 MHz cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
PERC H730 mini on 5.6
Dear list, i'm in progress in installing 5.6 stable on a Poweredge R730. This system has a PERC H730 mini raid controller. The OpenBSD installer aborts with the following message when fdisk wants read disk geometry: snip fdisk: DIOCGPDINFO: Input/output error fdisk: Can't get disk geometry, please use [-chs] to specify. MBR has invalid signature; not showing it. ... snap I tried to specify chs manually with no success. Then I tried a current snapshot with no problems. But for the productive setup I want using the stable release. What are my options? Waiting still May 1st until 5.7 will be released? Thanks in advance, Joerg [demime 1.01d removed an attachment of type application/pkcs7-signature which had a name of smime.p7s]
Re: PERC H730 mini on 5.6
function 6 not configured Intel E5 v3 DDRIO rev 0x02 at pci11 dev 20 function 7 not configured Intel E5 v3 Thermal rev 0x02 at pci11 dev 21 function 0 not configured Intel E5 v3 Thermal rev 0x02 at pci11 dev 21 function 1 not configured Intel E5 v3 Error rev 0x02 at pci11 dev 21 function 2 not configured Intel E5 v3 Error rev 0x02 at pci11 dev 21 function 3 not configured Intel E5 v3 TA rev 0x02 at pci11 dev 22 function 0 not configured Intel E5 v3 DDR Broadcast rev 0x02 at pci11 dev 22 function 6 not configured Intel E5 v3 DDR Broadcast rev 0x02 at pci11 dev 22 function 7 not configured Intel E5 v3 Thermal rev 0x02 at pci11 dev 23 function 0 not configured Intel E5 v3 DDRIO rev 0x02 at pci11 dev 23 function 4 not configured Intel E5 v3 DDRIO rev 0x02 at pci11 dev 23 function 5 not configured Intel E5 v3 DDRIO rev 0x02 at pci11 dev 23 function 6 not configured Intel E5 v3 DDRIO rev 0x02 at pci11 dev 23 function 7 not configured Intel E5 v3 PCU rev 0x02 at pci11 dev 30 function 0 not configured Intel E5 v3 PCU rev 0x02 at pci11 dev 30 function 1 not configured Intel E5 v3 PCU rev 0x02 at pci11 dev 30 function 2 not configured Intel E5 v3 PCU rev 0x02 at pci11 dev 30 function 3 not configured Intel E5 v3 PCU rev 0x02 at pci11 dev 30 function 4 not configured Intel E5 v3 VCU rev 0x02 at pci11 dev 31 function 0 not configured Intel E5 v3 VCU rev 0x02 at pci11 dev 31 function 2 not configured isa0 at mainbus0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo pckbc0 at isa0 port 0x60/5 uhub2 at uhub0 port 1 vendor 0x8087 product 0x800a rev 2.00/0.05 addr 2 uhub3 at uhub2 port 5 vendor 0x05e3 USB2.0 Hub rev 2.00/7.02 addr 3 uhub4 at uhub3 port 4 vendor 0x04b4 product 0x6560 rev 2.00/0.08 addr 4 uhidev0 at uhub4 port 4 configuration 1 interface 0 vendor 0x10d5 UKU-M02 1.7 rev 1.10/0.00 addr 5 uhidev0: iclass 3/1 ukbd0 at uhidev0 wskbd0 at ukbd0: console keyboard, using wsdisplay0 uhidev1 at uhub4 port 4 configuration 1 interface 1 vendor 0x10d5 UKU-M02 1.7 rev 1.10/0.00 addr 5 uhidev1: iclass 3/1, 3 report ids uhid at uhidev1 reportid 1 not configured uhid at uhidev1 reportid 2 not configured uhid at uhidev1 reportid 3 not configured uhub5 at uhub2 port 6 no manufacturer Gadget USB HUB rev 2.00/0.00 addr 6 uhidev2 at uhub5 port 1 configuration 1 interface 0 Avocent Keyboard/Mouse Function rev 2.00/0.00 addr 7 uhidev2: iclass 3/1 ukbd1 at uhidev2 wskbd1 at ukbd1 mux 1 wskbd1: connecting to wsdisplay0 uhidev3 at uhub5 port 1 configuration 1 interface 1 Avocent Keyboard/Mouse Function rev 2.00/0.00 addr 7 uhidev3: iclass 3/1 uhid at uhidev3 not configured uhidev4 at uhub5 port 1 configuration 1 interface 2 Avocent Keyboard/Mouse Function rev 2.00/0.00 addr 7 uhidev4: iclass 3/1 uhid at uhidev4 not configured umass0 at uhub5 port 4 configuration 1 interface 0 Avocent Mass Storage Function rev 2.00/0.00 addr 8 umass0: using SCSI over Bulk-Only scsibus4 at umass0: 2 targets, initiator 0 cd1 at scsibus4 targ 1 lun 0: iDRAC, Virtual CD, 0329 SCSI0 5/cdrom removable sd1 at scsibus4 targ 1 lun 1: iDRAC, Virtual Floppy, 0329 SCSI0 0/direct removable uhub6 at uhub1 port 1 vendor 0x8087 product 0x8002 rev 2.00/0.05 addr 2 softraid0 at root scsibus5 at softraid0: 256 targets root on rd0a swap on rd0b dump on rd0b wskbd1: disconnecting from wsdisplay0 wskbd1 detached ukbd1 detached uhidev2 detached uhidev3 detached uhidev4 detached uhidev2 at uhub5 port 1 configuration 1 interface 0 Avocent Keyboard/Mouse Function rev 2.00/0.00 addr 7 uhidev2: iclass 3/1 ukbd1 at uhidev2 wskbd1 at ukbd1 mux 1 wskbd1: connecting to wsdisplay0 uhidev3 at uhub5 port 1 configuration 1 interface 1 Avocent Keyboard/Mouse Function rev 2.00/0.00 addr 7 uhidev3: iclass 3/1 uhid at uhidev3 not configured uhidev4 at uhub5 port 1 configuration 1 interface 2 Avocent Keyboard/Mouse Function rev 2.00/0.00 addr 7 uhidev4: iclass 3/1 uhid at uhidev4 not configured snap Am 10.03.2015 um 12:27 schrieb Joerg Streckfuss: Dear list, i'm in progress in installing 5.6 stable on a Poweredge R730. This system has a PERC H730 mini raid controller. The OpenBSD installer aborts with the following message when fdisk wants read disk geometry: snip fdisk: DIOCGPDINFO: Input/output error fdisk: Can't get disk geometry, please use [-chs] to specify. MBR has invalid signature; not showing it. ... snap I tried to specify chs manually with no success. Then I tried a current snapshot with no problems. But for the productive setup I want using the stable release. What are my options? Waiting still May 1st until 5.7 will be released? Thanks in advance, Joerg [demime 1.01d removed an attachment of type application/pkcs7-signature which had a name of smime.p7s]
Re: Accept two vlans
Am 07.08.2013 16:20, schrieb Christian Weisgerber: Well, you can either use two NICs on your gateway, one connected to a vlan1 port on the switch, the other to vlan2. Or you can can set up vlan1 and vlan2 on em0 and connect them to a trunk port on the switch. This is straight from my home gateway: == /etc/hostname.em0 == description Trunk up == /etc/hostname.vlan1 == description LAN vlan 1 vlandev em0 inet 172.16.0.1 255.255.255.0 NONE inet6 2001:6f8:124a::1 == /etc/hostname.vlan2 == description WLAN vlan 2 vlandev em0 inet 172.16.1.1 255.255.255.0 NONE inet6 2001:6f8:124a:1::1 I'm just a little bit curious. Why do you use VLANs instead of just a physical interface for each lan (wlan). Is it because VLANs give you a little bit more flexibility? By Joerg [demime 1.01d removed an attachment of type application/pkcs7-signature which had a name of smime.p7s]
Re: Relayd redirect does not work under high packet rate suddenly
Okay, I can reproduce the problem. In the nearly 80 % (by guess value) of cases the relayd stops forwarding packets in the given situation: - first the services of the master host goes down. - relayd switches to the backup pool. requests are redirected to the backup host. - master host revives. - relayd recognizes the initial master host as available and switches back. - Syns are stalled at the firewall. No blocked packets are logged. - after restarting relayd everything operates as desired. So the first assumption the issue is a consequence of a high packet rate is only true for the case of a unavailable master host which comes back after in a short time. I notice that when the described situation occurred, for every stalled syn the memory counter of pf increases by one. I set the debuglevel to warning (short test for notice generated about 400 lines per second in my logs) but no warnings or errors are logged. I changed the hard limits but this doesn't helped as well. stateshard limit 128000 src-nodes hard limit 128000 frags hard limit 5000 tableshard limit 1000 table-entries hard limit 20 I'm absolutely sure the states hard limit is sufficient. During my last test session pfctl -si shows me less than 30K states. What else I notice is that relayd won't cleanup its tables and anchors after a restart by its rc-script when configuration changes are done. I have to do a pfctl -a relayd/$anchor -Fa manually. If we cant solve it we have to search for an alternative solution. Perhaps could someone recommend HAproxy on OpenBSD as a replacement for relayd? Of course this would be the last resort. Thanks, Joerg dmesg: OpenBSD 5.0 (GENERIC.MP) #63: Wed Aug 17 10:14:30 MDT 2011 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 4280782848 (4082MB) avail mem = 4152709120 (3960MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xbf49c000 (84 entries) bios0: vendor Dell Inc. version 3.0.0 date 01/31/2011 bios0: Dell Inc. PowerEdge R710 acpi0 at bios0: rev 2 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP APIC SPCR HPET DM__ MCFG WD__ SLIC ERST HEST BERT EINJ SRAT TCPA SSDT acpi0: wakeup devices PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 32 (boot processor) cpu0: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 2926.45 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,S S,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,LONG cpu0: 256KB 64b/line 8-way L2 cache cpu0: apic clock running at 132MHz cpu1 at mainbus0: apid 34 (application processor) cpu1: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 2926.00 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,S S,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,LONG cpu1: 256KB 64b/line 8-way L2 cache cpu2 at mainbus0: apid 50 (application processor) cpu2: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 2926.00 MHz cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,S S,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,LONG cpu2: 256KB 64b/line 8-way L2 cache cpu3 at mainbus0: apid 52 (application processor) cpu3: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 2926.00 MHz cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,S S,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,LONG cpu3: 256KB 64b/line 8-way L2 cache cpu4 at mainbus0: apid 33 (application processor) cpu4: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 2926.00 MHz cpu4: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,S S,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,LONG cpu4: 256KB 64b/line 8-way L2 cache cpu5 at mainbus0: apid 35 (application processor) cpu5: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 2926.00 MHz cpu5: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,S S,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,LONG cpu5: 256KB 64b/line 8-way L2 cache cpu6 at mainbus0: apid 51 (application processor) cpu6: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 2926.00 MHz cpu6: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,S S,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,LONG cpu6: 256KB 64b/line 8-way L2 cache cpu7 at mainbus0: apid 53 (application
Relayd redirect does not work under high packet rate suddenly
Hi list,
since an Upgrade to 5.0 of our pf-cluster we encoutered connection problems of
one of our webservers under high packet rate. We messured a load of about 6
million and more hits per day. The webserver serves little static content around
a few KByte.
I'm not really sure if this problem belongs to the upgrade but it occured the
first time two days after it.
The pf-cluster runs a relayd session which checks the availability of the
webserver. If it fails the relayd redirects traffic to a backup pool of three
other servers. Not a very sophisticated setup.
The problem indicates as follows: Suddenley one day the Webserver wasn't
available anymore from outside. Before we messured a high packet rate but this
wasn't a new situation. After some debugging we realized that the firewall
absorbed the syns. Tcpdump shows incoming syns but no outgoing. No blocked
packets were logged and the relayd showed 100% availability of the webserver. A
failover to the backup node solved nothing, so this was also true for the backup
node. Perhaps this results from using pfsync between both nodes. We decided to
configure relayd using the backup pool by deactivating the master host. This
step solved it and the services was available again from the outside. After
looking for a needle in a haystack we restarted the relayd an voila the redirect
to the master operated as desired. But after 20 min we encountered the same
behaviour.
At the moment we have deactivated the redirect for the webserver but of course
this is not what we want. I have no clue where i can looking for.
Enclosed some configurations and specs:
relayd.conf:
$public=A.B.C.D
$master=A.B.C.E
$backup=A.B.C.F
$backup_2= A.B.C.G
$backup_3= A.B.C.H
timeout 3000
interval 5
log updates
table master { $master retry 2 }
table backup { $backup $backup_2 $backup_3 retry 2 }
redirect webserver {
listen on $public port http interface trunk0
tag INFO_IF
forward to master check http / code 200
forward to backup check http / code 200
}
We set the state limit of pf to 128000.
# pfctl -sm:
stateshard limit 128000
src-nodes hard limit1
frags hard limit 5000
tableshard limit 1000
table-entries hard limit 20
# pfctl -si
Status: Enabled for 32 days 04:18:51 Debug: err
...
State Table Total Rate
current entries44109
searches 16910570167 6082.2/s
inserts547129329 196.8/s
removals 547085220 196.8/s
Counters
match 560027964 201.4/s
bad-offset 00.0/s
fragment 130010.0/s
short79460032.9/s
normalize 14960.0/s
memory 41469515 14.9/s
bad-timestamp 00.0/s
congestion 5860.0/s
ip-option 00.0/s
proto-cksum00.0/s
state-mismatch1785410.1/s
state-insert 3540.0/s
state-limit 380.0/s
src-limit2320.0/s
synproxy 509491094 183.2/s
The memory counter makes me a little bit nervous. Does that mean PF cannot
allocate memory for new states? We have 4GB of Ram. I think this should be
enough for a limit of 128000 states. By the way, we have fairly complex ruleset
about 1300 rules.
vmstat reports no failed allocations:
# vmstat -m | grep -E 'pfstatepl|Fail'
NameSize Requests FailInUse Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
pfstatepl304 567101330 045917 194790 190762 4028 7320 0 88
Specs:
2 X PowerEdge R710 running OpenBSD 5.0 amd64 MP
INTEL XEON X5647 @ 2.93GHZ 4C
4 GB RAM
Regards,
Joerg
Re: asymmetric CARP firewall layout
Am 20.07.2011 00:31, schrieb Kapetanakis Giannis: On 19/07/11 20:03, Joerg Streckfuss wrote: Hi list, i have the following testsetup with four firewall nodes connected to three networks: network A |--| || CARP || || || +--+--+ +--+--++--+--+ +--+--+ | fw1 | | fw2 || fw3 | | fw4 | +--+--+ +--+--++--+--+ +--+--+ || || | CARP | | CARP | |--||--| network B network C As you can see all four nodes are connected to network A but only fw1 and fw2 are connected to network B. On the other side only fw3 and fw4 are connected to network C. For network A all nodes form a CARP cluster. The order of priority for which node is in master mode is: fw1 - fw2 - fw3 - fw4. For network B fw1 and fw2 form a CARP cluster and order of priority is fw1 - fw2. And last but not least for network C fw3 and fw4 form a CARP cluster and order of priority is fw3 - fw4. Preempting is active on all nodes. The point which gives me a headache is that normaly fw3 is master for network C but backup for network A. Not very suprising. I know this is a very uncommon setup but it works for me for many days know. A failover to node fw3 respectively fw4 on network A performs as expected. Are there any possible site effects i have overlooked. Many thanks in advance, Joerg If fw1 is master for network A, how do you route traffic from A to C? This is not really a problem because it is not required. On of the main requirements is that the hosts on network A are all using the same gateway and the routing into and out of network A is always symmetric. My description of network B and C was a bit ambiguous. So let me go a littler bit deeper into the details: network A |--| || CARP || || || +--+--+ +--+--++--+--+ +--+--+ | fw1 | | fw2 || fw3 | | fw4 | +--+--+ +--+--++--+--+ +--+--+ || || | CARP | | CARP | |--||--| | | | | +--+--+ +--+--+ | R1 | | R2 | +--+--+ +--+--+ | | |___| / \ / Internet Cloud \ \ / \_/ R1 and R2 are routers which are gateways to the internet. So the only purpose for network B and C is connecting the routers with the firewalls. R2 is only for backup. I would put fw1 fw2 in CARP A1 and fw3 fw4 in CARP A2 (different vhid, different virt IP) or make all firewalls listen on all networks (A,B,C) with no asymmetry. As mentioned above routing should be always symmetric. If on of the hosts of network A is using gate A1 and another is using gate A2 the routing is asymmetric. regards, Joerg
asymmetric CARP firewall layout
Hi list, i have the following testsetup with four firewall nodes connected to three networks: network A |--| || CARP || || || +--+--+ +--+--++--+--+ +--+--+ | fw1 | | fw2 || fw3 | | fw4 | +--+--+ +--+--++--+--+ +--+--+ || || | CARP | | CARP | |--||--| network B network C As you can see all four nodes are connected to network A but only fw1 and fw2 are connected to network B. On the other side only fw3 and fw4 are connected to network C. For network A all nodes form a CARP cluster. The order of priority for which node is in master mode is: fw1 - fw2 - fw3 - fw4. For network B fw1 and fw2 form a CARP cluster and order of priority is fw1 - fw2. And last but not least for network C fw3 and fw4 form a CARP cluster and order of priority is fw3 - fw4. Preempting is active on all nodes. The point which gives me a headache is that normaly fw3 is master for network C but backup for network A. Not very suprising. I know this is a very uncommon setup but it works for me for many days know. A failover to node fw3 respectively fw4 on network A performs as expected. Are there any possible site effects i have overlooked. Many thanks in advance, Joerg
relayd: possible to redirect IPv4 requests to IPv6 pool?
Dear list, it's just an idea but in times like these where IPv4 adresses are a scarce resource, i think about the following purpose: Can it be possible to use the relayd to redirect IPv4 Requests to a IPv6 pool of Servers? Regards, Jvrg [demime 1.01d removed an attachment of type application/pkcs7-signature which had a name of smime.p7s]
Intel PRO/1000 QP on Dell R610 and OpenBSD 4.7
Hi list, we bought two Dell R610 Servers with four built-in Broadcom BCM5709 nics. Additionally we installed one Intel PRO/1000 QP quad port nic. There are no problems with the Broadcoms but something strange happens to the Intel nic. Sometimes, almost always one to two ports of the intel card couldn't initialized. The OS comments this with the following message snip em1 at pci5 dev 0 function 1 Intel PRO/1000 QP (82576) rev 0x01: apic 1 int 14 (irq 10)em1: Hardware Initialization Failed em1: Unable to initialize the hardware snap We are runnig OpenBSD 4.7 stable and dmesg says: OpenBSD 4.7 (GENERIC) #558: Wed Mar 17 20:46:15 MDT 2010 dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC cpu0: Intel(R) Xeon(R) CPU L5506 @ 2.13GHz (GenuineIntel 686-class) 2.13 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16, xTPR real mem = 3479244800 (3318MB) avail mem = 3383246848 (3226MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 04/06/10, BIOS32 rev. 0 @ 0xfa040, SMBIOS rev. 2.6 @ 0xcf79c000 (83 entries) bios0: vendor Dell Inc. version 2.0.13 date 04/06/2010 bios0: Dell Inc. PowerEdge R610 acpi0 at bios0: rev 2 acpi0: tables DSDT FACP APIC SPCR HPET DM__ MCFG WD__ SLIC ERST HEST BERT EINJ SRAT TCPA SSDT acpi0: wakeup devices PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 16 (boot processor) cpu0: unknown i686 model 0x1a, can't get bus clock (0x0) cpu0: apic clock running at 133MHz cpu at mainbus0: not configured cpu at mainbus0: not configured cpu at mainbus0: not configured ioapic0 at mainbus0: apid 0 pa 0xfec0, version 20, 24 pins ioapic1 at mainbus0: apid 1 pa 0xfec8, version 20, 24 pins ioapic1: misconfigured as apic 0, remapped to apid 1 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (PEX1) acpiprt2 at acpi0: bus 2 (PEX3) acpiprt3 at acpi0: bus -1 (PEX4) acpiprt4 at acpi0: bus -1 (PEX5) acpiprt5 at acpi0: bus -1 (PEX6) acpiprt6 at acpi0: bus 4 (PEX7) acpiprt7 at acpi0: bus 8 (PEX9) acpiprt8 at acpi0: bus -1 (PEXA) acpiprt9 at acpi0: bus 3 (SBEX) acpiprt10 at acpi0: bus 9 (COMP) acpicpu0 at acpi0: C3, C1, PSS bios0: ROM list: 0xc/0x8000 0xc8000/0x5c00 0xec000/0x4000! ipmi at mainbus0 not configured cpu0: EST: PSS not yet available for this processor pci0 at mainbus0 bus 0: configuration mode 1 (bios) pchb0 at pci0 dev 0 function 0 Intel 5500 Host rev 0x13 ppb0 at pci0 dev 1 function 0 Intel X58 PCIE rev 0x13 pci1 at ppb0 bus 1 bnx0 at pci1 dev 0 function 0 Broadcom BCM5709 rev 0x20: apic 1 int 4 (irq 15) bnx1 at pci1 dev 0 function 1 Broadcom BCM5709 rev 0x20: apic 1 int 16 (irq 14) ppb1 at pci0 dev 3 function 0 Intel X58 PCIE rev 0x13 pci2 at ppb1 bus 2 bnx2 at pci2 dev 0 function 0 Broadcom BCM5709 rev 0x20: apic 1 int 0 (irq 15) bnx3 at pci2 dev 0 function 1 Broadcom BCM5709 rev 0x20: apic 1 int 10 (irq 14) ppb2 at pci0 dev 7 function 0 Intel X58 PCIE rev 0x13: apic 1 int 21 (irq 0) pci3 at ppb2 bus 4 ppb3 at pci3 dev 0 function 0 IDT 89HPES12N3A rev 0x0e pci4 at ppb3 bus 5 ppb4 at pci4 dev 2 function 0 IDT 89HPES12N3A rev 0x0e pci5 at ppb4 bus 6 em0 at pci5 dev 0 function 0 Intel PRO/1000 QP (82576) rev 0x01: apic 1 int 15 (irq 11), address 00:1b:21:61:58:b0 em1 at pci5 dev 0 function 1 Intel PRO/1000 QP (82576) rev 0x01: apic 1 int 14 (irq 10)em1: Hardware Initialization Failedem1: Unable to initialize the hardware ppb5 at pci4 dev 4 function 0 IDT 89HPES12N3A rev 0x0e pci6 at ppb5 bus 7 em2 at pci6 dev 0 function 0 Intel PRO/1000 QP (82576) rev 0x01: apic 1 int 6 (irq 15), address 00:1b:21:61:58:b4 em3 at pci6 dev 0 function 1 Intel PRO/1000 QP (82576) rev 0x01: apic 1 int 13 (irq 14)em3: Hardware Initialization Failedem3: Unable to initialize the hardware ppb6 at pci0 dev 9 function 0 Intel X58 PCIE rev 0x13: apic 1 int 21 (irq 0) pci7 at ppb6 bus 8 Intel X58 Misc rev 0x13 at pci0 dev 20 function 0 not configured Intel X58 GPIO rev 0x13 at pci0 dev 20 function 1 not configured Intel X58 RAS rev 0x13 at pci0 dev 20 function 2 not configured uhci0 at pci0 dev 26 function 0 Intel 82801I USB rev 0x02: apic 0 int 17 (irq 14) uhci1 at pci0 dev 26 function 1 Intel 82801I USB rev 0x02: apic 0 int 18 (irq 11) ehci0 at pci0 dev 26 function 7 Intel 82801I USB rev 0x02: apic 0 int 19 (irq 10) usb0 at ehci0: USB revision 2.0 uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb7 at pci0 dev 28 function 0 Intel 82801I PCIE rev 0x02 pci8 at ppb7 bus 3 mpi0 at pci8 dev 0 function 0 Symbios Logic SAS1068E rev 0x08: apic 0 int 16 (irq 15) scsibus0 at mpi0: 112 targets sd0 at scsibus0 targ 0 lun 0: Dell, VIRTUAL DISK, 1028 SCSI3 0/direct fixed sd0: 237952MB, 512 bytes/sec, 487325696 sec total ses0 at scsibus0 targ 8 lun 0: DP, BACKPLANE, 1.07 SCSI3 13/enclosure services fixed uhci2 at pci0 dev 29 function 0 Intel 82801I USB rev 0x02: apic 0 int 21
Re: Intel PRO/1000 QP on Dell R610 and OpenBSD 4.7
Am 04.06.2010 13:18, schrieb Sevan / Venture37: Test a snapshot to see if the issue still exists Sevan / Venture37 Okey, we tested the newest snapshot but the issue remains. Any other clue? Joerg [demime 1.01d removed an attachment of type application/pkcs7-signature which had a name of smime.p7s]
prioritizing carp interfaces
Hi list, I have a theoretical question regarding a CARP cluster and many CARP interfaces Assume we have a firewall comprising of two notes, each with 4 or more interfaces and only one uplink to the internet. The Cluster is in master/backup mode How does CARP behaves when on the master node two unimportantly interfaces fail and on the backup node only the uplink interface fails? Does CARP failover to the backup node and as consequence the whole network will be disconnected from the internet? In my mind one solution to avoid this situation is to rate the CARP interfaces. For example a more important interface gets a higher rate than a less important interface. Probably the ifstated deamon and the demotion counter are the topics to get around with this. Does anybody have experiences demotion couter and ifstated? Thanks in advance. Joerg -- Dipl.-Ing. (FH) Joerg Streckfuss, Phone: +49 40 808077-631 DFN-CERT Services GmbH, https://www.dfn-cert.de/, Phone +49 40 808077-555 Sitz / Register: Hamburg, AG Hamburg, HRB 88805, Ust-IdNr.: DE 232129737 Sachsenstra_e 5, 20097 Hamburg/Germany, CEO: Dr. Klaus-Peter Kossakowski [demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]
Re: prioritizing carp interfaces
Well, looks interesting, but I didn't try it. It maybe too complicated, when redundancy need to be as simply as possible. Instead of this, you can just add another node(s), this is the safest solution, I think. Well, another node implies two nodes for redundancy. And two independant firewall clusters means two independent rulsets to manage. I think i will try ifstated with a finite state machine based on ping test and demotion counter. -- Dipl.-Ing. (FH) Joerg Streckfuss, Phone: +49 40 808077-631 DFN-CERT Services GmbH, https://www.dfn-cert.de/, Phone +49 40 808077-555 Sitz / Register: Hamburg, AG Hamburg, HRB 88805, Ust-IdNr.: DE 232129737 Sachsenstra_e 5, 20097 Hamburg/Germany, CEO: Dr. Klaus-Peter Kossakowski [demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]
Re: relayd - conflict between outer and inner ip address
Am Mon, 10 Nov 2008 17:42:50 +0100
schrieb JC6rg StreckfuC [EMAIL PROTECTED]:
Hello,
I'm in the process to setup relayd as a loadbalancer, which will distribute
http request to three webservers. I think this is a really common setup.
I'm using OpenBSD 4.4
this is my config:
snip
www_public=10.0.0.1
www1= 10.0.0.1
www2= 10.0.0.2
www3= 10.0.0.3
interval 10
timeout 300
prefork 1
table hosts { $www1 $www2 $www3 }
redirect {
listen on $www_public port http
tag RELAYD
forward to hosts \
check http /index.html code 200 mode roundrobin
}
/snip
It seems that the first ip (10.0.0.1) which is also the public ip for the
webserver pool is unavailable. Each request, which should be forwarded to
the
first webserver will stuck for a moment and then relayd redirects it to
the next server in the pool. The ip address 10.0.0.1 will be skipped at any
time.
For me it looks like an ip conflict between relayd and the first webserver
If I take a different ip for $www_public, e.g 10.0.0.4 relayd works as
expected.
So is it mandatory for the directive 'listen on ...' to chose an ip address
which is not part of the webserver pool?
If it is possible to run relayd as described above i would prefer it
because
if
relayd stops working, the server with the outer ip address is still
reachable.
Otherwise in case of a failure the entire cluster will be unavailable.
Perhaps there are smarter ways to increase availability of relayd.
Regards, Joerg
Okey, i think i was missing some useful information about my setup
The loadbalancer consists of two dell 2850 server. Each system
is equipped with quad port network devices (d-link dfe580-tx supported by the
ste driver). Of course I'm using carp for redundancy between the boxes.
The configuration for relayd on the master is identical with the
configuration on the backup host.
box A (master)pfsyncbox B (backup)
running relayd running relayd
| |
+---+
|
VIP: 10.0.0.5
|
---
| | |
www1 www2www3
10.0.0.1 10.0.0.2 10.0.0.3
I hope this will help.
Regards, Joerg
Re: relayd - conflict between outer and inner ip address
Since this is redirect, it should work, providing you don't configure 10.0.0.1 as an IP address on the loadbalancer itself. I quite agree. The loadbalancer is configured with IP address 10.0.0.5 (CARP). Only the directive listen on ... for the rediect in the relayd configuration uses IP 10.0.0.1.
Re: how to manage big pf-rulesets in a comfortable way
Hi Marc, Thanks for your advice but i have already tested fwbuilder. The builder is nice to edit a big ruleset, but i dislike the concept of global- and interface-policy. In global policy-section i missed the direction for packets. An example: If you want to edit some antispoof rules, you have to use the interface policies because of the direction and so you have to write more rules than only say antispoof for $ext_if inet in pf.conf. Futhermore i missed some features like synproxy, statefull tracking options an bandwith management. cheers Joerg. Am Donnerstag, den 02.02.2006, 14:17 +0100 schrieb Marc Peters: hi joerg, you may want to have a look at firewall builder (www.fwbuilder.org). it can produce rulesets for pf, but you should have a look at the conf later on and check the ruleset if it fits your needs. hth, marc -- Joerg Streckfuss, DFN-CERT Services GmbH PGP RSA/2048, E0D4BD3F, 90 C3 FB 4A CB D3 20 70 6B 04 47 84 B5 3C 28 8C [demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]
how to manage big pf-rulesets in a comfortable way
Hi list, i need some hints to manage a pf ruleset of about more than 150 rules. In my company we want to design a firewall-cluster with about 10 interfaces. We plan to use two dell 1850 with two DFE-580TX quad port NIC's. Each interface points to a separate subnet. The cluster should use carp for redundancy. The problem is to manage the hole ruleset in a comfortable way. One of my ideas is to put the ruleset of each subnet into an extra file and load it into pf with anchors. This will reduce the main ruleset extremely. The disadvantage is that all macros listed in the main ruleset have to be listed in the subnet ruleset too - this is a little bit error-prone. In my opinion bandwith managment with separate files is not an elegant way as well. Interface groups are not the solution, because the subnet rulesets are too different. At the end, i have to put all rules into a single file. So is there a better way to handle big rulesets? Cheers Joerg. -- Joerg Streckfuss, DFN-CERT Services GmbH PGP RSA/2048, E0D4BD3F, 90 C3 FB 4A CB D3 20 70 6B 04 47 84 B5 3C 28 8C [demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]
errata 001_perl.patch
hi list. last night i patched my openbsd-3.8 soekris-box. Everything went fine. I've got another box for firewalling with 512MB-flash standard setup, but without any compiler-suite installed. Of course i want to patch this box as soon as possible. shoud i copy the complete perl-files to this box? or is there a smarter way to have an upgraded system? Regards, Joerg. -- Dipl.-Ing. Joerg StreckfuC fon:+49 40 - 41 11 66 86 cell: +49 179 - 49 88 51 0 mail: [EMAIL PROTECTED]
