Re: bnxt(4), myx(4), vr(4): refill timeouts: timeout_add(..., 0) -> timeout_add(..., 1)

[Hrvoje Popovski]

On 20.1.2020. 17:40, Scott Cheloha wrote:
> Appreciate the testing.

np, i like testing network stuff :)

> Given what dlg@ has said in the past I think there should only be a
> performance change in a livelock situation. 

yeah, that could be problem with this testing ...
kern.netlivelocks=6 after few minutes of sending 14Mpps through myx :)

> What metric did you use to compare performance?

i'm generating 64byte udp packets with pktgen through openbsd box (plain
forwarding only)
https://www.kernel.org/doc/Documentation/networking/pktgen.txt

on linux box i'm counting pps on interface from which i'm sending
packets to openbsd box and on interface on which i'm receiving packets
from openbsd box ..

on box with 12 x Xenon CPU E5-2620 v2 @ 2.10GHz, 2400.01 MHz, 06-3e-04
with or without this diff i'm getting 890Kpps of plain forwarding

Re: em(4) diff to test

[Hrvoje Popovski]

On 20.1.2020. 16:42, Martin Pieuchot wrote:
> Diff below is a refactoring of the actual em(4) code and defines that
> will allows me to present a shorter diff to interrupt multiple CPUs and
> make use of multiple queues.
> 
> It contains the following items:
> 
>   - Abstract the allocation/freeing of TX/RX ring into em_dma_malloc().
> This will ease the introduction of multiple rings.
> 
>   - Split the 82576 variant out of 82575.  The distinction is necessary
> when it comes to setting multiple queues.
> 
>   - Change multiple TX/RX related macro to take an index argument
> corresponding to a ring.  Currently only the index 0 and 1 are used.
> 
>   - Gather and print more stats counters
> 
>   - Switch to using a function, like FreeBSD, to translate 82542
> registers and get rid of a set of defines.
> 
> It has been tested one the models below, I'd like to be sure there isn't
> any fallout with this part before continuing the effort.
> 
>   em0 at pci0 dev 25 function 0 "Intel 82577LM" rev 0x06: msi
>   em0 at pci0 dev 25 function 0 "Intel 82579LM" rev 0x04: msi
>   em0 at pci0 dev 25 function 0 "Intel I218-V" rev 0x03: msi
>   em0 at pci0 dev 25 function 0 Intel I218-LM rev 0x04: msi
>   em0 at pci0 dev 31 function 6 "Intel I219-V" rev 0x21: msi

Hi,

i tested this diff on
em0 at pci7 dev 0 function 0 "Intel I350" rev 0x01: msi
em1 at pci7 dev 0 function 1 "Intel I350" rev 0x01: msi

with lots of ifconfig em up/down and sh netstart and box seems stable

Re: bnxt(4), myx(4), vr(4): refill timeouts: timeout_add(..., 0) -> timeout_add(..., 1)

[Hrvoje Popovski]

On 16.1.2020. 18:45, Scott Cheloha wrote:
> Here's a first batch of conversions: rx refill timeouts for bnxt(4),
> myx(4), and vr(4).  All of these can run during a softclock().  Will
> changing these to one tick break these drivers?

Hi all,

i tried this diff with myx and performance are the same as with plain
kernel.

myx0 at pci4 dev 0 function 0 "Myricom Z8E" rev 0x01: msi, model
10G-PCIE2-8BL2-2S, address 00:60:dd:45:ba:f8
myx1 at pci5 dev 0 function 0 "Myricom Z8E" rev 0x01: msi, model
10G-PCIE2-8BL2-2S, address 00:60:dd:45:ba:f9

acpicpu log in dmesg

[Hrvoje Popovski]

Hi all,

with today's snapshot i'm seeing acpipci log below in dmesg. is this ok?
do i need to report it ?


acpipci0 at acpi0 PC00: 0x 0x0011 0x0001
extent `acpipci0 pcibus' (0x0 - 0xff), flags=0
 0x40 - 0xff
extent `acpipci0 pciio' (0x0 - 0x), flags=0
 0x3b0 - 0x3df
 0xcf8 - 0xcff
 0x4000 - 0x
extent `acpipci0 pcimem' (0x0 - 0x), flags=0
 0x0 - 0xb
 0x10 - 0xe0ff
 0xfec0 - 0x63dbfff
 0x1 - 0x
acpicmos0 at acpi0
"IPI0001" at acpi0 not configured
acpipci1 at acpi0 PC01: 0x 0x0011 0x0001
extent `acpipci1 pcibus' (0x0 - 0xff), flags=0
 0x0 - 0x3f
 0x80 - 0xff
extent `acpipci1 pciio' (0x0 - 0x), flags=0
 0x0 - 0x3fff
 0x7000 - 0x
extent `acpipci1 pcimem' (0x0 - 0x), flags=0
 0x0 - 0x8fff
 0xab00 - 0x47e7fff
 0x63dc000 - 0x
acpipci2 at acpi0 PC02: 0x 0x0011 0x0001
extent `acpipci2 pcibus' (0x0 - 0xff), flags=0
 0x0 - 0x7f
 0xc0 - 0xff
extent `acpipci2 pciio' (0x0 - 0x), flags=0
 0x0 - 0x6fff
 0xa000 - 0x
extent `acpipci2 pcimem' (0x0 - 0x), flags=0
 0x0 - 0xaaff
 0xc600 - 0x2bf3fff
 0x47e8000 - 0x
acpipci3 at acpi0 PC03: 0x 0x0011 0x0001
extent `acpipci3 pcibus' (0x0 - 0xff), flags=0
 0x0 - 0xbf
extent `acpipci3 pciio' (0x0 - 0x), flags=0
 0x0 - 0x3af
 0x3e0 - 0x9fff
 0x1 - 0x
extent `acpipci3 pcimem' (0x0 - 0x), flags=0
 0x0 - 0x9
 0xc - 0xc5ff
 0xe100 - 0xff
 0x2bf4000 - 0x

pcidevs and usbdevs in Dell r7515

[Hrvoje Popovski]

Hi all,

in attachment you can find diff with some new AMD devices found in Dell
R7515.

pcidevs are from
https://raw.githubusercontent.com/pciutils/pciids/master/pci.ids

and usbdevs are from
https://usb-ids.gowdy.us/read/UD/1604/10c0
https://certification.ubuntu.com/catalog/component/1604:10c0

names for amd devices from 0x1490 to 0x1497 are little to general :)


dmesg with this diff

r7515# dmesg
OpenBSD 6.6-current (GENERIC.MP) #14: Wed Jan  8 18:20:24 CET 2020
hrv...@r7515.asd:/sys/arch/amd64/compile/GENERIC.MP
real mem = 68279922688 (65116MB)
avail mem = 66198118400 (63131MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.2 @ 0x699ad000 (72 entries)
bios0: vendor Dell Inc. version "1.1.6" date 10/02/2019
bios0: Dell Inc. PowerEdge R7515
acpi0 at bios0: ACPI 6.0
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP BERT HEST EINJ HPET APIC MCFG WSMT SLIC SSDT
SSDT CRAT CDIT IVRS SSDT
acpi0: wakeup devices PC00(S5) XHCI(S3) PC01(S5) XHCI(S3) PC02(S5)
XHCI(S3) PC03(S5) XHCI(S3)
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpihpet0 at acpi0: 14318180 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
ioapic0 at mainbus0: apid 240 pa 0xfec0, version 21, 24 pins, can't
remap
ioapic1 at mainbus0: apid 241 pa 0xe028, version 21, 32 pins, can't
remap
ioapic2 at mainbus0: apid 242 pa 0xc518, version 21, 32 pins, can't
remap
ioapic3 at mainbus0: apid 243 pa 0xaa18, version 21, 32 pins, can't
remap
ioapic4 at mainbus0: apid 244 pa 0xfd18, version 21, 32 pins, can't
remap
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD EPYC 7542 32-Core Processor, 2894.93 MHz, 17-31-00
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu0: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache, 128MB 64b/line disabled L3 cache
cpu0: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu0: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: AMD EPYC 7542 32-Core Processor, 2894.58 MHz, 17-31-00
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu1: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache, 128MB 64b/line disabled L3 cache
cpu1: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu1: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: AMD EPYC 7542 32-Core Processor, 2894.57 MHz, 17-31-00
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu2: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache, 128MB 64b/line disabled L3 cache
cpu2: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu2: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: AMD EPYC 7542 32-Core Processor, 2894.57 MHz, 17-31-00
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu3: 32KB 64b/line 8-way 

Re: Dell PowerEdge R7515

[Hrvoje Popovski]

On 20.12.2019. 16:45, Hrvoje Popovski wrote:
> Hi,
> 
> installation went very well but i can't reboot or halt box, it stops at
> syncing disks... done  and i need to power off ..
> 

cpu on that box is 32/64 cores, when HT is enabled i'm seeing this log

Dec 22 19:53:46 r7515 /bsd: klog: dropped 3272 bytes, message buffer full

first line in dmesg starts from here ..

r7515# dmesg
36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu2: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache, 128MB 64b/line disabled L3 cache
cpu2: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu2: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu2: smt 0, core 2, package 0

Re: ix(4) need support for X553

[Hrvoje Popovski]

On 18.11.2019. 0:31, Hrvoje Popovski wrote:
> On 14.11.2019. 1:06, Stuart Henderson wrote:
>> Apart from the obvious style(9) problems, can anyone give me guidance on
>> how to approach this diff? I'm quite uneasy at the amount of changes but 
>> these
>> nics are quite important, they're all over the place on Atom C3xxx boards
>> which are pretty much the main "next step up" from APUs.
>>
>> I have an A2SDi-2C-HLN4F that I need to put in service soon (not so
>> bothered about the ix on this box, I'm using 4xSFP+ ixl, but giving it
>> a spin) - applying my merged diff (https://junkpile.org/ixgbe.diff2)
> 
> with this diff box panic while booting at starting network
> 
> ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01: msi, address
> ix1 at pci1 dev 0 function 1 "Intel 82599" rev 0x01: msi, address
> em0 at pci7 dev 0 function 0 "Intel I350" rev 0x01: msi, address
> em1 at pci7 dev 0 function 1 "Intel I350" rev 0x01: msi, address
> 
> net.inet.ip.forwarding: 0 -> 1
> starting network
> panic: attempt to execute user address 0x0 in supervisor mode
> Stopped at  db_enter+0x10:  popq%rbp
> TIDPIDUID PRFLAGS PFLAGS  CPU  COMMAND
> *489100  47566  0 0x3  00K ifconfig
> db_enter() at db_enter+0x10
> panic(81cbe37d) at panic+0x128
> pageflttrap(8000224ebeb0,0) at pageflttrap+0x2db
> kerntrap(8000224ebeb0) at kerntrap+0x91
> alltraps_kern_meltdown() at alltraps_kern_meltdown+0x7b
> 0(80078000,80078000,aa0519036f37d371,80078000,8
> 0159300,7) at 0
> ixgbe_init(80078000) at ixgbe_init+0x36
> ixgbe_ioctl(80078048,8020690c,80159300) at ixgbe_ioctl+0x15a
> in_ifinit(80078048,80159300,8000224ec2d0,1) at
> in_ifinit+0xf3
> in_ioctl_change_ifaddr(8040691a,8000224ec2c0,80078048,1) at
> in_ioctl_change_ifaddr+0x350
> in_ioctl(8040691a,8000224ec2c0,80078048,1) at in_ioctl+0x12e
> ifioctl(fd83b36cad80,8040691a,8000224ec2c0,80004ee0) at
> ifioctl+0x99e
> sys_ioctl(80004ee0,8000224ec3d0,8000224ec430) at
> sys_ioctl+0x3c2
> syscall(8000224ec4a0) at syscall+0x389
> end trace frame: 0x8000224ec520, count: 0
> https://www.openbsd.org/ddb.html describes the minimum info required in bug
> reports.  Insufficient info makes it difficult to find and fix bugs.
> ddb{0}>

it seems that 82599 is affected by this diff, x552 and x540 are fine

Re: ix(4) need support for X553

[Hrvoje Popovski]

On 14.11.2019. 1:06, Stuart Henderson wrote:
> Apart from the obvious style(9) problems, can anyone give me guidance on
> how to approach this diff? I'm quite uneasy at the amount of changes but these
> nics are quite important, they're all over the place on Atom C3xxx boards
> which are pretty much the main "next step up" from APUs.
> 
> I have an A2SDi-2C-HLN4F that I need to put in service soon (not so
> bothered about the ix on this box, I'm using 4xSFP+ ixl, but giving it
> a spin) - applying my merged diff (https://junkpile.org/ixgbe.diff2)


with this diff box panic while booting at starting network

ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01: msi, address
ix1 at pci1 dev 0 function 1 "Intel 82599" rev 0x01: msi, address
em0 at pci7 dev 0 function 0 "Intel I350" rev 0x01: msi, address
em1 at pci7 dev 0 function 1 "Intel I350" rev 0x01: msi, address

net.inet.ip.forwarding: 0 -> 1
starting network
panic: attempt to execute user address 0x0 in supervisor mode
Stopped at  db_enter+0x10:  popq%rbp
TIDPIDUID PRFLAGS PFLAGS  CPU  COMMAND
*489100  47566  0 0x3  00K ifconfig
db_enter() at db_enter+0x10
panic(81cbe37d) at panic+0x128
pageflttrap(8000224ebeb0,0) at pageflttrap+0x2db
kerntrap(8000224ebeb0) at kerntrap+0x91
alltraps_kern_meltdown() at alltraps_kern_meltdown+0x7b
0(80078000,80078000,aa0519036f37d371,80078000,8
0159300,7) at 0
ixgbe_init(80078000) at ixgbe_init+0x36
ixgbe_ioctl(80078048,8020690c,80159300) at ixgbe_ioctl+0x15a
in_ifinit(80078048,80159300,8000224ec2d0,1) at
in_ifinit+0xf3
in_ioctl_change_ifaddr(8040691a,8000224ec2c0,80078048,1) at
in_ioctl_change_ifaddr+0x350
in_ioctl(8040691a,8000224ec2c0,80078048,1) at in_ioctl+0x12e
ifioctl(fd83b36cad80,8040691a,8000224ec2c0,80004ee0) at
ifioctl+0x99e
sys_ioctl(80004ee0,8000224ec3d0,8000224ec430) at
sys_ioctl+0x3c2
syscall(8000224ec4a0) at syscall+0x389
end trace frame: 0x8000224ec520, count: 0
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports.  Insufficient info makes it difficult to find and fix bugs.
ddb{0}>

ddb{0}> show registers
rdi   0x81ef0a18kprintf_mutex
rsi  0x5
rbp   0x8000224ebd40
rbx   0x8000224ebdf0
rdx   0xc800121a
rcx0x286
rax  0x1
r80x8000224ebd00
r90x8015937c
r10   0x42179c57eeeb011c
r11   0xe24334526c571546
r12 0x38
r13   0x8000224ebd50
r140x100
r15   0x81cbe37dcy_pio_rec+0x3a81
rip   0x81b1a660db_enter+0x10
cs   0x8
rflags 0x282
rsp   0x8000224ebd40
ss  0x10
db_enter+0x10:  popq%rbp
ddb{0}>


dmesg
OpenBSD 6.6-current (GENERIC.MP) #5: Mon Nov 18 00:13:49 CET 2019
hrv...@r620-1.srce.hr:/sys/arch/amd64/compile/GENERIC.MP
real mem = 17115840512 (16322MB)
avail mem = 16584732672 (15816MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xcf42c000 (99 entries)
bios0: vendor Dell Inc. version "2.8.0" date 06/26/2019
bios0: Dell Inc. PowerEdge R620
acpi0 at bios0: ACPI 3.0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC SPCR HPET DMAR MCFG WDAT SLIC ERST HEST
BERT EINJ T
CPA PC__ SRAT SSDT
acpi0: wakeup devices PCI0(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 4 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.46 MHz, 06-3e-04
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM
EP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 2, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 6 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.00 MHz, 06-3e-04
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,V
MX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SM

Re: TSC synchronization on MP machines

[Hrvoje Popovski]

On 6.8.2019. 22:29, Paul Irofti wrote:
> Hi,
> 
> Here is a fourth diff addressing all the issues so far, that have been
> mainly pointed out by kettenis@, thanks!
> 
> Changes:
>   - stop resetting the observed drift as it does not affect tsc
> re-initialization on resume, thus removing all changes from
> acpi_machdep.c
>   - fix comment and put a temporary pretty printf of resume
>   - rename cpu_cc_skew to ci_tsc_skew
>   - remove unfinished code using MSR_TSC for synchronization (to
> be added later on together with the missing IA32_TSC_ADJUST
> wrmsr commands)
> 
> All other technical issues were discussed and settled in private and
> require no change to the former diff.
> 
> 
> For testing you can also use the regress test after booting with tsc as
> default clock and waiting for an hour or so to let the clocks go wild:
> 
>   # cd /usr/src/regress/sys/kern/clock_gettime
>   # make regress
> 
> There is another test program flying around the mailing lists I guess,
> but I could not locate it now so if someone is kind enough to reply with
> the code, that would be lovely!
> 
> Paul

Hi,

I applied this diff on Dell R6415 with AMD EPYC 7551P with 32/64 cores,
run regress and test program that tb@ pointed out .. and everything seem
right ..


r6415# sysctl kern.timecounter.hardware
kern.timecounter.hardware=tsc


r6415# dmesg | grep tsc_timecounter_init
tsc_timecounter_init: TSC skew=0 observed drift=0
tsc_timecounter_init: TSC skew=-260 observed drift=0
tsc_timecounter_init: TSC skew=-240 observed drift=0
tsc_timecounter_init: TSC skew=-470 observed drift=0
tsc_timecounter_init: TSC skew=120 observed drift=0
tsc_timecounter_init: TSC skew=-480 observed drift=0
tsc_timecounter_init: TSC skew=-520 observed drift=0
tsc_timecounter_init: TSC skew=-440 observed drift=0
tsc_timecounter_init: TSC skew=-10 observed drift=0
tsc_timecounter_init: TSC skew=-460 observed drift=0
tsc_timecounter_init: TSC skew=-490 observed drift=0
tsc_timecounter_init: TSC skew=-440 observed drift=0
tsc_timecounter_init: TSC skew=110 observed drift=0
tsc_timecounter_init: TSC skew=-470 observed drift=0
tsc_timecounter_init: TSC skew=-490 observed drift=0
tsc_timecounter_init: TSC skew=-440 observed drift=0
tsc_timecounter_init: TSC skew=-30 observed drift=0
tsc_timecounter_init: TSC skew=-470 observed drift=0
tsc_timecounter_init: TSC skew=-490 observed drift=0
tsc_timecounter_init: TSC skew=-430 observed drift=0
tsc_timecounter_init: TSC skew=110 observed drift=0
tsc_timecounter_init: TSC skew=-480 observed drift=0
tsc_timecounter_init: TSC skew=-470 observed drift=0
tsc_timecounter_init: TSC skew=-450 observed drift=0
tsc_timecounter_init: TSC skew=20 observed drift=0
tsc_timecounter_init: TSC skew=-470 observed drift=0
tsc_timecounter_init: TSC skew=-470 observed drift=0
tsc_timecounter_init: TSC skew=-450 observed drift=0
tsc_timecounter_init: TSC skew=110 observed drift=0
tsc_timecounter_init: TSC skew=-460 observed drift=0
tsc_timecounter_init: TSC skew=-480 observed drift=0
tsc_timecounter_init: TSC skew=-440 observed drift=0
tsc_timecounter_init: TSC skew=-10 observed drift=0
tsc_timecounter_init: TSC skew=-460 observed drift=0
tsc_timecounter_init: TSC skew=-490 observed drift=0
tsc_timecounter_init: TSC skew=-450 observed drift=0
tsc_timecounter_init: TSC skew=130 observed drift=0
tsc_timecounter_init: TSC skew=-460 observed drift=0
tsc_timecounter_init: TSC skew=-480 observed drift=0
tsc_timecounter_init: TSC skew=-430 observed drift=0
tsc_timecounter_init: TSC skew=70 observed drift=0
tsc_timecounter_init: TSC skew=-480 observed drift=0
tsc_timecounter_init: TSC skew=-510 observed drift=0
tsc_timecounter_init: TSC skew=-430 observed drift=0
tsc_timecounter_init: TSC skew=140 observed drift=0
tsc_timecounter_init: TSC skew=-460 observed drift=0
tsc_timecounter_init: TSC skew=-490 observed drift=0
tsc_timecounter_init: TSC skew=-430 observed drift=0
tsc_timecounter_init: TSC skew=20 observed drift=0
tsc_timecounter_init: TSC skew=-490 observed drift=0
tsc_timecounter_init: TSC skew=-500 observed drift=0
tsc_timecounter_init: TSC skew=-430 observed drift=0
tsc_timecounter_init: TSC skew=-380 observed drift=0
tsc_timecounter_init: TSC skew=-460 observed drift=0
tsc_timecounter_init: TSC skew=-490 observed drift=0
tsc_timecounter_init: TSC skew=-470 observed drift=0
tsc_timecounter_init: TSC skew=0 observed drift=0
tsc_timecounter_init: TSC skew=-460 observed drift=0
tsc_timecounter_init: TSC skew=-480 observed drift=0
tsc_timecounter_init: TSC skew=-440 observed drift=0
tsc_timecounter_init: TSC skew=130 observed drift=0
tsc_timecounter_init: TSC skew=-450 observed drift=0
tsc_timecounter_init: TSC skew=-510 observed drift=0
tsc_timecounter_init: TSC skew=-480 observed drift=0
r6415# dmesg | grep tsc_timecounter_init | wc -l
  64



OpenBSD 6.5-current (GENERIC.MP) #4: Wed Aug  7 13:45:08 CEST 2019
hrv...@r6415.lala:/sys/arch/amd64/compile/GENERIC.MP
real mem = 

Re: move to interface rx queue backpressure for if_rxr livelock detection

[Hrvoje Popovski]

On 1.7.2019. 3:16, David Gwynne wrote:
> interface rx queue processing includes detection of when the stack
> becomes too busy to process packets.
> 
> there's three stages to this mechanism. firstly, everything is fine
> and the packets are simply queued for processing. the second is the
> "pressure_return" stage where the interface has queued a few times,
> but the stack hasn't run to process them. ifiq_input returns 1 in
> this situation to notify the nic that it should start to slow down.
> the last stage is the "pressure_drop" stage where the nic has
> continued to queue packets and the stack still hasnt run. in this
> stage it drops the packets and returns 1.
> 
> independently, the stack looks for lost clock ticks (because the stack
> traditionally blocked softclock ticks) as a livelock detection
> mechanism. this no longer works that well now we're in an MP worls.
> firstly, the stack could be running on a different cpu to the clock and
> therefore wont block it. secondly, the stack runs in a thread and doesnt
> raise the spl, so it shouldnt be blocking clock interrupts even if it is
> sharing a cpu now.
> 
> therefore the traditional livelock detection mechanism doesnt work and
> should be moved away from. the replacement is getting nics that
> implement rx ring moderation to look at the return value of the rx queue
> input function and telling the rings to slow down. that is what this
> diff does.
> 
> i've compiled it on amd64, which covers most of the drivers, but there's
> a few in fdt that i did blind and havent tested. ive tested a couple of
> the interfaces, but more testing would be appreciated.


Hi,

without this diff when box is under pressure ifconfig output can take up
from 10 to 20 min to finish...
   11m26.31s real 0m00.01s user 1m37.16s system


with this diff and
net.link.ifrxq.pressure_return=4
net.link.ifrxq.pressure_drop=8

it takes under minute
0m40.55s real 0m00.00s user 0m00.80s system


every time numbers will be different, but this diff makes my test box
smoother under pressure ...

tcpdump: send_fd: sendmsg(3): Broken pipe

[Hrvoje Popovski]

Hi all,

when closing tcpdump while sending high rate traffic over box i'm
getting log below:

x3550m4# tcpdump -ni ix1
^C
x3550m4# tcpdump: send_fd: sendmsg(3): Broken pipe
tcpdump: send_fd: sendmsg: expected sent 1 got -1

OpenBSD 6.5-current (GENERIC.MP) #6: Tue Jun  4 15:05:10 MDT 2019
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

Re: Pump my sched: fewer SCHED_LOCK() & kill p_priority

[Hrvoje Popovski]

On 2.6.2019. 21:41, Martin Pieuchot wrote:
> On 01/06/19(Sat) 18:55, Martin Pieuchot wrote:
>> Diff below exists mainly for documentation and test purposes.  If
>> you're not interested about how to break the scheduler internals in
>> pieces, don't read further and go straight to testing!

> Updated diff to use IPL_SCHED and rebased to apply on top of -current :) 

i'm running this diff together with proctreelk diff on openbsd desktop
with gnome and samba server and everything seems fine 

Re: stack trace / free(0) in isascan()

[Hrvoje Popovski]

On 9.5.2019. 11:14, Sebastien Marie wrote:
> On Thu, May 09, 2019 at 10:55:44AM +0200, Hrvoje Popovski wrote:
>>
>> with this diff i'm getting new traces
> 
> it is (somehow) expected.
> 
> the commit that starts showing traces do the following:
> - when there is missing size on free() reports it (with a backtrace to know 
> the caller)
> - but report only a fixed number of calls (5), because else users will be mad
> 
> so by correcting some sizes it makes others calls to free() to be visible.
> 
>> free with zero size: (127)
>> Starting stack trace...
>> free(8013f800,7f,0,8013f800,cf43c4f465ef43f8,0) at free+0xd8
>> uhidev_attach(80071200,8014ed00,8000224a40a0,80071200,89eb6e07df884e85,80071200)
>>  at uhidev_attach+0x1b4
> 
>> free with zero size: (127)
>> Starting stack trace...
>> free(8013f800,7f,0,8013f800,cf43c4f465284bc3,0) at free+0xd8
>> hid_report_size(80070c00,41,0,0,764c887b264d079f,0) at 
>> hid_report_size+0x10f
> 
>> free with zero size: (127)
>> Starting stack trace...
>> free(8013f800,7f,0,8013f800,cf43c4f465284185,0) at free+0xd8
>> hid_is_collection(80070c00,41,ff,10006,a6cb281b8426ee7e,81cf60e0)
>>  at hid_is_collection+0xe9
> 
>> free with zero size: (127)
>> Starting stack trace...
>> free(8013f800,7f,0,8013f800,cf43c4f465284185,0) at free+0xd8
>> hid_is_collection(80070c00,41,ff,10001,a6cb281b844568dc,81cf6118)
>>  at hid_is_collection+0xe9
> 
>> free with zero size: (127)
>> Starting stack trace...
>> free(8013f800,7f,0,8013f800,cf43c4f465284185,0) at free+0xd8
>> hid_is_collection(80070c00,41,ff,10002,a6cb281b844568fc,3) at 
>> hid_is_collection+0xe9
> 
> I am leaving others free() calls to people that would like to play this game 
> too.
> 

traces from different pc

free with zero size: (2)
Starting stack trace...
free(800dc000,2,0,800dc000,bccbb3f663e5b90b,0) at free+0xd8
azalia_mixer_delete(800d9410,800d9410,12eae783e6f10d36,800d9410,815389e0,82004a20)
at azalia_mixer_delete+0x30
azalia_codec_delete(800d9410,800d9410,fb5581cc816e9ad6,800b6c00,814e84ed,82004a50)
at azalia_codec_delete+0x1d
azalia_init_codecs(800b6c00,800b6c00,d1d8bb30647efc90,82004b90,800b6c00,0)
at azalia_init_codecs+0x344
azalia_pci_attach(800c6800,800b6c00,82004b90,800c6800,6a09b45b25c7514b,800c6800)
at azalia_pci_attach+0x21f
config_attach(800c6800,81d2c1f8,82004b90,815ad6c0,fb1ef9caf20ebad0,8000d800)
at config_attach+0x1ee
pci_probe_device(800c6800,8000d800,0,0,271a74bc17e998d4,0) at
pci_probe_device+0x4c0
pci_enumerate_bus(800c6800,0,0,800c6800,7473f8bb614e5ef1,80026100)
at pci_enumerate_bus+0xb7
config_attach(80026100,81d2be08,82004db8,8143f230,fb1ef9caf2179750,82004db8)
at config_attach+0x1ee
mainbus_attach(0,80026100,0,0,7473f8bb614e5ef1,0) at
mainbus_attach+0x280
config_attach(0,81d2b938,0,0,fb1ef9caf259a843,2c5dc00) at
config_attach+0x1ee
cpu_configure(fb1ef9caf20e9d69,2c5dc00,0,80027000,810dce43,82004f00)
at cpu_configure+0x33
main(0,0,2c5dc00,0,10ff8c0,1) at main+0x4a9
end trace frame: 0x0, count: 244
End of stack trace.
free with zero size: (2)
Starting stack trace...
free(800b9e00,2,0,800b9e00,bccbb3f663f8b5ee,0) at free+0xd8
azalia_codec_delete(800d9410,800d9410,fb5581cc816e9ad6,800b6c00,814e8505,82004a50)
at azalia_codec_delete+0x35
azalia_init_codecs(800b6c00,800b6c00,d1d8bb30647efc90,82004b90,800b6c00,0)
at azalia_init_codecs+0x344
azalia_pci_attach(800c6800,800b6c00,82004b90,800c6800,6a09b45b25c7514b,800c6800)
at azalia_pci_attach+0x21f
config_attach(800c6800,81d2c1f8,82004b90,815ad6c0,fb1ef9caf20ebad0,8000d800)
at config_attach+0x1ee
pci_probe_device(800c6800,8000d800,0,0,271a74bc17e998d4,0) at
pci_probe_device+0x4c0
pci_enumerate_bus(800c6800,0,0,800c6800,7473f8bb614e5ef1,80026100)
at pci_enumerate_bus+0xb7
config_attach(80026100,81d2be08,82004db8,8143f230,fb1ef9caf2179750,82004db8)
at config_attach+0x1ee
mainbus_attach(0,80026100,0,0,7473f8bb614e5ef1,0) at
mainbus_attach+0x280
config_attach(0,81d2b938,0,0,fb1ef9caf259a843,2c5dc00) at
config_attach+0x1ee
cpu_configure(fb1ef9caf20e9d69,2c5dc00,0,80027000,810dce43,82004f00)
at cpu_con

Re: stack trace / free(0) in isascan()

[Hrvoje Popovski]

On 9.5.2019. 10:35, Sebastien Marie wrote:
> On Thu, May 09, 2019 at 10:12:49AM +0200, Hrvoje Popovski wrote:
>> Hi all,
>>
>> i update kernel from cvs few minutes ago and i'm seeing this stack trace
>> in dmesg. i'm just reporting it.
> 
> The principe of the game is to look at the free() call that still use 0
> as size, and found the right size to fill it.
> 
>> free with zero size: (2)
>> Starting stack trace...
>> free(80074100,2,0,80074100,1d5a05b47900fbf4,81d287e8)
>>  at free+0xd8
>> isascan(80101200,80074100,9563e4d9af15796a,81618690,80101200,81d16d01)
>>  at isascan+0x3f8
> 
> Here, isascan() calls free() on dev (a struct device).
> 
> the related malloc() call is done by config_make_softc() at line 248:
>247  config_attach(parent, dev, , isaprint);
>248  dev = config_make_softc(parent, cf);
> 
> The size used for malloc() is :
> 
> kern/subr_autoconf.c
>417  struct device *
>418  config_make_softc(struct device *parent, struct cfdata *cf)
>419  {
>420  struct device *dev;
>421  struct cfdriver *cd;
>422  struct cfattach *ca;
>423
>424  cd = cf->cf_driver;
>425  ca = cf->cf_attach;
>426  if (ca->ca_devsize < sizeof(struct device))
>427  panic("config_make_softc");
>428
>429  /* get memory for all device vars */
>430  dev = malloc(ca->ca_devsize, M_DEVBUF, M_NOWAIT|M_ZERO);
>431  if (dev == NULL)
>432  panic("config_make_softc: allocation for device softc 
> failed");
> 
> 
> So calling free() with cf->cf_attach->ca_devsize should be fine.
> Diff below.
> 
> OK ?
> 

with this diff i'm getting new traces

dmesg

OpenBSD 6.5-current (GENERIC.MP) #2: Thu May  9 10:48:28 CEST 2019
r...@r620-1.srce.hr:/sys/arch/amd64/compile/GENERIC.MP
real mem = 17115840512 (16322MB)
avail mem = 16587034624 (15818MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xcf42c000 (99 entries)
bios0: vendor Dell Inc. version "2.7.0" date 05/23/2018
bios0: Dell Inc. PowerEdge R620
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC SPCR HPET DMAR MCFG WDAT SLIC ERST HEST
BERT EINJ TCPA PC__ SRAT SSDT
acpi0: wakeup devices PCI0(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 4 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.45 MHz, 06-3e-04
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 2, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 6 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.00 MHz, 06-3e-04
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 3, package 0
cpu2 at mainbus0: apid 8 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.00 MHz, 06-3e-04
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 4, package 0
cpu3 at mainbus0: apid 16 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.00 MHz, 06-3e-04
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,

stack trace

[Hrvoje Popovski]

Hi all,

i update kernel from cvs few minutes ago and i'm seeing this stack trace
in dmesg. i'm just reporting it.


free with zero size: (2)
Starting stack trace...
free(80074100,2,0,80074100,1d5a05b47900fbf4,81d287e8)
at free+0xd8
isascan(80101200,80074100,9563e4d9af15796a,81618690,80101200,81d16d01)
at isascan+0x3f8
config_scan(81618690,80101200,788843d91069b7bd,8006d600,82005c80,81d16d20)
at config_scan+0xc6
config_attach(8006d600,81d285f0,82005c80,81349ac0,5d5d0c36f785b613,80021560)
at config_attach+0x1ee
pcib_callback(8006d600,8006d600,5d5d0c36f7ced75e,0,81d15068,81c3e5f0)
at pcib_callback+0x57
config_process_deferred_children(8001ce00,8001ce00,9a28fdbe853a8244,80027100,82005db8,81d11808)
at config_process_deferred_children+0x8a
config_attach(80027100,81d25f38,82005db8,81715070,5d5d0c36f7c06324,82005db8)
at config_attach+0x1f6
mainbus_attach(0,80027100,0,0,1fd016762ddbd092,0) at
mainbus_attach+0x280
config_attach(0,81d25a68,0,0,5d5d0c36f7e2cbd7,0) at
config_attach+0x1ee
cpu_configure(707e1942dfa51104,0,0,80028000,8153e763,82005f00)
at cpu_configure+0x33
main(0,0,0,0,0,1) at main+0x4a9
end trace frame: 0x0, count: 246
End of stack trace.

Re: sfp module info and diagnostics

[Hrvoje Popovski]

On 8.4.2019. 22:13, Stuart Henderson wrote:
> On 2019/04/08 19:55, Hrvoje Popovski wrote:
>> On 8.4.2019. 11:33, David Gwynne wrote:
>>> this updates the ifconfig part of the diff
>> This is great feature... thank you ..
>> it would be great if dBm could be exported via snmp :)
> You can do this via net-snmp already:
> http://sysadvent.blogspot.com/2008/12/day-4-extending-net-snmps-snmpd.html?m=1
> 
> Better integration would be nice one day, but it adds complication (not
> least it wants some caching mechanism, rather than triggering a slow NIC
> operation every time a query comes in), and what already works is enough
> to save a lot of time, disruption and expense (especially if €quinix
> £emote hand$ are involved!) moving fibres to alternative equipment to
> check light levels. Which is why I deliberately didn't mention the
> S word ;)
> 

ok ok, no S word :)

Re: sfp module info and diagnostics

[Hrvoje Popovski]

On 8.4.2019. 19:55, Hrvoje Popovski wrote:
> On 8.4.2019. 11:33, David Gwynne wrote:
>> this updates the ifconfig part of the diff
> 
> This is great feature... thank you ..

maybe to put laser wavelength in sff output?

ix0 - 1000BASE-LX
x3550m4# ifconfig ix0 sff
ix0: identifier SFP (03)
connector: LC (07)
vendor: FiberStore
product: SFP1G-LX-31
revision: (unknown)
serial: F175EX02389
date: 2017-05-31
temperature: 39.33 C
vcc: 3.39 V
tx-bias: 15.35 mA
tx-power: -6.13 dBm
rx-power: -5.58 dBm average

SFP 33 Temperature  = 31.285C
SFP 33 Voltage  = 3.333V
SFP 33 Tx Bias Current  = 18.400mA
SFP 33 Tx Power = -6.0467dBm
SFP 33 Rx Power = -5.1670dBm



ix0 - 10GBASE-ER
x3550m4# ifconfig ix0 sff
ix0: identifier SFP (03)
connector: LC (07)
vendor: OEM
product: CSS-907A15DE-15
revision: 1.0
serial: 1803070025
date: 2018-03-08
temperature: 37.88 C
vcc: 3.31 V
tx-bias: 72.39 mA
tx-power: 0.90 dBm
rx-power: -0.14 dBm average

SFP+ 33 Temperature  = 24.453C
SFP+ 33 Voltage  = 3.252V
SFP+ 33 Tx Bias Current  = 74.740mA
SFP+ 33 Tx Power = 1.1598dBm
SFP+ 33 Rx Power = -0.4479dBm

it's strange that with ix0 media is 10GSFP+Cu. i think that is should be
10GbaseER ?

ix0: flags=8843 mtu 1500
media: Ethernet autoselect (10GSFP+Cu full-duplex,rxpause,txpause)
status: active
supported media:
media 10GSFP+Cu
media autoselect



ixl1 - 10GBASE-SR
x3550m4# ifconfig ixl1 sff
ixl1: identifier SFP (03)
connector: LC (07)
vendor: OEM
product: 10GB-SFP-SR-H
revision: 10
serial: HXP96S02
date: 2014-03-03
temperature: 30.80 C
vcc: 3.33 V
tx-bias: 5.85 mA
tx-power: -3.31 dBm
rx-power: -4.40 dBm average

SFP+ 34 Temperature  = 35.398C
SFP+ 34 Voltage  = 3.277V
SFP+ 34 Tx Bias Current  = 10.884mA
SFP+ 34 Tx Power = -2.6930dBm
SFP+ 34 Rx Power = -3.3838dBm

Re: sfp module info and diagnostics

[Hrvoje Popovski]

On 8.4.2019. 11:33, David Gwynne wrote:
> this updates the ifconfig part of the diff

This is great feature... thank you ..
it would be great if dBm could be exported via snmp :)


switch - Dell S4810
ix0 - sfp+ 10GBASE-SR optics
ix1 - sfp 1000BASE-SX optics
ixl0 - 10G passive DAC cables


x3550m4# ifconfig ix0 sff
ix0: identifier SFP (03)
connector: LC (07)
vendor: Intel Corp
product: FTLX8571D3BCV-IT
revision: A
serial: MTB07YW
date: 2015-03-26
temperature: 39.21 C
vcc: 3.36 V
tx-bias: 7.97 mA
tx-power: -2.66 dBm
rx-power: -4.22 dBm average


switch side
SFP+ 33 Temperature  = 32.133C
SFP+ 33 Voltage  = 3.283V
SFP+ 33 Tx Bias Current  = 10.728mA
SFP+ 33 Tx Power = -2.6978dBm
SFP+ 33 Rx Power = -2.2643dBm

dBm values on openbsd and switch should be similar, right ?



x3550m4# ifconfig ix1 sff
ix1: identifier SFP (03)
connector: LC (07)
vendor: CISCO-FINISAR
product: FTRJ-8519-7D-CSC
revision: (unknown)
serial: H11H167
date: 2004-01-23




x3550m4# ifconfig ixl0 sff
ixl0: identifier SFP (03)
connector: Copper Pigtail (21)
vendor: Amphenol
product: 616740005
revision: C
serial: CN0358VV6751650
date: 2016-07-07

Re: bypass interface input queues for vlan(4)

[Hrvoje Popovski]

On 23.2.2019. 10:35, David Gwynne wrote:
> On Fri, Feb 22, 2019 at 09:56:58AM -0300, Martin Pieuchot wrote:
>> On 22/02/19(Fri) 15:01, David Gwynne wrote:
>>> On Thu, Feb 21, 2019 at 04:29:27PM -0300, Martin Pieuchot wrote:
>>>> On 21/02/19(Thu) 14:19, David Gwynne wrote:
>>>>> right now we add vlan_input as a possible input handler on the parent
>>>>> interface, and if the packet is for a vlan we take it and pretend we
>>>>> received it on the vlan interface by calling if_input against that mbuf.
>>>>>
>>>>> as mpi notes, the if input queue stuff looks like a lot of work,
>>>>> especially for a single packet. my opinion is that we got away with
>>>>> the if input stuff we've done to try and encourage an mpsafe network
>>>>> stack because we amortised the cost of it over many packets off the
>>>>> hardware ring. vlan does it a packet at a time.
>>>>>
>>>>> this moves the handling of vlan packets back into ether_input by
>>>>> calling vlan_input directly on packets that are either marked as vlan
>>>>> tagged or have a vlan ethertype. note that we have to do that anyway,
>>>>> this just makes it explicit.
>>>>>
>>>>> vlan_input is then tweaked to implement all the important bits of if
>>>>> input. part of what if input does is count the packets. because vlan
>>>>> already has per cpu counters for bypassing queues on output, we can use
>>>>> them again for input from any cpu. if i ever get round to making a
>>>>> driver handle multiple rx rings this means we can rx vlan packets
>>>>> concurrently, they don't get serialised to a single if input q.
>>>>>
>>>>> finally, hrvoje popovski has tested this diff and get's a significant
>>>>> bump with it. on a machine that can forward 1100Kpps without vlan, it
>>>>> goes from 790Kpps with vlan to 870Kpps. On a box that can do 730Kpps
>>>>> without vlans, it goes from 550Kpps with vlan to 840Kpps. We're
>>>>> still trying to figure that last one out, but it does appear to be
>>>>> faster.
>>>>>
>>>>> thoughts? ok?
>>>> Why do we need to move stuff to ether_input() if all we want is to
>>>> bypass ifiq_input()?  Isn't a 3 line diff enough^^ ?
>>> Fair point. It turns out it's not quite three lines, but it's still
>>> smaller.
>> I'm unhappy to see the bpf & packet magic reappear in pseudo-drivers.
>>
>> This is going to spread in every pseudo-driver, no?  So why not keeping
>> it in the new API?   Should we document if_input() vs if_input_one()?
>> Should we assert that if_input_one() is only called from a network
>> thread?  If yes, should we pick a better name?
> Maybe. How's if_vinput? And as you suggest, it can do some more of
> the magic? Let me try converting a few more drivers before we
> burden it with constraints.


Hi all,

this diff really nicely increase forwarding performance over vlan. i
tested it with this loop
vlan300 destroy && sh netstart && sleep 5 && ifconfig vlan400 destroy &&
sh netstart && sleep 5 && ifconfig ix3 down && sh netstart

and box seems stable ..

carp demote counter

[Hrvoje Popovski]

hi all,

while playing around with carp, pfsync, pflow and doing ifconfig pfsync0
destroy && sh netstart in loop i noticed that carp demote counter
becomes really high

r710-2# ifconfig -g carp
carp: carp demote count 4321

don't know is this normal or not but problem is that i can't lower that
counter with ifconfig -g carp -carpdemote command... demote counter by
it self is dropping as it should

carp: pfsync0 demoted group carp by -1 to 4288 (pfsync destroy)
carp: pfsync0 demoted group pfsync by -1 to 32 (pfsync destroy)
carp: pfsync0 demoted group carp by 32 to 4320 (pfsync init)
carp: pfsync0 demoted group pfsync by 32 to 32 (pfsync init)
carp: pfsync0 demoted group carp by 1 to 4321 (pfsync bulk start)
carp: pfsync0 demoted group pfsync by 1 to 33 (pfsync bulk start)


r710-2# ifconfig -g carp -carpdemote 4321
ifconfig: invalid carp demotion: too large

r710-2# ifconfig -g carp -carpdemote 129
ifconfig: invalid carp demotion: too large

r710-2# ifconfig -g carp -carpdemote 128
ifconfig: SIOCSIFGATTR: Invalid argument

r710-2# ifconfig -g carp -carpdemote 12
ifconfig: SIOCSIFGATTR: Invalid argument

r710-2# ifconfig -g carp -carpdemote 1
ifconfig: SIOCSIFGATTR: Invalid argument

r710-2# ifconfig -g carp
carp: carp demote count 4321


OpenBSD 6.4-current (GENERIC.MP) #366: Thu Oct 18 01:06:52 MDT 2018
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 25739890688 (24547MB)
avail mem = 24950530048 (23794MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xcf49c000 (84 entries)
bios0: vendor Dell Inc. version "6.6.0" date 05/22/2018
bios0: Dell Inc. PowerEdge R710
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC SPCR HPET DMAR MCFG WDAT SLIC ERST HEST
BERT EINJ SRAT TCPA
acpi0: wakeup devices PCI0(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 32 (boot processor)
cpu0: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 3059.36 MHz, 06-2c-02
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 1
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 132MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 0 (application processor)
cpu1: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 3058.99 MHz, 06-2c-02
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 0, package 0
cpu2 at mainbus0: apid 34 (application processor)
cpu2: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 3058.99 MHz, 06-2c-02
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 1
cpu3 at mainbus0: apid 2 (application processor)
cpu3: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 3059.00 MHz, 06-2c-02
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 1, package 0
cpu4 at mainbus0: apid 50 (application processor)
cpu4: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 3058.99 MHz, 06-2c-02
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 0, core 9, package 1
cpu5 at mainbus0: apid 18 (application processor)
cpu5: Intel(R) Xeon(R) CPU X5647 @ 2.93GHz, 3059.00 MHz, 06-2c-02
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN

Re: acpi panic on dell r640 and r740xd

[Hrvoje Popovski]

On 17.10.2018. 9:47, Luthing wrote:
> Did you find something for making this work?
> 
> Regards


could you try install snapshot or wait for few day when 6.4 will be stable ?

Re: pfsync: avoid a recursion on PF_LOCK

[Hrvoje Popovski]

On 27.9.2018. 23:37, Alexandr Nedvedicky wrote:
> On Thu, Sep 27, 2018 at 11:30:09PM +0200, Hrvoje Popovski wrote:
>> On 27.9.2018. 18:34, Alexandr Nedvedicky wrote:
>>> Mentioning parallelism: there is yet another change you need to perform
>>> in order to get more pf_test() instances running. Currently there
>>> is only single input task, which processes inbound packets. In order
>>> to allow more input tasks one has to change NET_TASKQ constant found
>>> in net/if.c. Patch below tells kernel to use two input tasks:
>> shouldn't this be - Patch below tells kernel to use four input tasks?
> absolutely correct, it should be four. The patch below makes kernel
> to use 4 input tasks.

i'm running this diff in production for few days and firewall seems stable

Re: ix0/1/2/3 at pci8 dev 0 function 0 "Intel X553 SFP+" rev 0x11: msi, address 00:30:18:xxxxxxx

[Hrvoje Popovski]

On 18.9.2018. 18:18, sven falempin wrote:
> On Tue, Sep 18, 2018 at 4:39 AM Hrvoje Popovski  wrote:
> 
>> On 17.9.2018. 22:32, sven falempin wrote:
>>> Dear Tech reader,
>>>
>>> I am recently working on  Intel(R) Atom(TM) CPU C3758  intel devices.
>>> SFP Intel card are not working in 6.3/current openBSD base
>>>
>>> I did patch intel driver reading netbsd, freebsd and intel code of ixgbe
>>> driver.
>>>
>>> I am now transferring data between two openBSD at ~1.50 Gb/s
>>> for more than 48 hours ( been looping all week end )
>>>
>>> First, i d like to find other user with ix card to check for regression !
>>> Secondly, can i get some feedback on how to test 10Gb /s transfer
>>>  - i usually download ramfs file through nginx or use iperf .
>>> Third, how can i get a patch accepted into base : ie, how do i clean this
>>> work ?
>>
>> Hi,
>>
>> user here. Thank you for doing this. I think that your primary concern
>> at this point should be stability of this driver.
>> While transferring data over ix interfaces you could try shutting down
>> interfaces and bring them up. maybe something like this in loop:
>>
>> ifconfig ix0 down && ifconfig ix0 up && ifconfig ifconfig ix1 down &&
>> ifconfig ix1 up && ifconfig ix0 down && ifconfig ix1 down && sh netstart
>>
>>
> that is working okay, and unplugging too
> 
> 
>>
>> if you have some sx or lx sfp+ modules insert/remove them in ix
>> interfaces and stuff link that.
>>
> 
> I only have one type of sfp+ with me
> 
> 
>>
>> regarding performance testing if you have other box with 10G interfaces
>> you could directly connect these boxes and generate lots of traffic over
>> your driver and doing all that crazy stuff..
>>
>>
> i only have openbsd denverton hardware and it s kinda hard to sustain 10Gb
> of traffic
> 
> 
> I had a request to extract the github patch file directly inside the email,
> i m thinking the 17 K lines would not make sense inside the mail.
> 
> Is there someone with ixgbe hardware ( especially with a fan ? )

i think that x540-t2 does have fan. i will test your diff with x520 and
x540.

Re: ix0/1/2/3 at pci8 dev 0 function 0 "Intel X553 SFP+" rev 0x11: msi, address 00:30:18:xxxxxxx

[Hrvoje Popovski]

On 17.9.2018. 22:32, sven falempin wrote:
> Dear Tech reader,
> 
> I am recently working on  Intel(R) Atom(TM) CPU C3758  intel devices.
> SFP Intel card are not working in 6.3/current openBSD base
> 
> I did patch intel driver reading netbsd, freebsd and intel code of ixgbe
> driver.
> 
> I am now transferring data between two openBSD at ~1.50 Gb/s
> for more than 48 hours ( been looping all week end )
> 
> First, i d like to find other user with ix card to check for regression !
> Secondly, can i get some feedback on how to test 10Gb /s transfer
>  - i usually download ramfs file through nginx or use iperf .
> Third, how can i get a patch accepted into base : ie, how do i clean this
> work ?

Hi,

user here. Thank you for doing this. I think that your primary concern
at this point should be stability of this driver.
While transferring data over ix interfaces you could try shutting down
interfaces and bring them up. maybe something like this in loop:

ifconfig ix0 down && ifconfig ix0 up && ifconfig ifconfig ix1 down &&
ifconfig ix1 up && ifconfig ix0 down && ifconfig ix1 down && sh netstart


if you have some sx or lx sfp+ modules insert/remove them in ix
interfaces and stuff link that.

regarding performance testing if you have other box with 10G interfaces
you could directly connect these boxes and generate lots of traffic over
your driver and doing all that crazy stuff..

witness log

[Hrvoje Popovski]

Hi,

i'm sorry if this witness log is noise. this box is samba and nfs server
and transmission client. i can drop to ddb if needed

lock order reversal:
 1st 0xff020bf5e730 vmmaplk (>lock) @
/usr/src/sys/uvm/uvm_map.c:4433
 2nd 0xff01ed655d68 inode (>i_lock) @
/usr/src/sys/ufs/ufs/ufs_vnops.c:1559
lock order ">i_lock"(rrwlock) -> ">lock"(rwlock) first seen at:
#0  witness_checkorder+0x4b4
#1  _rw_enter+0x68
#2  vm_map_lock_ln+0xbc
#3  uvm_map+0x1a1
#4  km_alloc+0x16a
#5  pool_multi_alloc_ni+0xbb
#6  pool_p_alloc+0x56
#7  pool_do_get+0xe4
#8  pool_get+0xaf
#9  ufsdirhash_build+0x31e
#10 ufs_lookup+0x19d
#11 VOP_LOOKUP+0x4f
#12 vfs_lookup+0x27e
#13 namei+0x226
#14 start_init+0xb2
lock order ">lock"(rwlock) -> ">i_lock"(rrwlock) first seen at:
#0  witness_checkorder+0x4b4
#1  _rw_enter+0x68
#2  _rrw_enter+0x3e
#3  VOP_LOCK+0x3d
#4  vn_lock+0x34
#5  uvn_io+0x1b8
#6  uvm_pager_put+0x109
#7  uvn_flush+0x424
#8  uvm_map_clean+0x3e7
#9  syscall+0x32a
#10 Xsyscall_untramp+0xc0


OpenBSD 6.3-current (GENERIC.MP) #81: Tue Jun  5 07:23:00 MDT 2018
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8456089600 (8064MB)
avail mem = 8122085376 (7745MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xe87b1 (86 entries)
bios0: vendor Hewlett-Packard version "J01 v02.29" date 04/04/2016
bios0: Hewlett-Packard HP Compaq 8200 Elite CMT PC
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC SSDT MCFG HPET SSDT SLIC TCPA
acpi0: wakeup devices PS2K(S3) PS2M(S3) BR20(S4) EUSB(S3) USBE(S3)
PEX0(S4) PEX1(S4) PEX2(S4) PEX3(S4) PEX4(S4) PEX5(S4) PEX6(S4) PEX7(S4)
P0P1(S4) P0P2(S4) P0P3(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3293.17 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,SENSOR,ARAT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
using xsaveopt
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3292.53 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,SENSOR,ARAT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3292.53 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,SENSOR,ARAT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3292.53 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,SENSOR,ARAT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xe000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 5 (BR20)
acpiprt2 at acpi0: bus 1 (PEX0)
acpiprt3 at acpi0: bus -1 (PEX1)
acpiprt4 at acpi0: bus -1 (PEX2)
acpiprt5 at acpi0: bus -1 (PEX3)
acpiprt6 at acpi0: bus 2 (PEX4)
acpiprt7 at acpi0: bus -1 (PEX5)
acpiprt8 at acpi0: bus 3 (PEX6)
acpiprt9 at acpi0: bus 4 (PEX7)
acpiprt10 at acpi0: bus -1 (P0P1)
acpiprt11 at acpi0: bus -1 (P0P2)
acpiprt12 at acpi0: bus -1 (P0P3)
acpiprt13 at acpi0: bus -1 (P0P4)
acpicpu0 at acpi0: C1(1000@1 halt), PSS
acpicpu1 at acpi0: C1(1000@1 halt), PSS
acpicpu2 at acpi0: C1(1000@1 halt), PSS
acpicpu3 at acpi0: C1(1000@1 halt), PSS
acpicmos0 at acpi0
tpm0 at acpi0: TPM_ addr 0xfed4/0x5000: Infineon SLB9635 1.2 rev 0x10
"INT3F0D" at acpi0 not configured
acpibtn0 at acpi0: PWRB
"PNP0C14" at acpi0 not configured
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 3293 MHz: speeds: 3301, 3300, 3100, 2900, 2700,
2500, 

Re: Unlock 16 network-related syscalls

[Hrvoje Popovski]

On 25.5.2018. 10:35, Martin Pieuchot wrote:
> Updated diff that should prevent reported hangs, as analyzed by tb@ and
> visa@.


with this diff i can't reproduce lockup ..

tnx ..

Re: Unlock 16 network-related syscalls

[Hrvoje Popovski]

On 23.5.2018. 15:29, Theo Buehler wrote:
> On Wed, May 23, 2018 at 02:39:20PM +0200, Martin Pieuchot wrote:
>> On 23/05/18(Wed) 11:14, Hrvoje Popovski wrote:
>>> On 22.5.2018. 17:03, Theo Buehler wrote:
>>>> I applied the diff, made syscalls, then built and installed a new
>>>> kernel. With that, I ran into a reliable complete lockup on my x230 by
>>>> starting in an xterm
>>>>
>>>> # make -j 3 build 2>&1 | tee -a /home/theo/buildlog
>>>>
>>>> and then navigating firefox to youtube and trying to start some video.
>>>>
>>>> Unfortunately, I don't know how to provide you with any more useful
>>>> information.
>>>>
>>>> The machine becomes completely unresponsive, the mouse pointer is
>>>> frozen, I'm unable to break into ddb, and the machine is no longer
>>>> pingable.
>>>>
>>> same as tb@. my box is having transmission, samba and nfs and everything
>>> seemed fine until i started make -j4 build then it became unresponsive.
>> I need to find some time to reproduce the hang.  In the meantime if you
>> want to try a WITNESS kernel, that might give us more clues.
>>

hi,

i can repeat lockup but with or without WITNESS i can't get any log and
i can't drop to ddb ... maybe to try with MP_LOCKDEBUG?

Re: Unlock 16 network-related syscalls

[Hrvoje Popovski]

On 22.5.2018. 17:03, Theo Buehler wrote:
> I applied the diff, made syscalls, then built and installed a new
> kernel. With that, I ran into a reliable complete lockup on my x230 by
> starting in an xterm
> 
> # make -j 3 build 2>&1 | tee -a /home/theo/buildlog
> 
> and then navigating firefox to youtube and trying to start some video.
> 
> Unfortunately, I don't know how to provide you with any more useful
> information.
> 
> The machine becomes completely unresponsive, the mouse pointer is
> frozen, I'm unable to break into ddb, and the machine is no longer
> pingable.
> 

same as tb@. my box is having transmission, samba and nfs and everything
seemed fine until i started make -j4 build then it became unresponsive.

Dell PowerEdge R7425

[Hrvoje Popovski]

i'm sending this mail for archive.


CPU: 2 x AMD EPYC 7551

booting stops without any log:
http://kosjenka.srce.hr/~hrvoje/zaprocvat/r7425_01.jpg

if i want to disable acpicpu entering boot config stops right away
http://kosjenka.srce.hr/~hrvoje/zaprocvat/r7425_02.jpg

collected acpidump, lshw, lspci, lsusb, cpuinfo from linux:
http://kosjenka.srce.hr/~hrvoje/zaprocvat/r7425.tar


if there's anything else that i can do, please tell me ...

Dell PowerEdge R6415

[Hrvoje Popovski]

i'm sending this mail for archive.


CPU: 1 x AMD EPYC 7551P

booting stops with this panic:
http://kosjenka.srce.hr/~hrvoje/zaprocvat/r6415_01.jpg

collected acpidump, lshw, lspci, lsusb, cpuinfo from linux:
http://kosjenka.srce.hr/~hrvoje/zaprocvat/R6415.tar


if there's anything else that i can do, please tell me ...

Re: in6_ioctl(): use read lock for SIOCGIF*_IN6

[Hrvoje Popovski]

On 3.5.2018. 1:53, Theo Buehler wrote:
> On Wed, May 02, 2018 at 04:25:20PM +0200, Hrvoje Popovski wrote:
>> On 2.5.2018. 12:16, Theo Buehler wrote:
>>> Here's a further refactoring of in6_ioctl() that splits out the
>>> SIOCGIF*_IN6 cases into the new function in6_ioctl_get(), where we only
>>> need a read lock, similar to ifioctl() and the pending patch for
>>> in_ioctl(). We get rid of one of the four switches in the function body
>>> and error out early on on unknown ioctls before grabbing a lock instead
>>> of doing nothing until the end of the function.
>>>
>>> After grabbing the lock in the body of in6_ioctl(), we now only deal
>>> with SIOCAIFADDR_IN6 and SIOCDIFADDR_IN6. This will need more cleanup
>>> and streamlining in a later step.
>>>
>>> I didn't really know what to do with the big comment. I left it
>>> essentially where it was. Suggestions welcome.
>> Hi,
>>
>> i'm testing this diff on top on -current tree fetched hour ago. i'm
>> forwarding ip6 traffic over vlan on trunk and doing ip6 client server
>> with iperf3 while destroying and recreating pseudo interfaces
>>
>> for now everything seems stable
>>
> Thanks, that's good to know.  However, I overlooked a shadowing issue.
> There was a local re-declaration of error, making the function succeed
> more often than it should. This additional piece is needed:
> 
>   switch(cmd) {
>   case SIOCAIFADDR_IN6:
>   {
>   -   int plen, error = 0, newifaddr = 0;
>   +   int plen, newifaddr = 0;
> 
> This is the only change to the previous submission.


still stable :)

Re: in6_ioctl(): use read lock for SIOCGIF*_IN6

[Hrvoje Popovski]

On 2.5.2018. 12:16, Theo Buehler wrote:
> Here's a further refactoring of in6_ioctl() that splits out the
> SIOCGIF*_IN6 cases into the new function in6_ioctl_get(), where we only
> need a read lock, similar to ifioctl() and the pending patch for
> in_ioctl(). We get rid of one of the four switches in the function body
> and error out early on on unknown ioctls before grabbing a lock instead
> of doing nothing until the end of the function.
> 
> After grabbing the lock in the body of in6_ioctl(), we now only deal
> with SIOCAIFADDR_IN6 and SIOCDIFADDR_IN6. This will need more cleanup
> and streamlining in a later step.
> 
> I didn't really know what to do with the big comment. I left it
> essentially where it was. Suggestions welcome.

Hi,

i'm testing this diff on top on -current tree fetched hour ago. i'm
forwarding ip6 traffic over vlan on trunk and doing ip6 client server
with iperf3 while destroying and recreating pseudo interfaces

for now everything seems stable

Re: interface queue transmit mitigation (again)

[Hrvoje Popovski]

On 28.3.2018. 11:42, Hrvoje Popovski wrote:
> On 28.3.2018. 3:28, David Gwynne wrote:
>> On Thu, Mar 15, 2018 at 03:25:46PM +0100, Martin Pieuchot wrote:
>>> On 14/03/18(Wed) 13:00, David Gwynne wrote:
>>>> this adds transmit mitigation back to the tree.
>>>>
>>>> it is basically the same diff as last time. the big difference this
>>>> time is that all the tunnel drivers all defer ip_output calls, which
>>>> avoids having to play games with NET_LOCK in the ifq transmit paths.
>>> Comments inline.
>>>
>>>> +  if (ifq_len(ifq) >= min(4, ifq->ifq_maxlen)) {
>>> Why 4?  DragonFly recently bumped `ifsq_stage_cntmax' to 16.  Did you
>>> try other values?  They also have an XXX comment that this value should
>>> be per-interface.  Why?
>> their default was 4, and they'd done some research on it. if they
>> moved to 16 there would be a reason for it.
> Hi all,
> 
> with this diff i'm getting 1.43Mpps on
> 12 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.01 MHz
> 
> with plain kernel i'm getting 1.12Mpps and with older diff's i was
> getting 1.31Mpps ...
> 
> if i execute ifconfig ix0 down while generating traffic over everything stop


I've tested this diff with today's tree and i can't repeat problem with
or without -fretpoline diff.

Re: interface queue transmit mitigation (again)

[Hrvoje Popovski]

On 28.3.2018. 11:42, Hrvoje Popovski wrote:
> Hi all,
> 
> with this diff i'm getting 1.43Mpps on
> 12 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.01 MHz
> 
> with plain kernel i'm getting 1.12Mpps and with older diff's i was
> getting 1.31Mpps ...

On box with 6 x Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.46 MHz
with or without this diff i'm getting 1.75 Mpps and i can't trigger
freeze although i left ifconfig down/up over weekend and it's still running.

Stability of forwarding performance seems more stable with this diff
than without it, meaning that there aren't oscillation in forwarding as
before.

Re: interface queue transmit mitigation (again)

[Hrvoje Popovski]

On 28.3.2018. 3:28, David Gwynne wrote:
> On Thu, Mar 15, 2018 at 03:25:46PM +0100, Martin Pieuchot wrote:
>> On 14/03/18(Wed) 13:00, David Gwynne wrote:
>>> this adds transmit mitigation back to the tree.
>>>
>>> it is basically the same diff as last time. the big difference this
>>> time is that all the tunnel drivers all defer ip_output calls, which
>>> avoids having to play games with NET_LOCK in the ifq transmit paths.
>> Comments inline.
>>
>>> +   if (ifq_len(ifq) >= min(4, ifq->ifq_maxlen)) {
>> Why 4?  DragonFly recently bumped `ifsq_stage_cntmax' to 16.  Did you
>> try other values?  They also have an XXX comment that this value should
>> be per-interface.  Why?
> their default was 4, and they'd done some research on it. if they
> moved to 16 there would be a reason for it.

Hi all,

with this diff i'm getting 1.43Mpps on
12 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.01 MHz

with plain kernel i'm getting 1.12Mpps and with older diff's i was
getting 1.31Mpps ...

if i execute ifconfig ix0 down while generating traffic over everything stop

x3550m4# ifconfig ix0 down
^C

from ddb:

ddb{0}> ps
   PID TID   PPIDUID  S   FLAGS  WAIT  COMMAND
   287   54967   4723  0  3 0x3  tqbar ifconfig
  4723  369240  1  0  30x10008b  pause ksh
 78652   12963  1  0  30x100083  ttyin getty
 77445  380566  1  0  30x100083  ttyin getty
  7708   16964  1  0  30x100083  ttyin getty
  6466  480278  1  0  30x100083  ttyin getty
 10683  361200  1  0  30x100083  ttyin ksh
  33222446  1  0  30x100098  poll  cron
  81878185  1 99  30x100090  poll  sndiod
 38828  292448  1110  30x100090  poll  sndiod
 96602  349758  17921 95  30x100092  kqreadsmtpd
 59159  244097  17921103  30x100092  kqreadsmtpd
 72520  357622  17921 95  30x100092  kqreadsmtpd
 11196  442980  17921 95  30x100092  kqreadsmtpd
 19374  184986  17921 95  30x100092  kqreadsmtpd
 99758  239851  17921 95  30x100092  kqreadsmtpd
 17921  468052  1  0  30x100080  kqreadsmtpd
 96705  480305  1  0  30x80  selectsshd
 10784  513637  74642 83  30x100092  poll  ntpd
 74642  349129  19763 83  30x100092  poll  ntpd
 19763  118713  1  0  30x100080  poll  ntpd
 80820  281787  61744 73  30x100090  kqreadsyslogd
 61744  358228  1  0  30x100082  netio syslogd
 92438  103007  37416115  30x100092  kqreadslaacd
 27600  284603  37416115  30x100092  kqreadslaacd
 37416  302849  1  0  30x80  kqreadslaacd
 31025  461354  0  0  3 0x14200  pgzerozerothread
 87259  136299  0  0  3 0x14200  aiodoned  aiodoned
 40842   63462  0  0  3 0x14200  syncerupdate
 434254  0  0  3 0x14200  cleaner   cleaner
 20219  234861  0  0  3 0x14200  reaperreaper
 49370  510675  0  0  3 0x14200  pgdaemon  pagedaemon
 34415  210813  0  0  3 0x14200  bored crynlk
 98085  523911  0  0  3 0x14200  bored crypto
 88352  390863  0  0  3 0x14200  usbtskusbtask
 36743   62252  0  0  3 0x14200  usbatsk   usbatsk
 38389  456819  0  0  3  0x40014200  acpi0 acpi0
 93162   81423  0  0  7  0x40014200idle11
 58166   30480  0  0  7  0x40014200idle10
 55398  115831  0  0  7  0x40014200idle9
96   42736  0  0  7  0x40014200idle8
 63465  183206  0  0  7  0x40014200idle7
 79804  340505  0  0  7  0x40014200idle6
 42987  392463  0  0  7  0x40014200idle5
 94478  284414  0  0  7  0x40014200idle4
 45914   13592  0  0  7  0x40014200idle3
 14508  513956  0  0  7  0x40014200idle2
 16424  111283  0  0  7  0x40014200idle1
 60252  298958  0  0  3 0x14200  bored sensors
 80523  235128  0  0  3 0x14200  tqbar softnet
 40231  252516  0  0  3 0x14200  bored systqmp
 97996  128366  0  0  3 0x14200  bored systq
 78639  112346  0  0  3  0x40014200  bored softclock
*60124   95946  0  0  7  0x40014200idle0
 1  232514  0  0  30x82  wait  init
 0   0 -1  0  3 0x10200  scheduler swapper
ddb{0}>


ddb{0}> tr /p 0t54967
sleep_finish(800023d3c528,81a3863c) at sleep_finish+0x70

acpi panic on dell r640 and r740xd

[Hrvoje Popovski]

Hi all,

i'm sending diagnostics from dell r640 and dell r740xd here just for
archive. i collected acpidump,lspci,lsusb,cpuinfo from linux and dmesg
from bsd.rd.

http://kosjenka.srce.hr/~hrvoje/zaprocvat/dell_r640.tar
http://kosjenka.srce.hr/~hrvoje/zaprocvat/dell_r740xd.tar

this is follow-up on this mail
https://marc.info/?l=openbsd-misc=152147634518421=2

I tried to install snapshot on dell r640 and dell r740xd and
installation went well but booting stops with the same acpi panic on
both machines...
http://kosjenka.srce.hr/~hrvoje/zaprocvat/dell_r640/dell_r640_bsd.jpg
http://kosjenka.srce.hr/~hrvoje/zaprocvat/dell_r740xd/dell_r740xd_bsd.jpg

if there is something else that is needed from these boxes i will be
glad to send.

Re: interface tx mitigation, with NET LOCK fixes

[Hrvoje Popovski]

On 8.1.2018. 2:13, David Gwynne wrote:
> this is tx mitigation again, ie, defer calling an interfaces start
> routine until at least 4 packets are queued, or a task fires.
> 
> the task firing is a problem for things like gif or vxlan that encap
> a packet in ip and send it through the ip stack again. the ip stack
> expects NET_RLOCK to be held. that is implicitly true when sending
> out of the network stack, but not when the bundle task fires.
> 
> this has the bundle tasks take the network read lock on behalf of
> the start routines, like the stack does. this avoids having to patch
> every driver to cope with this.
> 
> tests?

with this diff i'm having same performance boost as with older versions..

from 1.1Mpps to 1.3Mpps on box with
12 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz

Re: mp-safe carp_iamatch6()

[Hrvoje Popovski]

On 22.11.2017. 15:48, Martin Pieuchot wrote:
> Hrvoje Popovski reported the following panic when testing my diff to
> unlock protocol inputs function:
> 
>  panic() at panic+0x128
>  __assert(814d1114,800022755d80,809ce000,800022755e20)
>  carp_ourether(ff0003c0c290,809ce000) at carp_ourether
>  nd6_ns_input(20,0,809c9800) at nd6_ns_input+0x4cf
>  icmp6_input(18,81a95af0,1,3a) at icmp6_input+0x3d4
>  
> ip_deliver(800022756020,80002275602c,80019080,817ee880)
>  ip6intr() at ip6intr+0x7b
> 
> The problem comes from carp_iamatch6() which is not yet MP-safe.  Since
> it is now identical to carp_iamatch(), let's use this one instead.


Hi,

with this diff i can't trigger panic as before.

Thank you ...

Re: multiple interface input queues

[Hrvoje Popovski]

On 14.11.2017. 5:42, David Gwynne wrote:
> this replaces the single mbuf_queue and task in struct ifnet with
> a new ifiqueue structure modelled on ifqueues.


i've tested this diff with ix, myx, em and bge with down/up interfaces
and everything is working fine ...

Re: try to bundle multiple packets on an ifq before sending

[Hrvoje Popovski]

On 10.11.2017. 1:58, David Gwynne wrote:
> this makes ifq_start try to wait for 4 packets before calling
> if->if_qstart.
> 
> this is based on work sephe did in dragonflybsd, and described in
> a comment in their sys/net/if.c. there's a link to it here:
> https://github.com/DragonFlyBSD/DragonFlyBSD/blob/master/sys/net/if.c#L2976-L2987
> 
> the tests we've done generally show a performance bump, but otherwise
> no degradation in performance.
> 
> the mechanism for bundling packets is to but schedule a task to
> service the queue later. if 4 packets get accumulated before the
> task runs, it's cancelled and the code runs the start routine
> directly.
> 
> the most significant difference this implementation has to the dfly
> one is that our ifqs dont (currently) track the number of bytes on
> the q. dfly sends if it can bundle 4 packets, or up to mtu bytes
> worth of packets. this implementation only looks at the number of
> packets.
> 
> the taskq the ifq uses is one of the softnets, which is assigned
> when the ifq is initted and unconditionally used during the ifq's
> lifetime. because ifq work could now be pending in a softnet taskq,
> ifq_barrier also needs to put a barrier in the taskq. this is
> implemented using taskq_barrier, which i wrote ages ago but didn't
> have a use case for at the time.
> 
> tests? ok?

Hi all,

i've tested plain ip4 forwarding performance with and without this diff
and here are results.
12 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.35 MHz

ix (82599) - sending 14Mpps
plain forwarding - 1.18Mpps
patched forwarding - 1.31Mpps

myx (8BL2-2S) - sending 14Mpps
plain forwarding - 900Kpps
patched forwarding - 1.10Mpps

em (i350) - sending 1.4Mpps
plain forwarding - 1Mpps
patched forwarding - 1.17Mpps

bge (5720) - sending 1.4Mpps
plain forwarding - oscillate from 780Kpps to 940Kpps
patched forwarding - oscillate from 850Kpps to 1Mpps


on box with
6 x Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.55 MHz
and with ix (82599) forwarding performance are the same with or without
diff, which is 1.75Mpps


will play with this diff over weekend just to see if there will be some
ill effect

Re: sysctl_int(), sysctl_struct() & MP work

[Hrvoje Popovski]

On 12.9.2017. 15:53, Martin Pieuchot wrote:
> Diff below reduces the scope of the NET_LOCK(), this time in sysctl
> path.  It is interesting for multiple reasons:
> 
> - It reduces the contention on the NET_LOCK(), which should improve
>   the overall latency on the system when counters are frequently
>   queried.  Accesses to read-only operations and per-CPU counters
>   are no longer protected by the NET_LOCK().
> 
> - It allows per-CPU counters to be accessed in parallel.  counters_read(9)
>   is now executed without holding the NET_LOCK().
> 
> - sysctl_mq() is now MP-safe, by serializing access using the mq's
>   mutex.  However a dance is required to not hold a mutex around
>   copyin(9)/copyout(9).
> 
> - The NET_LOCK() is now taken around all sysctl_int(), sysctl_struct()
>   and sysctl_int_arr().  This is not nice but it will allow people to
>   fix parts of the sysctl path independently.
> 
> Note that all data structure currently protected in these sysctl paths
> do not necessarily need the NET_LOCK().  Take CARP's carp_opts[] for
> example.  Does it need the NET_LOCK()?  Is it at all the right primitive?
> Well interested readers can answer with diffs and explanations to these
> questions

Hi all,

i'm running this and "kqueue & rwlock" diff on primary firewall with:
carp, pf, pflow, pfsync, snmpd, dhcpd, dhcp sync, ipsec, remote pflog
and syslog logging

and it seems that everything is working nicely ...

Re: refactoring of pf_find_or_create_ruleset()

[Hrvoje Popovski]

On 1.9.2017. 22:57, Alexandr Nedvedicky wrote:
> as you can see the kernel sets ruleset.anchor to NULL (see pfattach() and then
> do also a 'grep -n kludge pf_ioctl.c'), while userland links it to
> pf_main_anchor.
> 
> I've remember to changing 'parent != NULL' to 'parent != _main_anchor' in
> pf_create_anchor() just to make regression tests passing.  Fortunately you did
> run my code in kernel. With change above my patch works for kernel as well as
> for regression tests.
> 
> updated patch is attached.
> 
> thanks and
> regards
> sasha


Hi,

with this patch i can't trigger panic with or without WITH_PF_LOCK if
that's matter for some reason.

pf conf:

# pfctl -nvf pf.conf
set skip on { lo em0 }
set limit states 100
block drop all
anchor "test1" on ix3 all {
  pass all flags S/SA
  anchor "test11" all {
pass all flags S/SA
  }
}
anchor "test2" on ix2 all {
  pass all flags S/SA
  anchor "test21" all {
pass all flags S/SA
  }
}


thank you sasha for great work on MP pf :)

Re: 1M routes or 1M arp entries

[Hrvoje Popovski]

On 14.8.2017. 16:48, Simon Mages wrote:
> Hi,
> 
> you may want to take a look into /etc/login.conf
> login.conf(5), cap_mkdb(1)
> 
> In this file you can fiddle with you limit maxima
> for login classes.
> 
> BR
> Simon
> 

Thank you, i will do that ...

Re: 1M routes or 1M arp entries

[Hrvoje Popovski]

On 14.8.2017. 16:03, Alexander Bluhm wrote:
> On Mon, Aug 14, 2017 at 03:52:56PM +0200, Hrvoje Popovski wrote:
>> # netstat -rnf inet
>> netstat: Cannot allocate memory
> 
> Have you tried to increase ulimit -d ?

it seems that i can decrease it but not increase it, or i don't know how
to do it properly :)

# ulimit -d
33554432

# ulimit -d 33554433

# ulimit -d
33554432

1M routes or 1M arp entries

[Hrvoje Popovski]

Hi all,

when openbsd imports cca 1M routes or more and if i want to see them
with "netstat -rn" i'm getting "Cannot allocate memory". bgpd can see
all routes. i don't think that this is real problem but full bgp table
is cca 700K routes.


# bgpctl show ip bgp mem
RDE memory statistics
   1245184 IPv4 unicast network entries using 47.5M of memory
   2490368 rib entries using 152M of memory
   2490368 prefix entries using 152M of memory
 1 BGP path attribute entries using 120B of memory
 1 BGP AS-PATH attribute entries using 37B of memory,
   and holding 1 references
 0 BGP attributes entries using 0B of memory
   and holding 0 references
 0 BGP attributes using 0B of memory
RIB using 352M of memory

# bgpctl show ip bgp | wc -l
 1245188

# netstat -rnf inet
netstat: Cannot allocate memory


same happens with arp. if cca 1M arp entries are injected with "arp" and
"netstat -rn" i'm getting "Cannot allocate memory". of course that this
is extremely ridiculous example, but i would be good if i can a least
delete arp entries.


# vmstat -m | egrep "Name|arp"
NameSize Requests FailInUse Pgreq Pgrel Npage Hiwat Minpg
Maxpg Idle
arp   56  14819030   983053 13950   104 13846 13846 0
 80

# arp -an
HostEthernet AddressNetif ExpireFlags
arp: malloc: Cannot allocate memory

# arp -ad
arp: malloc: Cannot allocate memory

# netstat -rnf inet
netstat: Cannot allocate memory

Re: NET_LOCK() w/o argument

[Hrvoje Popovski]

On 11.8.2017. 19:56, Martin Pieuchot wrote:
> Two weeks ago I remove the splsoftnet()/splx() dance inside the
> NET_LOCK().  Turns out we found a single bug, a missing splx()
> in net/if_spppsubr.c.
> 
> I believe it's time to move forward and completely remove the
> argument.  This will allow us to do more funky dances with the
> NET_LOCK().

i'm testing this diff with pf, pfsync, ipsec, carp, vlan, trunk, full
bgp table and that kind of stuff and for now boxes seems stable...

Re: Reducing NET_LOCK() contention

[Hrvoje Popovski]

On 2.8.2017. 11:00, Alexandr Nedvedicky wrote:
> Hello,
> 
> On Wed, Aug 02, 2017 at 10:10:51AM +0200, Martin Pieuchot wrote:
>> On 18/07/17(Tue) 15:55, Martin Pieuchot wrote:
>>> When forwarding a lot of traffic with 10G interfaces contention on the
>>> NET_LOCK() is "visible".  Each time you type "ifconfig" you can go grab
>>> a coffee...
>>>
>>> The problem has a name: pf_purge_thread().  This thread is created by
>>> default and run even if you don't have PF enabled.  Every `hz' it wakes
>>> up, grab the lock and go to sleep. 
>>>
>>> Since the execution of this thread is serialized with the `softnet' task,
>>> it makes more sense to execute it in the same context.  This reduce the
>>> NET_LOCK() contention and implicitly preempt the packet processing.
>>>
>>> Diff below improves the situation with PF disabled, I didn't test with
>>> PF enabled.
>>
>> Updated diff that includes sashan@ suggestions and do not stop the purge
>> task when PF is disabled.  Otherwise some states are not purged until PF
>> is re-enabled.  This can be optimized later.
>>
> 
> thank Hrvoje for spotting the glitch.

i should have spotted it with the first diff, but then i haven't
disabled pf while generating traffic ...

Re: Reducing NET_LOCK() contention

[Hrvoje Popovski]

On 2.8.2017. 10:10, Martin Pieuchot wrote:
> On 18/07/17(Tue) 15:55, Martin Pieuchot wrote:
>> When forwarding a lot of traffic with 10G interfaces contention on the
>> NET_LOCK() is "visible".  Each time you type "ifconfig" you can go grab
>> a coffee...
>>
>> The problem has a name: pf_purge_thread().  This thread is created by
>> default and run even if you don't have PF enabled.  Every `hz' it wakes
>> up, grab the lock and go to sleep. 
>>
>> Since the execution of this thread is serialized with the `softnet' task,
>> it makes more sense to execute it in the same context.  This reduce the
>> NET_LOCK() contention and implicitly preempt the packet processing.
>>
>> Diff below improves the situation with PF disabled, I didn't test with
>> PF enabled.
> Updated diff that includes sashan@ suggestions and do not stop the purge
> task when PF is disabled.  Otherwise some states are not purged until PF
> is re-enabled.  This can be optimized later.

Hi all,

with this diff states are purged as expected and ifconfig, netstat and
arp outputs are little faster then before.

Re: /etc/rc: kernel relinking failed

[Hrvoje Popovski]

On 18.7.2017. 22:59, Theo Buehler wrote:
> On Tue, Jul 18, 2017 at 10:55:59PM +0200, Hrvoje Popovski wrote:
>> Hi all,
>>
>> i have checkout cvs tree few minutes ago and i'm seeing this log.
>>
>> Jul 18 22:47:36 x3550m4 /etc/rc: kernel relinking failed; see
>> /usr/share/compile/GENERIC.MP/relink.log
> Sorry, this is my mistake. I completely forgot to make a current.html
> entry.
> 
> You need to build and install a new config(8) before building and
> installing a new kernel. Will fix this asap.
> 


thank you ...

/etc/rc: kernel relinking failed

[Hrvoje Popovski]

Hi all,

i have checkout cvs tree few minutes ago and i'm seeing this log.

Jul 18 22:47:36 x3550m4 /etc/rc: kernel relinking failed; see
/usr/share/compile/GENERIC.MP/relink.log


here it is:

http://kosjenka.srce.hr/~hrvoje/zaprocvat/relink.log



OpenBSD 6.1-current (GENERIC.MP) #8: Tue Jul 18 22:43:03 CEST 2017
r...@x3550m4.srce.hr:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34314846208 (32725MB)
avail mem = 33269080064 (31727MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7e67b000 (84 entries)
bios0: vendor IBM version "-[D7E158DUS-2.40]-" date 04/10/2017
bios0: IBM IBM System x3550 M4 -[791425Z]-
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP TCPA ERST HEST HPET APIC MCFG OEM0 OEM1 SLIT
SRAT SLIC SSDT SSDT SSDT SSDT DMAR
acpi0: wakeup devices MRP1(S4) DCC0(S4) MRP3(S4) MRP5(S4) EHC2(S5)
PEX0(S5) PEX7(S5) EHC1(S5) IP2P(S3) MRPB(S4) MRPC(S4) MRPD(S4) MRPF(S4)
MRPG(S4) MRPH(S4) MRPI(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.29 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: TSC frequency 2100292710 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 8 (application processor)
cpu4: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 0, core 4, package 0
cpu5 at mainbus0: apid 10 (application processor)
cpu5: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 0, core 5, package 0
cpu6 at mainbus0: apid 32 (application processor)
cpu6: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2100.00 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 0, core 0, package 1
cpu7 at mainbus0: apid 34 (application processor)
cpu7: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 

Re: serial console and ddb

[Hrvoje Popovski]

On 3.7.2017. 23:42, Stuart Henderson wrote:
> The phrase "break sequence" is often used, but it's a bit of a misnomer.
> When a serial port is connected but not actively transmitting data the tx
> line is usually held high. A "break" is when that line is low for more 
> than a frame duration (the length of which depends on the port speed).
> 
> If the tx line is only being held low for a short time during reboot,
> I wonder if you might get lucky by selecting a lower port speed.
> (Avoid going too low, especially if you have enabled extra debug
> messages, or it's likely to interfere with other kernel operations
> while it's printing to the console.)

Yes, on 38400 everything seems fine 

Thank you for nice info ...

serial console and ddb

[Hrvoje Popovski]

Hi all,

i'm having two firewalls fw1 and fw2 and on fw1 i'm sending console
output to com0.

root@fw1:~
# cat /etc/boot.conf
stty com0 115200
set tty com0

root@fw1:~
# cat /etc/ttys | grep tty00
tty00   "/usr/libexec/getty std.115200" vt220   on  secure


on fw2 i'm using "cu -s 115200" to play with wonderful net MP stuff on
fw1 :)

on fw1 in sysctl i'm having ddb.console=1 ... and here's funny part...

when i reboot fw2 and while fw2 is booting at some point it seems that
fw2 send some break sequence to fw1 and fw1 drops to ddb prompt.
when i put ddb.console=0 everything seems fine ...

i don't know is this feature or bug :)


ddb{0}> show panic
the kernel did not panic


ddb{0}> trace
db_enter(3f8,0,8120bffa,10,800020942c78,286) at db_enter+0x9
comintr(804ab000,80484d00,800020942d88,1,819a2de0,f
fff80434e80) at comintr+0x263
intr_handler(800020942d88,80434e80,0,80430a00,0,9)
at intr_
handler+0x6f
Xintr_ioapic_edge4() at Xintr_ioapic_edge4+0xc9
--- interrupt ---
acpicpu_idle(8000e470,819a2de0,819a2df0,0,2b39538b2ae4a
2bd,286) at acpicpu_idle+0x242
cpu_idle_cycle(819a2de0,2a2,0,819a2de0,8153c300,0)
at c
pu_idle_cycle+0x10
end trace frame: 0x0, count: -6
ddb{0}>


ddb{0}> ps
   PID TID   PPIDUID  S   FLAGS  WAIT  COMMAND
  7884   58547  97227  0  30x100083  ttyin ksh
 97227   68404  42285   1000  30x10008b  pause ksh
 42285  462270  70007   1000  30x90  selectsshd
 70007  366696  42175  0  30x82  poll  sshd
 12377  204501   4833 68  30x90  selectisakmpd
  4833  361757  1  0  30x80  netio isakmpd
 35330  141196  81120  0  30x100083  ttyin ksh
 81120  186861  99676   1000  30x10008b  pause ksh
 99676  194692  84798   1000  30x90  selectsshd
 84798  341107  42175  0  30x82  poll  sshd
 74439  375429  89626606  30x90  kqreadladvd
 89626  327310  1  0  30x80  kqreadladvd
 61156  173029  1  0  30x100083  ttyin ksh
 69646  359364  1  0  30x100083  ttyin getty
 24831  485313  1  0  30x100083  ttyin getty
 91314   50076  1  0  30x100083  ttyin getty
  2000  320701  1  0  30x100083  ttyin getty
 31108  172209  1  0  30x100083  ttyin ksh
 20670   84974  1  0  30x100098  poll  cron
 490198325  85011 95  30x100092  kqreadsmtpd
 23296  311537  85011103  30x100092  kqreadsmtpd
 55584  511240  85011 95  30x100092  kqreadsmtpd
 35318  468152  85011 95  30x100092  kqreadsmtpd
 62158   26960  85011 95  30x100092  kqreadsmtpd
 99316  206735  85011 95  30x100092  kqreadsmtpd
 85011  169871  1  0  30x100080  kqreadsmtpd
 42175  473577  1  0  30x80  selectsshd
 63878   52970  63236 83  30x100092  poll  ntpd
 63236  471563  87173 83  30x100092  poll  ntpd
 87173  400722  1  0  30x100080  poll  ntpd
 94958  312911  16156 74  30x100090  bpf   pflogd
 16156  348728  1  0  30x100080  netio pflogd
 59623  441013  22590 73  30x100090  kqreadsyslogd
 22590  131486  1  0  30x100082  netio syslogd
 54640  387075  0  0  3 0x14200  pgzerozerothread
  7129  108777  0  0  3 0x14200  aiodoned  aiodoned
 98767  424884  0  0  3 0x14200  syncerupdate
 78627  423389  0  0  3 0x14200  cleaner   cleaner
 84685  379372  0  0  3 0x14200  reaperreaper
 44237   80666  0  0  3 0x14200  pgdaemon  pagedaemon
 27951   68797  0  0  3 0x14200  bored crynlk
 77707  278117  0  0  3 0x14200  bored crypto
 23962  391218  0  0  3 0x14200  pftm  pfpurge
 90317  414156  0  0  3 0x14200  usbtskusbtask
 172318781  0  0  3 0x14200  usbatsk   usbatsk
 27192  165218  0  0  3  0x40014200  acpi0 acpi0
 79629  515002  0  0  7  0x40014200idle5
 15440   67305  0  0  7  0x40014200idle4
 20528  342900  0  0  7  0x40014200idle3
 70659  319916  0  0  7  0x40014200idle2
 30844  491566  0  0  7  0x40014200idle1
 77845  252780  0  0  3 0x14200  bored sensors
 30764   5  0  0  3 0x14200  bored softnet
 59884  521485  0  0  3 0x14200  bored systqmp
 90717  141157  0  0  3 0x14200  bored systq
 77096   49912  0  0  

Re: pfsync and option WITH_PF_LOCK

[Hrvoje Popovski]

On 9.6.2017. 16:31, Alexandr Nedvedicky wrote:
>> Hi all,
>>
>> with this diff i don't see any traces as before.
>>
> 
> thanks a lot for quick testing.
> 
> regards
> sasha
> 

Hi,

i'm running latest snapshot with WITH_PF_LOCK in production and for now
everything is fine .. will see tomorrow when angry students turn on
their phones :)

carp
pf
pfsync
pflow
dhcpd + sync
tcpdump -lnqttti pflog0 2> error.log | /usr/bin/logger -t pf -p local2.info


many thanks for pf mp development ...

Re: pfsync and option WITH_PF_LOCK

[Hrvoje Popovski]

On 9.6.2017. 14:55, Alexandr Nedvedicky wrote:
> Hello,
> 
> 
> On Fri, Jun 09, 2017 at 01:11:01PM +0200, Alexander Bluhm wrote:
>> On Fri, Jun 09, 2017 at 10:55:46AM +0200, Alexandr Nedvedicky wrote:
>>> would it make sense to commit a poor man's solution below, before pfsync(4)
>>> will get to shape? The idea is to grab PF_LOCK, just before we pass the data
>>> to PF for further processing.
>> Whatever helps to make progress with pf is fine.  We should not
>> delay unlocking pf until someone steps in and refactors pfsync.
>>
>> OK bluhm@
>>
> I still would like to ask Hrvoje to give it a try first. I believe the fix
> should work, but I could not try it as I don't have working pfsync set up
> available for testing.
> 
> thanks and
> regards
> sasha
> 


Hi all,

with this diff i don't see any traces as before.

pfsync and option WITH_PF_LOCK

[Hrvoje Popovski]

Hi all,

i'm getting these traces with "option WITH_PF_LOCK" enaled.
Setup is quite standard, trunk, vlan, carp, pfsync 


splassert: pf_state_insert: want 1 have 0
splassert: pf_remove_state: want 1 have 0


with kern.splassert=2

splassert: pf_state_insert: want 1 have 0
Starting stack trace...
pf_state_insert(80442a00,800020958c58,800020958c50,ff0786c9c898,8087afe8,ff0786c9c898)
at pf_state_insert+0x64
pfsync_state_import(ff00754db23c,2,ff00754db23c,108,1,0) at
pfsync_state_import+0x6bb
pfsync_in_ins(ff00754db23c,108,1,2,130,2c) at pfsync_in_ins+0x151
pfsync_input(800020958df0,800020958dfc,f0,2,0,800020958dfc)
at pfsync_input+0x371
ip_deliver(800020958df0,800020958dfc,f0,2,0,0) at ip_deliver+0x10f
ip_local(ff0074da0900,800020958e50,0,0,4,fdbfa4928cfbc85b) at
ip_local+0x271
ipintr(5,3b7,8162b4de,800020958e50,ff0074da0900,800020958e50)
at ipintr+0x1e
if_netisr(0,800020958eb0,800020958eb0,80019080,8116ebf0,800020958eb0)
at if_netisr+0x1b5
taskq_thread(80019080,2a2,80019080,8137bea0,0,800020958f10)
at taskq_thread+0x79
end trace frame: 0x0, count: 248
End of stack trace.


splassert: pf_remove_state: want 1 have 0
Starting stack trace...
pf_remove_state(ff0786c9c9d0,800020958ca0,c,ff0787323f18,ff0074bbc390,800020958d20)
at pf_remove_state+0x43
pfsync_in_del_c(ff0074bbc3e8,c,1,2,e0,d8) at pfsync_in_del_c+0x68
pfsync_input(800020958df0,800020958dfc,f0,2,0,800020958dfc)
at pfsync_input+0x371
ip_deliver(800020958df0,800020958dfc,f0,2,0,0) at ip_deliver+0x10f
ip_local(ff007575e400,800020958e50,0,0,4,fdbfa4928cfbc85b) at
ip_local+0x271
ipintr(5,3b7,8162b4de,800020958e50,ff007575e400,800020958e50)
at ipintr+0x1e
if_netisr(0,800020958eb0,800020958eb0,80019080,8116ebf0,800020958eb0)
at if_netisr+0x1b5
taskq_thread(80019080,2a2,80019080,8137bea0,0,800020958f10)
at taskq_thread+0x79
end trace frame: 0x0, count: 249
End of stack trace.

Re: routing sockets vs KERNEL_LOCK()

[Hrvoje Popovski]

On 6.6.2017. 13:03, Martin Pieuchot wrote:
> On 05/06/17(Mon) 16:52, Martin Pieuchot wrote:
>> On 05/06/17(Mon) 12:12, Martin Pieuchot wrote:
>>> Routing sockets are not protected by the NET_LOCK().  That's one of the
>>> boundaries of the network stack.  That's whhy claudio@ sent some diffs
>>> to no longer require the KERNEL_LOCK() to protect them.
>>>
>>> But right now some rtm_* functions can be executed without
>>> KERNEL_LOCK().  That's wrong.  Diff below fixes that and move the
>>> KERNEL_LOCK() further down in rtrequest(9) since the NET_LOCK() is
>>> enough to protect the other data structures.
>>>
>>> The scariest bits come from default router advertisements, but I guess
>>> that slaacd(8) saved us ;)
>> Also change a KERNEL_ASSERT_LOCKED() into a NET_ASSERT_LOCK() in
>> rt_{set,put}gwroute().  Theses can now be called without KERNEL_LOCK()
>> when inserting an ARP entry.
> Hrvoje Popovski found the hard way that rtrequest(RTM_RESOLVE...) still
> need the KERNEL_LOCK() for malloc(9)/free(9).

Hi,

with this patch i can't trigger panic as before.

Thank you.

Re: Unlock IP forwarding paths

[Hrvoje Popovski]

On 30.5.2017. 11:48, Martin Pieuchot wrote:
> On 30/05/17(Tue) 10:45, Martin Pieuchot wrote:
>> Diff below moves IPv4 & IPv6 incoming/forwarding path, PIPEX ppp
>> processing and IPv4 & IPv6 dispatch functions outside the KERNEL_LOCK().
>>
>> We currently rely on the NET_LOCK() serializing access to most global
>> data structures for that.  IP input queues are no longer used in the
>> forwarding case.  They still exist as boundary between the network and
>> transport layers because TCP/UDP & friends still need the KERNEL_LOCK().
>>
>> Since we do not want to grab the NET_LOCK() for every packet, the
>> softnet thread will do it once before processing a batch.  That means
>> the L2 processing path, which is currently running without lock, will
>> now run with the NET_LOCK().
>>
>> IPsec is the bridge of this layer.  A bad player.  Since IPsec isn't
>> ready to run without KERNEL_LOCK(), the softnet thread will grab the
>> KERNEL_LOCK() as soon as ``ipsec_in_use'' is set.
>>
>> I tried to document as much as possible the current design in my
>> commit messages and in the comment below.  Please ask if something
>> isn't clear.
> Hrvoje Popovski found that ip{,6}_send_dispatch() also need the IPsec
> dance.
> 
> Updated diff below.


i'm confirming that i can't reproduce panic with this diff ...

Re: Test wanted: IPv4 forwarding w/o KERNEL_LOCK()

[Hrvoje Popovski]

On 28.5.2017. 21:25, Alexander Bluhm wrote:
> On Sun, May 28, 2017 at 07:41:23PM +0200, Hrvoje Popovski wrote:
>> it seems that with
>> $OpenBSD: ip_ipip.c,v 1.81 2017/05/28 13:59:05 bluhm Exp $
>> i can't compile this diff anymore ... with 1.80 everything is fine ...
>> just noticed, nothing else ...
> As I am commiting parts of it, the diff does not apply from time
> to time.  But I am merging it anyway, so I can provide updates.


Thank you

Re: Test wanted: IPv4 forwarding w/o KERNEL_LOCK()

[Hrvoje Popovski]

On 28.5.2017. 15:07, Alexander Bluhm wrote:
> On Fri, May 26, 2017 at 03:07:51PM +0200, Martin Pieuchot wrote:
>> This diff get rids of the ipintrq for forwarding.  The queue is now used
>> for packets delivered locally which still need the KERNEL_LOCK().
> After committing some cleanup mpi@'s diff looks like this now.

Hi all,

it seems that with

$OpenBSD: ip_ipip.c,v 1.81 2017/05/28 13:59:05 bluhm Exp $

i can't compile this diff anymore ... with 1.80 everything is fine ...

just noticed, nothing else ...

Re: Test wanted: IPv4 forwarding w/o KERNEL_LOCK()

[Hrvoje Popovski]

On 28.5.2017. 15:07, Alexander Bluhm wrote:
> After committing some cleanup mpi@'s diff looks like this now.
> 
> bluhm


Hi all,

with cvs tree fetched few minutes ago and with this diff i'm getting
traces in attchment.

setup: carp on vlan on trunk (lacp) on 2 x ix

there are so many net diffs, maybe i forget something...


# ifconfig
lo0: flags=8049 mtu 32768
index 6 priority 0 llprio 3
groups: lo
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
inet 127.0.0.1 netmask 0xff00
ix0: flags=8b43
mtu 1500
lladdr 90:e2:ba:d7:0f:fc
index 1 priority 0 llprio 3
trunk: trunkdev trunk0
media: Ethernet autoselect (10GbaseSR full-duplex,rxpause,txpause)
status: active
ix1: flags=8b43
mtu 1500
lladdr 90:e2:ba:d7:0f:fc
index 2 priority 0 llprio 3
trunk: trunkdev trunk0
media: Ethernet autoselect (10GbaseSR full-duplex,rxpause,txpause)
status: active
em0: flags=8802 mtu 1500
lladdr 0c:c4:7a:da:cd:24
index 3 priority 0 llprio 3
media: Ethernet autoselect (none)
status: no carrier
em1: flags=8843 mtu 1500
lladdr 0c:c4:7a:da:cd:25
index 4 priority 0 llprio 3
media: Ethernet autoselect (1000baseT
full-duplex,master,rxpause,txpause)
status: active
inet 192.168.0.2 netmask 0xff00 broadcast 10.10.0.255
enc0: flags=0<>
index 5 priority 0 llprio 3
groups: enc
status: active
carp700: flags=8843 mtu 1500
lladdr 00:00:5e:00:01:01
index 9 priority 15 llprio 3
carp: INIT carpdev vlan700 vhid 1 advbase 1 advskew 100
groups: carp
status: invalid
inet 10.10.155.236 netmask 0x
pfsync0: flags=41 mtu 1500
index 10 priority 0 llprio 3
pfsync: syncdev: em1 maxupd: 128 defer: off
groups: carp pfsync
pflog0: flags=141 mtu 33136
index 11 priority 0 llprio 3
groups: pflog
vlan700: flags=8943 mtu 1500
lladdr 90:e2:ba:d7:0f:fc
index 13 priority 0 llprio 3
vlan: 700 parent interface: trunk0
vnetid: 700
parent: trunk0
groups: vlan egress
inet 10.10.155.235 netmask 0xfff8 broadcast 10.10.155.239
trunk0: flags=8943 mtu 1500
lladdr 90:e2:ba:d7:0f:fc
index 15 priority 0 llprio 3
trunk: trunkproto lacp
trunk id: [(8000,90:e2:ba:d7:0f:fc,407D,,),
 (8000,00:1f:26:3d:d4:00,0067,,)]
trunkport ix1 active,collecting,distributing
trunkport ix0 active,collecting,distributing
groups: trunk
media: Ethernet autoselect
status: active




OpenBSD 6.1-current (GENERIC.MP) #0: Sun May 28 15:39:58 CEST 2017
hrv...@fw2.bcbn.lo:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34224922624 (32639MB)
avail mem = 33181880320 (31644MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.0 @ 0xec9b0 (59 entries)
bios0: vendor American Megatrends Inc. version "2.0a" date 08/02/2016
bios0: Supermicro X10SRW-F
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SPMI MCFG UEFI HPET WDDT SSDT
NITR SSDT SSDT PRAD DMAR HEST BERT ERST EINJ
acpi0: wakeup devices IP2P(S4) XHCI(S4) EHC1(S4) EHC2(S4) RP01(S4)
RP02(S4) RP03(S4) RP04(S4) RP05(S4) RP06(S4) RP07(S4) RP08(S4) BR1A(S4)
BR1B(S4) BR2A(S4) BR2B(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-1650 v4 @ 3.60GHz, 3600.53 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: TSC frequency 3600526760 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-1650 v4 @ 3.60GHz, 3600.00 MHz
cpu1:

Re: DDB causing lost keystrokes on Dell iDRAC console (not inside ddb)

[Hrvoje Popovski]

On 17.5.2017. 14:59, Alan McKay wrote:
> Looks like this never got a response and we are seeing the same issue.
> 
> OpenBSD 6.0, Dell 330, iDRAC 8, trying to get a console with the
> iDRAC.  Works great with Linux or ESXi on the box, but OpenBSD does
> not work.  There is major keystroke loss which makes it impossible to
> use.
> 
> Is there a way to disable DDB without recompiling?
> 


If you can install latest snapshot. mpi@ make it work




CVSROOT:/cvs
Module name:src
Changes by: m...@cvs.openbsd.org2017/05/12 03:16:55

Modified files:
sys/dev/hid: hidkbd.c
sys/dev/wscons : wskbd.c wskbdvar.h
sys/dev/usb: ukbd.c

Log message:
Introduce a new keyboard console hook to enter ddb(4) and make ukbd(4)
use it.

Instead of defering every input of a USB console keyboard to a timeout
via a queue of one element, only differ entering ddb(4) once a matching
control sequenece has been typed.

This prevent loosing inputs when a USB console keyboard is "too fast".

Fix a problem reported by matthieu@, Adam McDougall and Hrvoje Popovski.

ok stsp@, dlg@

Re: Fix for USB keyboards eating keys, a DDB story

[Hrvoje Popovski]

On 10.5.2017. 15:22, Martin Pieuchot wrote:
> This big hammer of delaying every input via a timeout introduced a nasty
> side effect.  Since only one element can be queued, we can lose inputs
> if the keyboard is too fast.
> 
> Here are some bug reports:
> 
>   https://marc.info/?l=openbsd-bugs=147284987417838=2
>   https://marc.info/?l=openbsd-tech=142082432912041=1


Hi all,

i've applied this patch on boxes below and remote virtual console no
longer lose their characters:   

Dell R620 - iDRAC7
Dell R630 - iDRAC8
Supermicro 1018R-WR - iKVM
IBM x3550M4 - IMM2 - was working and working after patch

i don't have HP ILO console to test on...

mpi many thanks for this patch, it's a lifesaver

Re: Test wanted: IPv4 forwarding w/o KERNEL_LOCK()

[Hrvoje Popovski]

On 23.2.2017. 13:24, Alexander Bluhm wrote:
> On Wed, Feb 22, 2017 at 06:22:56PM +0100, Martin Pieuchot wrote:
>> I'd appreciate if you could test this diff and report regressions.
> 
> I did run the regression tests with it.  Everything worked fine.
> 
>> This cannot be tested if you're using NFS, pflow(4) or BFD.
> 
> NFS test failed, pflow test passed, no test exists for BFD.
> 
> bluhm
> 

i'm running this diff in production on primary firewall with

carp
pf
pfsync
isakmpd -K4
sasyncd
dhcpd
dhcpd sync
tcpdump -lnqttti pflog0

and firewall seems stable and happy ..

Re: Test wanted: IPv4 forwarding w/o KERNEL_LOCK()

[Hrvoje Popovski]

On 23.2.2017. 13:24, Alexander Bluhm wrote:
> On Wed, Feb 22, 2017 at 06:22:56PM +0100, Martin Pieuchot wrote:
>> I'd appreciate if you could test this diff and report regressions.
> 
> I did run the regression tests with it.  Everything worked fine.
> 
>> This cannot be tested if you're using NFS, pflow(4) or BFD.
> 
> NFS test failed, pflow test passed, no test exists for BFD.
> 
> bluhm
> 


with pflow you get this :)

login: panic: kernel diagnostic assertion "_kernel_lock_held()" failed:
file "/usr/src/sys/kern/uipc_socket2.c", line 339
Stopped at  Debugger+0x9:   leave
TIDPIDUID PRFLAGS PFLAGS  CPU  COMMAND
 291012  78916 770x100010  04  dhcpd
 416382  63659 730x100010  03  syslogd
*160454  99141  0 0x14000  0x2002  softnet
Debugger() at Debugger+0x9
panic() at panic+0xfe
__assert() at __assert+0x25
sblock() at sblock+0x8e
sosend() at sosend+0xda
copy_flow_ipfix_4_to_m() at copy_flow_ipfix_4_to_m+0xbd
pflow_pack_flow_ipfix() at pflow_pack_flow_ipfix+0x185
export_pflow_if() at export_pflow_if+0x1de
export_pflow() at export_pflow+0x63
pf_remove_state() at pf_remove_state+0x61
pf_test_state() at pf_test_state+0x312
pf_test() at pf_test+0x285
ip_output() at ip_output+0x706
ip_forward() at ip_forward+0x1d7
end trace frame: 0x80002a157de0, count: 0
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports.  Insufficient info makes it difficult to find and fix bugs.

Re: Help with the NET_LOCK()

[Hrvoje Popovski]

On 25.1.2017. 7:32, Martin Pieuchot wrote:
> I just enabled the NET_LOCK() again and I'm looking for test reports.
> Please go build a kernel from sources or wait for the next snapshot,
> run it and report back.
> 
> If you're looking for some small coding tasks related to the NET_LOCK()
> just do:
> 
>   # sysctl kern.splassert=2
>   # sysctl kern.pool_debug=2


while browsing samba shares or copy files over samba i'm seeing this trace


splassert: yield: want 0 have 1
Starting stack trace...
yield() at yield+0xac
pool_get() at pool_get+0x1ca
softdep_freefile() at softdep_freefile+0x43
ffs_inode_free() at ffs_inode_free+0x27
ufs_inactive() at ufs_inactive+0x158
VOP_INACTIVE() at VOP_INACTIVE+0x35
vrele() at vrele+0x65
unp_detach() at unp_detach+0x59
uipc_usrreq() at uipc_usrreq+0x2cd
soclose() at soclose+0x78
soo_close() at soo_close+0x1c
fdrop() at fdrop+0x2c
closef() at closef+0xcb
sys_close() at sys_close+0x60
syscall() at syscall+0x27b
--- syscall (number 6) ---
end trace frame: 0x0, count: 242
0x22c3209083a:
End of stack trace.


while copying files over local disk there aren't any traces


this is gnome desktop with today's -current


# mount
/dev/sd0a on / type ffs (local)
/dev/sd0l on /home type ffs (local, noatime, nodev, nosuid, softdep)
/dev/sd0d on /tmp type ffs (local, noatime, nodev, nosuid, softdep)
/dev/sd0f on /usr type ffs (local, noatime, nodev, softdep)
/dev/sd0g on /usr/X11R6 type ffs (local, noatime, nodev, softdep)
/dev/sd0h on /usr/local type ffs (local, noatime, nodev, wxallowed, softdep)
/dev/sd0k on /usr/obj type ffs (local, noatime, nodev, nosuid, softdep)
/dev/sd0j on /usr/src type ffs (local, noatime, nodev, nosuid, softdep)
/dev/sd0e on /var type ffs (local, noatime, nodev, nosuid, softdep)


# cat /etc/sysctl.conf
ddb.console=1
kern.bufcachepercent=90
kern.pool_debug=2
kern.splassert=2



OpenBSD 6.0-current (GENERIC.MP) #0: Fri Feb  3 13:57:23 CET 2017
dow...@viziogot.srce.hr:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8456093696 (8064MB)
avail mem = 8195158016 (7815MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xe87b1 (86 entries)
bios0: vendor Hewlett-Packard version "J01 v02.29" date 04/04/2016
bios0: Hewlett-Packard HP Compaq 8200 Elite CMT PC
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC SSDT MCFG HPET SSDT SLIC TCPA
acpi0: wakeup devices PS2K(S3) PS2M(S3) BR20(S4) EUSB(S3) USBE(S3)
PEX0(S4) PEX1(S4) PEX2(S4) PEX3(S4) PEX4(S4) PEX5(S4) PEX6(S4) PEX7(S4)
P0P1(S4) P0P2(S4) P0P3(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3293.26 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: TSC frequency 3293258160 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3292.52 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3292.52 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3292.52 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xe000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 5 (BR20)
acpiprt2 at acpi0: bus 1 (PEX0)
acpiprt3 at acpi0: bus -1 (PEX1)
acpiprt4 at acpi0: bus -1 

Re: Help with the NET_LOCK()

[Hrvoje Popovski]

On 31.1.2017. 21:35, David Hill wrote:
> On Tue, Jan 31, 2017 at 09:11:37PM +0100, Alexander Bluhm wrote:
>> On Tue, Jan 31, 2017 at 12:14:35PM -0500, David Hill wrote:
>>> with mpi@'s suggestion to pass a struct mbuf * 
>> We call mbuf variables m and mbuf pointer mp.  So you should rename
>> *mp to m.
>>
>> The different policy who has to free the mbuf with
>> if (op == PRCO_SETOPT)
>> m_free(*mp);
>> is not nice.  I think it would be better if all the freeing is
>> done in sosetopt and sogetopt.  But this requires more thought
>> and should not be in this diff.  A possible next step.
>>
>> bluhm
>>
> I was thinking sosetopt in a separate diff..
> 
> Updated diff.


In a link below i put whole reboot log from console with source which
includes latest dhill@ commit. There are cca 20K lines in netlock.log


http://kosjenka.srce.hr/~hrvoje/netlock.log

Re: Help with the NET_LOCK()

[Hrvoje Popovski]

On 31.1.2017. 18:14, David Hill wrote:
> On Tue, Jan 31, 2017 at 10:43:26AM +0100, Martin Pieuchot wrote:
>> On 27/01/17(Fri) 14:33, David Hill wrote:
>>> [...] 
>>> Forgot a file...   Try this:
>> Is it now possible to pass a 'struct mbuf *' instead of a 'struct mbuf **'
>> to the pr_ctloutput() functions?
>>
>> Changing the signature would ensure we do not miss a call.  This would
>> also simplify the SETOPT case.
>>
> with mpi@'s suggestion to pass a struct mbuf * 


one trace less :)

tnx ..

Re: Help with the NET_LOCK()

[Hrvoje Popovski]

On 25.1.2017. 7:32, Martin Pieuchot wrote:
> I just enabled the NET_LOCK() again and I'm looking for test reports.
> Please go build a kernel from sources or wait for the next snapshot,
> run it and report back.
> 
> If you're looking for some small coding tasks related to the NET_LOCK()
> just do:
> 
>   # sysctl kern.splassert=2
>   # sysctl kern.pool_debug=2
>   
> Then watch for the traces on your console.



source is fetched half an hour ago and i think that this one is new


splassert: yield: want 0 have 1
Starting stack trace...
yield() at yield+0xac
pool_get() at pool_get+0x1ca
pf_osfp_add() at pf_osfp_add+0x145
pfioctl() at pfioctl+0x13f8
VOP_IOCTL() at VOP_IOCTL+0x44
vn_ioctl() at vn_ioctl+0x77
sys_ioctl() at sys_ioctl+0x1b1
syscall() at syscall+0x27b
--- syscall (number 54) ---
end of kernel
end trace frame: 0x2, count: 249
0xd71fb72f43a:
End of stack trace.

Re: Help with the NET_LOCK()

[Hrvoje Popovski]

On 27.1.2017. 20:33, David Hill wrote:
> On Fri, Jan 27, 2017 at 08:09:36PM +0100, Hrvoje Popovski wrote:
>> On 27.1.2017. 19:14, David Hill wrote:
>>>> splassert: yield: want 0 have 1
>>>> Starting stack trace...
>>>> yield() at yield+0xac
>>>> pool_get() at pool_get+0x1ca
>>>> m_get() at m_get+0x28
>>>> ip_ctloutput() at ip_ctloutput+0x4bf
>>>> sogetopt() at sogetopt+0x7e
>>>> sys_getsockopt() at sys_getsockopt+0xbf
>>>> syscall() at syscall+0x27b
>>>> --- syscall (number 118) ---
>>>> end of kernel
>>>> end trace frame: 0x3, count: 250
>>>> 0x978bdd844a:
>>>> End of stack trace.
>>>>  
>>>>
>>> Attempted to solve this and am running with this diff:
>>
>> Hi,
>>
>> i applied you patch and i'm still seeing this trace
>>
>>
>> splassert: yield: want 0 have 1
>> Starting stack trace...
>> yield() at yield+0xac
>> pool_get() at pool_get+0x1ca
>> m_get() at m_get+0x28
>> ip_ctloutput() at ip_ctloutput+0x4bf
>> sogetopt() at sogetopt+0xa1
>> sys_getsockopt() at sys_getsockopt+0xbf
>> syscall() at syscall+0x27b
>> --- syscall (number 118) ---
>> end of kernel
>> end trace frame: 0x3, count: 250
>> 0x178f12db8f1a:
>> End of stack trace.
>>
>>
>> and this one i'm seeing for first time, maybe because of this diff
>>
>> splassert: yield: want 0 have 1
>> Starting stack trace...
>> yield() at yield+0xac
>> malloc() at malloc+0x406
>> ip_setmoptions() at ip_setmoptions+0x248
>> ip_ctloutput() at ip_ctloutput+0x461
>> sosetopt() at sosetopt+0x8e
>> sys_setsockopt() at sys_setsockopt+0x12d
>> syscall() at syscall+0x27b
>> --- syscall (number 105) ---
>> end of kernel
>> end trace frame: 0x1f83, count: 250
>> 0x91243a37f1a:
>> End of stack trace.
>>
> Forgot a file...   Try this:


With this patch i can't see syscall 118

tnx ...

Re: Help with the NET_LOCK()

[Hrvoje Popovski]

On 27.1.2017. 19:14, David Hill wrote:
>> splassert: yield: want 0 have 1
>> Starting stack trace...
>> yield() at yield+0xac
>> pool_get() at pool_get+0x1ca
>> m_get() at m_get+0x28
>> ip_ctloutput() at ip_ctloutput+0x4bf
>> sogetopt() at sogetopt+0x7e
>> sys_getsockopt() at sys_getsockopt+0xbf
>> syscall() at syscall+0x27b
>> --- syscall (number 118) ---
>> end of kernel
>> end trace frame: 0x3, count: 250
>> 0x978bdd844a:
>> End of stack trace.
>>  
>>
> Attempted to solve this and am running with this diff:


Hi,

i applied you patch and i'm still seeing this trace


splassert: yield: want 0 have 1
Starting stack trace...
yield() at yield+0xac
pool_get() at pool_get+0x1ca
m_get() at m_get+0x28
ip_ctloutput() at ip_ctloutput+0x4bf
sogetopt() at sogetopt+0xa1
sys_getsockopt() at sys_getsockopt+0xbf
syscall() at syscall+0x27b
--- syscall (number 118) ---
end of kernel
end trace frame: 0x3, count: 250
0x178f12db8f1a:
End of stack trace.


and this one i'm seeing for first time, maybe because of this diff

splassert: yield: want 0 have 1
Starting stack trace...
yield() at yield+0xac
malloc() at malloc+0x406
ip_setmoptions() at ip_setmoptions+0x248
ip_ctloutput() at ip_ctloutput+0x461
sosetopt() at sosetopt+0x8e
sys_setsockopt() at sys_setsockopt+0x12d
syscall() at syscall+0x27b
--- syscall (number 105) ---
end of kernel
end trace frame: 0x1f83, count: 250
0x91243a37f1a:
End of stack trace.

Re: Help with the NET_LOCK()

[Hrvoje Popovski]

On 25.1.2017. 7:32, Martin Pieuchot wrote:
> I just enabled the NET_LOCK() again and I'm looking for test reports.
> Please go build a kernel from sources or wait for the next snapshot,
> run it and report back.
> 
> If you're looking for some small coding tasks related to the NET_LOCK()
> just do:
> 
>   # sysctl kern.splassert=2
>   # sysctl kern.pool_debug=2
>   
> Then watch for the traces on your console.

Hi,

i'm sending traces from firewall updated few minutes:

on that firewall i have:

carp
pf
pfsync
isakmpd -K4
sasyncd
dhcpd
dhcpd sync
tcpdump -lnqttti pflog0
pflow ipfix


carp2: state transition: MASTER -> BACKUP
splassert: yield: want 0 have 1
Starting stack trace...
yield() at yield+0xac
pool_get() at pool_get+0x1ca
srp_v_gc_start() at srp_v_gc_start+0x55
rtable_mpath_reprio() at rtable_mpath_reprio+0x14c
rt_if_linkstate_change() at rt_if_linkstate_change+0x10d
rtable_walk_helper() at rtable_walk_helper+0x5e
art_walk_apply() at art_walk_apply+0x40
art_table_walk() at art_table_walk+0x117
art_table_walk() at art_table_walk+0x145
art_table_walk() at art_table_walk+0x145
art_table_walk() at art_table_walk+0x145
art_table_walk() at art_table_walk+0x145
art_table_walk() at art_table_walk+0x145
art_table_walk() at art_table_walk+0x145
art_walk() at art_walk+0xe4
rtable_walk() at rtable_walk+0x62
rt_if_track() at rt_if_track+0x6f
if_linkstate() at if_linkstate+0x67
if_linkstate_task() at if_linkstate_task+0x3d
taskq_thread() at taskq_thread+0x6c
end trace frame: 0x0, count: 237
End of stack trace.


splassert: yield: want 0 have 1
Starting stack trace...
yield() at yield+0xac
pool_get() at pool_get+0x1ca
m_get() at m_get+0x28
ip_ctloutput() at ip_ctloutput+0x4bf
sogetopt() at sogetopt+0x7e
sys_getsockopt() at sys_getsockopt+0xbf
syscall() at syscall+0x27b
--- syscall (number 118) ---
end of kernel
end trace frame: 0x3, count: 250
0x1b14ee7fed7a:
End of stack trace.


splassert: yield: want 0 have 1
Starting stack trace...
yield() at yield+0xac
malloc() at malloc+0x406
ifmedia_ioctl() at ifmedia_ioctl+0x178
bnx_ioctl() at bnx_ioctl+0x144
ifioctl() at ifioctl+0x3e2
soo_ioctl() at soo_ioctl+0x22d
sys_ioctl() at sys_ioctl+0x1b1
syscall() at syscall+0x27b
--- syscall (number 54) ---
end of kernel
end trace frame: 0x4eea5d6f100, count: 249
0x4ee3fdabd7a:
End of stack trace.



splassert: yield: want 0 have 1
Starting stack trace...
yield() at yield+0xac
malloc() at malloc+0x406
if_attach_common() at if_attach_common+0x15e
if_attach() at if_attach+0x2f
carp_clone_create() at carp_clone_create+0x14d
if_clone_create() at if_clone_create+0xab
ifioctl() at ifioctl+0x33c
soo_ioctl() at soo_ioctl+0x22d
sys_ioctl() at sys_ioctl+0x1b1
syscall() at syscall+0x27b
--- syscall (number 54) ---
end of kernel
end trace frame: 0x7f7fb5af, count: 247
0x9f0a5b1b05a:
End of stack trace.


splassert: yield: want 0 have 1
Starting stack trace...
yield() at yield+0xac
malloc() at malloc+0x406
counters_read() at counters_read+0x61
ip_sysctl_ipstat() at ip_sysctl_ipstat+0x43
net_sysctl() at net_sysctl+0xf2
sys_sysctl() at sys_sysctl+0x213
syscall() at syscall+0x27b
--- syscall (number 202) ---
end of kernel
end trace frame: 0x7f7d9930, count: 250
0x334a33927fa:
End of stack trace.


splassert: yield: want 0 have 1
Starting stack trace...
yield() at yield+0xac
malloc() at malloc+0x406
esp_init() at esp_init+0x200
pfkeyv2_send() at pfkeyv2_send+0x170a
pfkey_output() at pfkey_output+0x87
raw_usrreq() at raw_usrreq+0x232
sosend() at sosend+0x2ec
dofilewritev() at dofilewritev+0x205
sys_write() at sys_write+0x89
syscall() at syscall+0x27b
--- syscall (number 4) ---
end of kernel
end trace frame: 0x1b0, count: 247
0x11850f016b1a:


splassert: yield: want 0 have 1
Starting stack trace...
yield() at yield+0xac
malloc() at malloc+0x406
import_identity() at import_identity+0x30
import_identities() at import_identities+0xd7
pfkeyv2_send() at pfkeyv2_send+0x1074
pfkey_output() at pfkey_output+0x87
raw_usrreq() at raw_usrreq+0x232
sosend() at sosend+0x2ec
dofilewritev() at dofilewritev+0x205
sys_write() at sys_write+0x89
syscall() at syscall+0x27b
--- syscall (number 4) ---
end of kernel
end trace frame: 0xe8, count: 246
0x11850f016b1a:
End of stack trace.

Re: tcpdump(63969): syscall 54 "tty"

[Hrvoje Popovski]

On 24.1.2017. 19:03, Sebastien Marie wrote:
> On Tue, Jan 24, 2017 at 03:32:25PM +0100, Hrvoje Popovski wrote:
>> Hi all,
>>
>> every time when quitting tcpdump with ^C i see that log on console.
>> Source is fetched few minutes ago ...
>>
>> Don't know is this good or bad so i'm sending it here ..
>>
>> tcpdump(63969): syscall 54 "tty"
>> tcpdump(87912): syscall 54 "tty"
>> tcpdump(35062): syscall 54 "tty"
>> tcpdump(68817): syscall 54 "tty"
>>
> 
> it is error related to pledge(2).
> 
> could you run tcpdump with ktrace -di, and interrupt it quickly with ^C ?
> 
> # ktrace -di tcpdump ...
> 
> A gdb backtrace is also welcome :)
> 
> 
> The purpose is to check:
>   - what are the pledge promises: 
> 
>  96021 tcpdump  CALL  pledge(0x34a0d6f4,0)
>  96021 tcpdump  STRU  pledge request="stdio rpath inet unix dns recvfd bpf"
>  96021 tcpdump  RET   pledge 0
> ...
>  29420 tcpdump  CALL  pledge(0x34a0d169,0)
>  29420 tcpdump  STRU  pledge request="stdio"
>  29420 tcpdump  RET   pledge 0
> 
>   - what are the argument passed to ioctl(2) which case pledge failure.
> 
>939 tcpdump  CALL  ioctl(4,TIOCSPGRP,0xcf7e2824)
>939 tcpdump  PLDG  ioctl, "tty", errno 1 Operation not permitted
>939 tcpdump  PSIG  SIGABRT SIG_DFL
>939 tcpdump  NAMI  "tcpdump.core"
> 
> (here is a error I faked as I don't reproduce your problem).
> 
> Thanks.
> 


Hi,

sorry for noise, I haven't updated userland. After updating userland I
can't see this log.

tcpdump(63969): syscall 54 "tty"

[Hrvoje Popovski]

Hi all,

every time when quitting tcpdump with ^C i see that log on console.
Source is fetched few minutes ago ...

Don't know is this good or bad so i'm sending it here ..


OpenBSD 6.0-current (GENERIC.MP) #15: Tue Jan 24 15:09:53 CET 2017
hrv...@fw02bcbn.srce.hr:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 12854988800 (12259MB)
avail mem = 12460756992 (11883MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xcf49c000 (84 entries)
bios0: vendor Dell Inc. version "6.4.0" date 07/23/2013
bios0: Dell Inc. PowerEdge R610
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC SPCR HPET DM__ MCFG WD__ SLIC ERST HEST
BERT EINJ SRAT TCPA
acpi0: wakeup devices PCI0(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 32 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2527.38 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: TSC frequency 2527376740 Hz
cpu0: smt 0, core 0, package 1
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 133MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 0 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2527.01 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 0, package 0
cpu2 at mainbus0: apid 34 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2527.01 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 1
cpu3 at mainbus0: apid 2 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2527.01 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 1, package 0
cpu4 at mainbus0: apid 50 (application processor)
cpu4: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2527.01 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 0, core 9, package 1
cpu5 at mainbus0: apid 18 (application processor)
cpu5: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2527.01 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 0, core 9, package 0
cpu6 at mainbus0: apid 52 (application processor)
cpu6: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2527.01 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 0, core 10, package 1
cpu7 at mainbus0: apid 20 (application processor)
cpu7: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2527.01 MHz
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu7: 256KB 64b/line 8-way L2 cache
cpu7: smt 0, core 10, package 0
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 20, 24 pins
ioapic1 at mainbus0: apid 1 pa 0xfec8, version 20, 24 pins
acpihpet0 at acpi0: 14318179 Hz
acpimcfg0 at acpi0 addr 0xe000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PEX1)
acpiprt2 at acpi0: bus 2 (PEX3)
acpiprt3 at acpi0: bus -1 (PEX4)
acpiprt4 at acpi0: bus -1 (PEX5)
acpiprt5 at acpi0: bus -1 (PEX6)

Re: NET_LOCK() for bpf(4)

[Hrvoje Popovski]

On 24.1.2017. 10:59, Martin Pieuchot wrote:
> ok?
> 
> Index: net/bpf.c
> ===
> RCS file: /cvs/src/sys/net/bpf.c,v
> retrieving revision 1.158
> diff -u -p -r1.158 bpf.c
> --- net/bpf.c 9 Jan 2017 19:15:01 -   1.158
> +++ net/bpf.c 21 Jan 2017 00:55:26 -
> @@ -624,9 +624,9 @@ bpfwrite(dev_t dev, struct uio *uio, int
>   if (d->bd_hdrcmplt && dst.ss_family == AF_UNSPEC)
>   dst.ss_family = pseudo_AF_HDRCMPLT;
>  
> - s = splsoftnet();
> + NET_LOCK(s);
>   error = ifp->if_output(ifp, m, (struct sockaddr *), NULL);
> - splx(s);
> + NET_UNLOCK(s);
>  
>  out:
>   bpf_put(d);
> 

Hi,

i'm running this patch on firewall in production with source fetched few
minutes ago and everything works fine ...

on that firewall i have:

carp
pf
pfsync
isakmpd -K4
sasyncd
dhcpd
dhcpd sync
tcpdump -lnqttti pflog0
pflow ipfix

Re: NET_LOCK() take 2, tests wanted!

[Hrvoje Popovski]

On 20.1.2017. 3:04, Martin Pieuchot wrote:
> Diff below enables the NET_LOCK(), again.
> 
> What's new?
> 
>  - The lock ordering problem with fdplock() should now be fixed.  It is
>also documented, fdplock() should be taken first if both locks are 
>needed.
> 
>  - Page faults involving NFS, or any other code path already holding the
>NET_LOCK(), have been fixed.  The recursion is similar to the existing
>vnode problem, so I simply added a hack there.
> 
>  - I added some NET_ASSERT_UNLOCKED() to help finding other possible
>deadlocks.


I sent this panic to mpi@ and sending here just for reference

I have box with:

carp
pf
pfsync
isakmpd -K4
sasyncd
dhcpd
dhcpd sync
tcpdump -lnqttti pflog0
pflow ipfix

OpenBSD/amd64 (trlababalan) (tty00)

login: panic: rw_enter: netlock locking against myself
Stopped at  Debugger+0x9:   leave
   TIDPIDUID PRFLAGS PFLAGS  CPU  COMMAND
*269805  88868  0 0x14000  0x2002  softnet
Debugger() at Debugger+0x9
panic() at panic+0xfe
rw_enter() at rw_enter+0x228
sosend() at sosend+0x114
copy_flow_ipfix_4_to_m() at copy_flow_ipfix_4_to_m+0xbd
pflow_pack_flow_ipfix() at pflow_pack_flow_ipfix+0x185
export_pflow_if() at export_pflow_if+0x1de
export_pflow() at export_pflow+0x63
pf_remove_state() at pf_remove_state+0x55
pfsync_in_del_c() at pfsync_in_del_c+0x49
pfsync_input() at pfsync_input+0x23d
ipv4_input() at ipv4_input+0x51d
ipintr() at ipintr+0x1e
if_netisr() at if_netisr+0x175
end trace frame: 0x80002a157f10, count: 0
https://www.openbsd.org/ddb.html describes the minimum info required in
bug reports.  Insufficient info makes it difficult to find and fix bugs.

ddb{2}> trace
Debugger() at Debugger+0x9
panic() at panic+0xfe
rw_enter() at rw_enter+0x228
sosend() at sosend+0x114
copy_flow_ipfix_4_to_m() at copy_flow_ipfix_4_to_m+0xbd
pflow_pack_flow_ipfix() at pflow_pack_flow_ipfix+0x185
export_pflow_if() at export_pflow_if+0x1de
export_pflow() at export_pflow+0x63
pf_remove_state() at pf_remove_state+0x55
pfsync_in_del_c() at pfsync_in_del_c+0x49
pfsync_input() at pfsync_input+0x23d
ipv4_input() at ipv4_input+0x51d
ipintr() at ipintr+0x1e
if_netisr() at if_netisr+0x175
taskq_thread() at taskq_thread+0x6c
end trace frame: 0x0, count: -15

ddb{2}> show panic
rw_enter: netlock locking against myself

ddb{2}> ps
   PID   PPID   PGRPUID  S   FLAGS  WAIT  COMMAND
   253  71076253  0  30x100083  ttyin ksh
 71076  35135  71076   1000  30x10008b  pause ksh
 35135  12657  12657   1000  30x90  selectsshd
 12657  66645  12657  0  30x82  poll  sshd
 87891  89813  12290  0  30x100080  netio tcpdump
  3270  1   3270  0  30x100083  ttyin getty
 66576  1  66576  0  30x100083  ttyin getty
 30959  1  30959  0  30x100083  ttyin getty
 62307  1  62307  0  30x100083  ttyin getty
  8961  1   8961  0  30x100083  ttyin getty
 93977  1  93977  0  30x100083  ttyin getty
 68538  12290  12290  0  30x100082  piperdlogger
 89813  12290  12290 76  30x100092  bpf   tcpdump
 12290  90126  12290  0  30x10008a  pause sh
 90126  80805  80805  0  30x100090  piperdcron
 80805  1  80805  0  30x100098  poll  cron
 45711  48097  16691623  30x90  nanosleep zabbix_agentd
 58316  48097  16691623  30x90  selectzabbix_agentd
 65495  48097  16691623  30x90  nanosleep zabbix_agentd
 48097  1  16691623  30x90  wait  zabbix_agentd
 84343  48458  48458606  30x90  kqreadladvd
 48458  1  48458  0  30x80  kqreadladvd
 33980  26552  26552 95  30x100092  kqreadsmtpd
 43918  26552  26552103  30x100092  kqreadsmtpd
 18812  26552  26552 95  30x100092  kqreadsmtpd
 30037  26552  26552 95  30x100092  kqreadsmtpd
 25308  26552  26552 95  30x100092  kqreadsmtpd
 30148  26552  26552 95  30x100092  kqreadsmtpd
 26552  1  26552  0  30x100080  kqreadsmtpd
 22016  1  22016 77  30x100090  poll  dhcpd
 96497  1  96497  0  30x80  kqreadsnmpd
 52396  1  52396 91  30x92  kqreadsnmpd
 62956  1  62956 91  30x92  kqreadsnmpd
 66645  1  66645  0  30x80  selectsshd
 45044  35532  34313 83  30x100092  poll  ntpd
 35532  34313  34313 83  30x100092  poll  ntpd
 34313  1  34313  0  30x100080  poll  ntpd
 24235   7637   7637 74  30x100090  bpf   pflogd
  7637  1   7637  0  30x80  netio pflogd
 72492  82652  82652 73  30x100090  kqreadsyslogd
 82652  

Re: NET_LOCK() pr_sysctl

[Hrvoje Popovski]

On 16.1.2017. 23:53, Alexander Bluhm wrote:
> Hrvoje Popovski has tested the diff and found some ugly
> pmap_unwire: wiring for pmap 0xff00075f5210 va 0x7f7d5000 didn't 
> change!
> kernel printfs.  The happens when sysctl(8) writes a value.
> 
> If oldp and newp are in the same page, I have called uvm_vsunlock()
> twice on the same address.  I have added a simple check that does
> not cover complicated overlappings but catches the common case.
> 
> Also I think PROT_READ for the newp should be enough.
> Does anybody know, why the oldp is mapped PROT_READ | PROT_WRITE?
> Is PROT_WRITE not sufficient?
> 
> bluhm

Hi,

with this new diff i don't see any logs and box seems stable ...

Re: pfkey vs splsoftnet()

[Hrvoje Popovski]

On 12.1.2017. 18:27, Hrvoje Popovski wrote:
> On 12.1.2017. 16:20, Martin Pieuchot wrote:
>> On 10/01/17(Tue) 10:37, Martin Pieuchot wrote:
>>> In pfkey_sendup() we call sorwakeup() which asserts for NET_LOCK(), so
>>> we are already at IPL_SOFTNET.
>>>
>>> pfkeyv2_send() is called via pfkey_output() which is also called with
>>> the NET_LOCK() held.
>>>
>>> Finally replace a comment above pfkeyv2_ipo_walk() by the corresponding
>>> assert.
>> Anybody tested or reviewed this diff?
>>
> 
> 
> I applied this diff in production box with
> 
> carp
> pf
> pfsync
> isakmpd -K4
> sasyncd
> dhcpd
> dhcpd sync
> tcpdump -lnqttti pflog0

and pflow ipfix

and still solid as rock :)

Re: pfkey vs splsoftnet()

[Hrvoje Popovski]

On 12.1.2017. 16:20, Martin Pieuchot wrote:
> On 10/01/17(Tue) 10:37, Martin Pieuchot wrote:
>> In pfkey_sendup() we call sorwakeup() which asserts for NET_LOCK(), so
>> we are already at IPL_SOFTNET.
>>
>> pfkeyv2_send() is called via pfkey_output() which is also called with
>> the NET_LOCK() held.
>>
>> Finally replace a comment above pfkeyv2_ipo_walk() by the corresponding
>> assert.
> Anybody tested or reviewed this diff?
> 


I applied this diff in production box with

carp
pf
pfsync
isakmpd -K4
sasyncd
dhcpd
dhcpd sync
tcpdump -lnqttti pflog0

and everything seems fine ...

Re: BFD: route get and route monitor

[Hrvoje Popovski]

On 23.12.2016. 16:57, Hrvoje Popovski wrote:
> On 21.12.2016. 23:15, Sebastian Benoit wrote:
>>> Hi,
>>>
>>> it seems that bfd is working with Force10 S4810 and Extreme Networks
>>> x460 switches. I can test it with cisco c6k5 if you want?
>>
>> Hei,
>>
>> i'm sure phessler (who might not read this for a couple of days) is happy
>> about any test you can do.
>>
>> And thanks for doing these tests!
>>
>> /Benno
> 
> Hi,
> 
> no bfd for me on Cisco c6k5. Will upgrade and report back.
> 
> Tnx for bfd, really great feature ...
> 
> 

Hi all,

at last i upgrade c6k5 and bfd is working

openbsd - 192.168.112.1

# route -n get 192.168.112.2 -bfd
   route to: 192.168.112.2
destination: 192.168.112.2
   mask: 255.255.255.255
  interface: ix1
 if address: 192.168.112.1
   priority: 3 ()
  flags: <UP,HOST,DONE,LLINFO,CLONED,CACHED,BFD>
BFD: async state up remote up laststate down error 0
 diag none remote none
 discr 2428102669 remote 1
 uptime 34s last state time 26m30s
 mintx 75 minrx 75 minecho 0 multiplier 3
 use   mtuexpire
  55 0   626
sockaddrs: <DST,GATEWAY,NETMASK,IFP,IFA>



c6k5 - 192.168.112.2

c6k5#sh bfd neighbors details

IPv4 Sessions
NeighAddr  LD/RD RH/RS State Int
192.168.112.1   1/2428102669 UpUp
Gi6/1
Session state is UP and not using echo function.
Session Host: Software
OurAddr: 192.168.112.2
Handle: 1
Local Diag: 0, Demand mode: 0, Poll bit: 1
MinTxInt: 75, MinRxInt: 75, Multiplier: 3
Received MinRxInt: 75, Received Multiplier: 3
Holddown (hits): 2436(0), Hello (hits): 750(185)
Rx Count: 205, Rx Interval (ms) min/max/avg: 520/840/595 last: 564 ms ago
Tx Count: 186, Tx Interval (ms) min/max/avg: 568/752/661 last: 388 ms ago
Elapsed time watermarks: 0 0 (last: 0)
Registered protocols: IPv4 Static CEF
Uptime: 00:02:02
Last packet: Version: 1  - Diagnostic: 0
 State bit: Up   - Demand bit: 0
 Poll bit: 0 - Final bit: 0
 C bit: 0
 Multiplier: 3   - Length: 24
 My Discr.: 2428102669   - Your Discr.: 1
 Min tx interval: 75 - Min rx interval: 75
 Min Echo interval: 0

Re: BFD: route get and route monitor

[Hrvoje Popovski]

On 21.12.2016. 23:15, Sebastian Benoit wrote:
>> Hi,
>>
>> it seems that bfd is working with Force10 S4810 and Extreme Networks
>> x460 switches. I can test it with cisco c6k5 if you want?
> 
> Hei,
> 
> i'm sure phessler (who might not read this for a couple of days) is happy
> about any test you can do.
> 
> And thanks for doing these tests!
> 
> /Benno

Hi,

no bfd for me on Cisco c6k5. Will upgrade and report back.

Tnx for bfd, really great feature ...

Re: BFD: route get and route monitor

[Hrvoje Popovski]

On 17.12.2016. 14:05, Peter Hessler wrote:
> Updated output, requested by Theo.  A normal get will show just the bfd
> state, use "-bfd" to get all of the information.
> 
> OK?
> 
> $ route -n get 203.0.113.9
>route to: 203.0.113.9
> destination: 203.0.113.9
>mask: 255.255.255.255
>   interface: em1
>  if address: 203.0.113.1
>priority: 4 (connected)
>   flags: 
> BFD: async state up remote up
>  use   mtuexpire
>83924 0   133 
> sockaddrs: 
> 
> $ route -n get 203.0.113.9 -bfd
>route to: 203.0.113.9
> destination: 203.0.113.9
>mask: 255.255.255.255
>   interface: em1
>  if address: 203.0.113.1
>priority: 4 (connected)
>   flags: 
> BFD: async state up remote up laststate down error 0
>  diag none remote neighbor-down
>  discr 186919089 remote 55
>  uptime 05d 2h07m29s
>  mintx 100 minrx 100 minecho 0 multiplier 3
>  use   mtuexpire
>83923 0   229 
> sockaddrs: 


Hi,

it seems that bfd is working with Force10 S4810 and Extreme Networks
x460 switches. I can test it with cisco c6k5 if you want?
Thank you.

bsdtest box is connected in this way:

192.168.11.1 - bsdtest ix0
192.168.11.2 - Dell Force10 S4810 9.11(0.0)

192.168.112.1 - bsdtest ix1
192.168.112.2 - Extreme Networks X460-48t 15.5.3.4


BFD between OpenBSD - Force10 S4810

root@bsdtest:~
# route -n get 192.168.11.2 -bfd
   route to: 192.168.11.2
destination: 192.168.11.2
   mask: 255.255.255.255
  interface: ix0
 if address: 192.168.11.1
   priority: 3 ()
  flags: 
BFD: async state up remote up laststate down error 0
 diag none remote none
 discr 2137079205 remote 4
 uptime 07m26s last state time 01s
 mintx 100 minrx 100 minecho 0 multiplier 3
 use   mtuexpire
 113 0   628
sockaddrs: 

root@bsdtest:~
# route -n monitor

sockaddrs: 
 192.168.11.2 192.168.11.1
got message of size 128 on Wed Dec 21 21:55:15 2016
RTM_BFD: bidirectional forwarding detection: len 128
BFD: async state up remote up


S4810#sh bfd neighbors detail

Session Discriminator: 4
Neighbor Discriminator: 0
Local Addr: 192.168.11.2
Local MAC Addr: 00:01:e8:8a:ea:53
Remote Addr: 192.168.11.1
Remote MAC Addr: 00:25:90:5d:ca:38
Int: TenGigabitEthernet 0/3
State: Up
Configured parameters:
 TX:  200ms, RX:  200ms, Multiplier: 3
Neighbor parameters:
 TX:0ms, RX:0ms, Multiplier: 0
Actual parameters:
 TX: 1000ms, RX: 1000ms, Multiplier: 3
Role: Active
Delete session on Down: False
Client Registered: RTM
Uptime: 00:06:49
Statistics:
 Number of packets received from neighbor: 514
 Number of packets sent to neighbor: 465
 Number of state changes: 2
 Number of messages from IFA about port state change: 0
 Number of messages communicated b/w Manager and Agent: 4



BFD between OpenBSD and Extreme

root@bsdtest:~
# route -n get 192.168.112.2 -bfd
   route to: 192.168.112.2
destination: 192.168.112.2
   mask: 255.255.255.255
  interface: ix1
 if address: 192.168.112.1
   priority: 3 ()
  flags: 
BFD: async state up remote up laststate down error 0
 diag none remote none
 discr 3953390566 remote 2
 uptime 10m17s last state time 01s
 mintx 100 minrx 100 minecho 0 multiplier 3
 use   mtuexpire
  87 0   511
sockaddrs: 

root@bsdtest:~
# route -n monitor

sockaddrs: 
 192.168.112.2 192.168.112.1
got message of size 128 on Wed Dec 21 21:56:30 2016
RTM_BFD: bidirectional forwarding detection: len 128
BFD: async state up remote up



ExtTestFW.30 # sh bfd session detail
Neighbor   : 192.168.112.1   Local   : 192.168.112.2
VR-Name: VR-Default  Interface   : obfd
Session Type   : Single Hop  State   : Up
Detect Time: 3000 ms Age : 550 ms
Discriminator (local/remote): 2 / 3953390566
Demand Mode (local/remote)  : Off / Off
Poll (local/remote) : Off / Off
Tx Interval (local/remote)  : 1000 / 1000 ms
Rx Interval (local/remote)  : 1000 / 1000 ms
oper Tx Interval: 1000 ms
oper Rx Interval: 1000 ms
Multiplier (local/remote)   : 3 / 3
Local Diag  : 0 (No Diagnostic)
Remote Diag : 0 (No Diagnostic)
Authentication  : None
Clients : STATIC
Uptime  : 00 days 00 hours 10 minutes 29 seconds
Up Count: 2
Last Valid Packet Rx: 20:52:44.173792
Last Packet Tx  : 

E5v4 pciids

[Hrvoje Popovski]

Hi all,

patch in attachment adds some E5v4 pciids that i'm seeing in supermicro
1018R-WR box with E5-1650 v4.




before patch:
vendor "Intel", unknown product 0x6f20 (class system subclass
miscellaneous, rev 0x01) at pci0 dev 4 function 0 not configured
vendor "Intel", unknown product 0x6f21 (class system subclass
miscellaneous, rev 0x01) at pci0 dev 4 function 1 not configured
vendor "Intel", unknown product 0x6f22 (class system subclass
miscellaneous, rev 0x01) at pci0 dev 4 function 2 not configured
vendor "Intel", unknown product 0x6f23 (class system subclass
miscellaneous, rev 0x01) at pci0 dev 4 function 3 not configured
vendor "Intel", unknown product 0x6f24 (class system subclass
miscellaneous, rev 0x01) at pci0 dev 4 function 4 not configured
vendor "Intel", unknown product 0x6f25 (class system subclass
miscellaneous, rev 0x01) at pci0 dev 4 function 5 not configured
vendor "Intel", unknown product 0x6f26 (class system subclass
miscellaneous, rev 0x01) at pci0 dev 4 function 6 not configured
vendor "Intel", unknown product 0x6f27 (class system subclass
miscellaneous, rev 0x01) at pci0 dev 4 function 7 not configured
vendor "Intel", unknown product 0x6fe4 (class system subclass
miscellaneous, rev 0x01) at pci7 dev 12 function 4 not configured
vendor "Intel", unknown product 0x6fe5 (class system subclass
miscellaneous, rev 0x01) at pci7 dev 12 function 5 not configured
vendor "Intel", unknown product 0x6ff9 (class system subclass
miscellaneous, rev 0x01) at pci7 dev 15 function 1 not configured
vendor "Intel", unknown product 0x6f34 (class DASP subclass Time and
Frequency, rev 0x01) at pci7 dev 16 function 1 not configured
vendor "Intel", unknown product 0x6f68 (class system subclass
miscellaneous, rev 0x01) at pci7 dev 22 function 0 not configured
vendor "Intel", unknown product 0x6f6e (class system subclass
miscellaneous, rev 0x01) at pci7 dev 22 function 6 not configured
vendor "Intel", unknown product 0x6f6f (class system subclass
miscellaneous, rev 0x01) at pci7 dev 22 function 7 not configured
vendor "Intel", unknown product 0x6fd0 (class system subclass
miscellaneous, rev 0x01) at pci7 dev 23 function 0 not configured
vendor "Intel", unknown product 0x6fb8 (class system subclass
miscellaneous, rev 0x01) at pci7 dev 23 function 4 not configured
vendor "Intel", unknown product 0x6fb9 (class system subclass
miscellaneous, rev 0x01) at pci7 dev 23 function 5 not configured
vendor "Intel", unknown product 0x6fba (class system subclass
miscellaneous, rev 0x01) at pci7 dev 23 function 6 not configured
vendor "Intel", unknown product 0x6fbb (class system subclass
miscellaneous, rev 0x01) at pci7 dev 23 function 7 not configured


after patch:
"Intel E5 v4 DMA" rev 0x01 at pci0 dev 4 function 0 not configured
"Intel E5 v4 DMA" rev 0x01 at pci0 dev 4 function 1 not configured
"Intel E5 v4 DMA" rev 0x01 at pci0 dev 4 function 2 not configured
"Intel E5 v4 DMA" rev 0x01 at pci0 dev 4 function 3 not configured
"Intel E5 v4 DMA" rev 0x01 at pci0 dev 4 function 4 not configured
"Intel E5 v4 DMA" rev 0x01 at pci0 dev 4 function 5 not configured
"Intel E5 v4 DMA" rev 0x01 at pci0 dev 4 function 6 not configured
"Intel E5 v4 DMA" rev 0x01 at pci0 dev 4 function 7 not configured
"Intel E5 v4 Cache" rev 0x01 at pci7 dev 12 function 4 not configured
"Intel E5 v4 Cache" rev 0x01 at pci7 dev 12 function 5 not configured
"Intel E5 v4 Cache" rev 0x01 at pci7 dev 15 function 1 not configured
"Intel E5 v4 R2PCIe Agent" rev 0x01 at pci7 dev 16 function 1 not configured
"Intel E5 v4 RAS" rev 0x01 at pci7 dev 22 function 0 not configured
"Intel E5 V4 DDRIO" rev 0x01 at pci7 dev 22 function 6 not configured
"Intel E5 V4 DDRIO" rev 0x01 at pci7 dev 22 function 7 not configured
"Intel E5 v4 Thermal" rev 0x01 at pci7 dev 23 function 0 not configured
"Intel E5 v4 DDRIO" rev 0x01 at pci7 dev 23 function 4 not configured
"Intel E5 v4 DDRIO" rev 0x01 at pci7 dev 23 function 5 not configured
"Intel E5 v4 DDRIO" rev 0x01 at pci7 dev 23 function 6 not configured
"Intel E5 v4 DDRIO" rev 0x01 at pci7 dev 23 function 7 not configured



full dmesg:

OpenBSD 6.0-current (GENERIC.MP) #1: Thu Dec 15 01:16:46 CET 2016
r...@x10srw-f.my.domain:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34224922624 (32639MB)
avail mem = 33183100928 (31645MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.0 @ 0xec9b0 (59 entries)
bios0: vendor American Megatrends Inc. version "2.0a" date 08/02/2016
bios0: Supermicro X10SRW-F
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SPMI MCFG UEFI HPET WDDT SSDT
NITR SSDT SSDT PRAD DMAR HEST BERT ERST EINJ
acpi0: wakeup devices IP2P(S4) XHCI(S4) EHC1(S4) EHC2(S4) RP01(S4)
RP02(S4) RP03(S4) RP04(S4) RP05(S4) RP06(S4) RP07(S4) RP08(S4) BR1A(S4)
BR1B(S4) BR2A(S4) BR2B(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: 

Re: Xeon-D R2PCIe Agent pcidevs

[Hrvoje Popovski]

On 8.12.2016. 14:32, Hrvoje Popovski wrote:
> Hi all,
> 
> i have this supermicro box:
> https://www.supermicro.nl/products/motherboard/Xeon/D/X10SDV-TP8F.cfm
> 
> dmesg without this patch shows :
> vendor "Intel", unknown product 0x6f34 (class DASP subclass Time and
> Frequency, rev 0x03) at pci12 dev 16 function 1 not configured
> 
> 
> with this patch:
> "Intel Xeon-D R2PCIe Agent" rev 0x03 at pci12 dev 16 function 1 not
> configured

Hi all,

slightly better diff than the previous one.

? XeonD_R2PCIe.diff
Index: pcidevs
===
RCS file: /cvs/src/sys/dev/pci/pcidevs,v
retrieving revision 1.1810
diff -u -p -r1.1810 pcidevs
--- pcidevs	6 Dec 2016 16:55:51 -	1.1810
+++ pcidevs	8 Dec 2016 13:15:43 -
@@ -4539,6 +4539,7 @@ product INTEL XEOND_RAS		0x6f2a	Xeon-D R
 product INTEL XEOND_IOAPIC	0x6f2b	Xeon-D I/O APIC
 product INTEL XEOND_IOAPIC_2	0x6f2c	Xeon-D I/O APIC
 product INTEL XEOND_HA_0	0x6f30	Xeon-D Home Agent
+product INTEL XEOND_R2PCIE	0x6f34	Xeon-D R2PCIe Agent
 product INTEL XEOND_QPI_R3_0	0x6f36	Xeon-D QPI Link
 product INTEL XEOND_QPI_R3_1	0x6f37	Xeon-D QPI Link
 product INTEL XEOND_QD_1	0x6f50	Xeon-D QuickData
Index: pcidevs.h
===
RCS file: /cvs/src/sys/dev/pci/pcidevs.h,v
retrieving revision 1.1804
diff -u -p -r1.1804 pcidevs.h
--- pcidevs.h	6 Dec 2016 16:56:08 -	1.1804
+++ pcidevs.h	8 Dec 2016 13:15:44 -
@@ -4544,6 +4544,7 @@
 #define	PCI_PRODUCT_INTEL_XEOND_IOAPIC	0x6f2b		/* Xeon-D I/O APIC */
 #define	PCI_PRODUCT_INTEL_XEOND_IOAPIC_2	0x6f2c		/* Xeon-D I/O APIC */
 #define	PCI_PRODUCT_INTEL_XEOND_HA_0	0x6f30		/* Xeon-D Home Agent */
+#define	PCI_PRODUCT_INTEL_XEOND_R2PCIE	0x6f34		/* Xeon-D R2PCIe Agent */
 #define	PCI_PRODUCT_INTEL_XEOND_QPI_R3_0	0x6f36		/* Xeon-D QPI Link */
 #define	PCI_PRODUCT_INTEL_XEOND_QPI_R3_1	0x6f37		/* Xeon-D QPI Link */
 #define	PCI_PRODUCT_INTEL_XEOND_QD_1	0x6f50		/* Xeon-D QuickData */
Index: pcidevs_data.h
===
RCS file: /cvs/src/sys/dev/pci/pcidevs_data.h,v
retrieving revision 1.1799
diff -u -p -r1.1799 pcidevs_data.h
--- pcidevs_data.h	6 Dec 2016 16:56:08 -	1.1799
+++ pcidevs_data.h	8 Dec 2016 13:15:44 -
@@ -15692,6 +15692,10 @@ static const struct pci_known_product pc
 	"Xeon-D Home Agent",
 	},
 	{
+	PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_XEOND_R2PCIE,
+	"Xeon-D R2PCIe Agent",
+	},
+	{
 	PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_XEOND_QPI_R3_0,
 	"Xeon-D QPI Link",
 	},

Xeon-D R2PCIe Agent pcidevs

[Hrvoje Popovski]

Hi all,

i have this supermicro box:
https://www.supermicro.nl/products/motherboard/Xeon/D/X10SDV-TP8F.cfm

dmesg without this patch shows :
vendor "Intel", unknown product 0x6f34 (class DASP subclass Time and
Frequency, rev 0x03) at pci12 dev 16 function 1 not configured


with this patch:
"Intel Xeon-D R2PCIe Agent" rev 0x03 at pci12 dev 16 function 1 not
configured



full dmesg with patch:

OpenBSD 6.0-current (GENERIC.MP) #10: Thu Dec  8 14:16:29 CET 2016
r...@bsdtest.srce.hr:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17055727616 (16265MB)
avail mem = 16534257664 (15768MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xed9b0 (47 entries)
bios0: vendor American Megatrends Inc. version "1.0a" date 05/05/2016
bios0: Supermicro Super Server
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SPMI MCFG UEFI DBG2 HPET WDDT
SSDT SSDT SSDT PRAD DMAR HEST BERT ERST EINJ
acpi0: wakeup devices IP2P(S4) XHCI(S4) EHC1(S4) EHC2(S4) RP07(S4)
RP08(S4) BR1A(S4) BR1B(S4) BR2A(S4) BR2B(S4) BR2C(S4) BR2D(S4) BR3A(S4)
BR3B(S4) BR3C(S4) BR3D(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.33 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.00 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec0, version 20, 24 pins
ioapic1 at mainbus0: apid 9 pa 0xfec01000, version 20, 24 pins
acpimcfg0 at acpi0 addr 0x8000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 255 (UNC0)
acpiprt1 at acpi0: bus 0 (PCI0)
acpiprt2 at acpi0: bus 1 (BR1A)
acpiprt3 at acpi0: bus 2 (BR1B)
acpiprt4 at acpi0: bus 4 (BR2C)
acpiprt5 at acpi0: bus 5 (BR3A)
acpiprt6 at acpi0: bus 7 (RP01)
acpiprt7 at acpi0: bus 8 (RP02)
acpiprt8 at acpi0: bus 9 (RP04)
acpiprt9 at acpi0: bus 10 (BR3B)
acpiprt10 at acpi0: bus 11 (RP05)
acpicpu0 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
acpicpu1 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
acpicpu2 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
acpicpu3 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
"ACPI0004" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"PNP0C33" at acpi0 not configured
"PNP0C14" at acpi0 not configured
"PNP0003" at acpi0 not configured
"PNP0C14" at acpi0 not configured
"PNP0501" at acpi0 not configured
"PNP0501" at acpi0 not configured
"IPI0001" at acpi0 not configured
acpibtn0 at acpi0: PWRB

Re: Intel 10GbE (ix) driver update - Looking for tests

[Hrvoje Popovski]

On 16.11.2016. 23:04, Mike Belopuhov wrote:
> Hi,
> 
> I've done a massive update of our ix(4) driver that brings
> support for X550 family of controllers including those
> integrated into new Xeon chips as well as QSFP support for
> X520 (82599) but this needs thorough testing.  If you're
> using Intel 10Gb controllers, please make sure that you
> either (or both!) test the complete diff found at this URL:
> http://gir.theapt.org/~mike/ixgbe.diff or next few snapshots
> that will (hopefully) contain bits of this monster diff.
> 
> To test the monster diff, make sure that you are running a
> recent snapshot and your kernel source code is up-to-date,
> then reset a few files to the specified revisions and
> remove the support file for X550:

Hi,

I'm testing plain forwarding on x520-da2 (82599) and on x552 SFP+ and it
seems that everything is working fine. Lots of ifconfig down/up, sh
netstart ix0/ix1 removing DAC cables inserting them, and it survived :)

performance:

x520-da2 (82599)
ix0 at pci6 dev 0 function 0 "Intel 82599" rev 0x01: msi, address
a0:36:9f:2e:96
:a0
ix1 at pci6 dev 0 function 1 "Intel 82599" rev 0x01: msi, address
a0:36:9f:2e:96
:a1

12 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.35 MHz

i'm getting 900Kpps


552 sfp+
ix0 at pci4 dev 0 function 0 "Intel X552" rev 0x00: msi, address
00:25:90:5d:ca:38
ix1 at pci4 dev 0 function 1 "Intel X552" rev 0x00: msi, address
00:25:90:5d:ca:39

4 x Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.33 MHz

i'm getting 850Kpps


will test more and report you back if something interesting shows up

thank you for doing this ...

Re: request for test: mfii

[Hrvoje Popovski]

On 26.10.2016. 5:50, YASUOKA Masahiko wrote:
> On Wed, 26 Oct 2016 10:26:19 +1100
> Jonathan Gray  wrote:
>> On Tue, Oct 25, 2016 at 05:29:55PM +0900, YASUOKA Masahiko wrote:
>>> I'm working on making mfii(4) bio(4) capable.
>>>
>>> If you have a machine which has mfii(4), I'd like you to test the diff
>>> following.  (It might be risky for production machines for this
>>> moment.)
>>>
>>> After the diff applied, bioctl(8) against the disk (eg. sd0) starts
>>> working and also "sysctl hw.sensors.mfii0" will appear.
>>>
>>> Especially if you can configure a hotspare, testing it is very
>>> helpful for me since I can't use a hotspare on my test machine.
> (snip)
>>> +   case BIOC_SATEST:
>>> +   cmd = MR_DCMD_SPEAKER_TEST;
>>> +   break;
>>> +   default:
>>> +   return (EINVAL);
>>> +   }
>>> +
>>> +   ccb = scsi_io_get(>sc_iopool, 0);
>>> +   rv = mfii_mgmt(sc, ccb, MR_DCMD_PD_SET_STATE, NULL,
>>> +   , sizeof(spkr), flags | SCSI_NOSLEEP);
>>
>> Should this be cmd rather than MR_DCMD_PD_SET_STATE?
>> The cmd values from the switch statement are not used.
> 
> Oops.  Yes, that's right.
> 
>>> +int
>>> +mfii_ioctl_blink(struct mfii_softc *sc, struct bioc_blink *bb)
> (snip)
>>> +   case BIOC_SBBLINK:
>>> +   case BIOC_SBALARM:
>>> +   cmd = MR_DCMD_PD_BLINK;
>>> +   break;
>>> +   default:
>>> +   rv = EINVAL;
>>> +   goto done;
>>> +   }
>>> +
>>> +   ccb = scsi_io_get(>sc_iopool, 0);
>>> +   rv = mfii_mgmt(sc, ccb, cmd, NULL, NULL, 0, SCSI_NOSLEEP);
>>> +   scsi_io_put(>sc_iopool, ccb);
> 
> Passing the mbox to mfii_mgmt() was missing.
> 
>>> + done:
>>> +   free(list, M_TEMP, sizeof(*list));
>>> +
>>> +   return (ENOTTY);
>>> +}
>>
>> Shouldn't this be return (rv) to return the EINVAL values?
>> With rv set to 0 before the 'done' to return 0 when there is no error?
> 
> Yes, that's also right.  Thanks.


Hi,

with this version boot stops here:

mfii0 at pci7 dev 0 function 0 "Symbios Logic MegaRAID SAS2208" rev
0x05: msi
mfii0: "ServeRAID M5110", firmware 23.34.0-0016, 512MB cache
scsibus1 at mfii0: 64 targets
sd0 at scsibus1 targ 0 lun 0:  SCSI3
0/direct fixed naa.600605b006c3a0b01a582bd010e4c053
sd0: 952720MB, 512 bytes/sector, 1951170560 sectors
mfii0: timeout on ccb 1008
mfii_mgmt cmd=50593792 failed status=255

Re: network performance fun

[Hrvoje Popovski]

On 25.10.2016. 0:22, Hrvoje Popovski wrote:
> On 24.10.2016. 23:36, Mike Belopuhov wrote:
>> On Mon, Oct 24, 2016 at 19:04 +0200, Hrvoje Popovski wrote:
>>> Hi all,
>>>
>>> OpenBSD box acts as transit router for /8 networks without pf and with
>>> this sysctls
>>>
>>> ddb.console=1
>>> kern.pool_debug=0
>>> net.inet.ip.forwarding=1
>>> net.inet.ip.ifq.maxlen=8192
>>>
>>> netstat
>>> 11/8   192.168.11.2   UGS0 114466419 - 8 ix0
>>> 12/8   192.168.12.2   UGS00 - 8 ix1
>>> 13/8   192.168.13.2   UGS00 - 8 myx0
>>> 14/8   192.168.14.2   UGS00 - 8 myx1
>>> 15/8   192.168.15.2   UGS00 - 8 em3
>>> 16/8   192.168.16.2   UGS0 89907239 - 8 em2
>>> 17/8   192.168.17.2   UGS0 65791508 - 8 bge0
>>> 18/8   192.168.18.2   UGS00 - 8 bge1
>>>
>>> while testing dlg@ "mcl2k2 mbuf clusters" patch with todays -current i
>>> saw that performance with plain -current drops for about 300Kpps vs
>>> -current from 06.10.2016. by bisecting cvs tree it seems that this
>>> commit is guilty for this
>>>
>>> http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/net/if_ethersubr.c?rev=1.240=text/x-cvsweb-markup
>>>
>>
>> I don't see how this change can affect performance in such a way
>> unless you're sending jumbo packets, but then the packet rates
>> are too high.  Are you 100% sure it's this particular change?
>>
> 
> No, no, i'm not 100% sure. I was doing this to try to find bottleneck:
> 
> cvs -q checkout -D "2016-10-XX" -P src
> 
> 2016-10-06 - 900kpps
> 2016-10-07 - 900kpps
> 2016-10-10 - 900kpps
> 2016-10-11 - 650kpps
> 2016-10-11 with if_ethersubr.c 1.239 - 900kpps
> ...
> 2016-10-14 - 650kpps
> 2016-10-14 with dlg@ patch - 900kpps
> 2016-10-14 with dlg@ patch and with if_ethersubr.c 1.239 - 880kpps
> 
> 2016-10-24 - results are in mail ...
> 
> and then i looked at networking diffs from 2016-10-10 and 2016-10-11 and
> it seems that if_ethersubr.c is guilty
> 
> tests was done over ix only ...
> 
> although as you can see with today's plain -current i'm getting 690kpps
> and with today's -current with if_ethersubr.c 1.239 i'm getting 910kpps
> 

just please see that bge performance are the same with if_ethersubr.c
1.239 or 1.241. i haven't test myx, will do that ...

Re: network performance fun

[Hrvoje Popovski]

On 24.10.2016. 23:36, Mike Belopuhov wrote:
> On Mon, Oct 24, 2016 at 19:04 +0200, Hrvoje Popovski wrote:
>> Hi all,
>>
>> OpenBSD box acts as transit router for /8 networks without pf and with
>> this sysctls
>>
>> ddb.console=1
>> kern.pool_debug=0
>> net.inet.ip.forwarding=1
>> net.inet.ip.ifq.maxlen=8192
>>
>> netstat
>> 11/8   192.168.11.2   UGS0 114466419 - 8 ix0
>> 12/8   192.168.12.2   UGS00 - 8 ix1
>> 13/8   192.168.13.2   UGS00 - 8 myx0
>> 14/8   192.168.14.2   UGS00 - 8 myx1
>> 15/8   192.168.15.2   UGS00 - 8 em3
>> 16/8   192.168.16.2   UGS0 89907239 - 8 em2
>> 17/8   192.168.17.2   UGS0 65791508 - 8 bge0
>> 18/8   192.168.18.2   UGS00 - 8 bge1
>>
>> while testing dlg@ "mcl2k2 mbuf clusters" patch with todays -current i
>> saw that performance with plain -current drops for about 300Kpps vs
>> -current from 06.10.2016. by bisecting cvs tree it seems that this
>> commit is guilty for this
>>
>> http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/net/if_ethersubr.c?rev=1.240=text/x-cvsweb-markup
>>
> 
> I don't see how this change can affect performance in such a way
> unless you're sending jumbo packets, but then the packet rates
> are too high.  Are you 100% sure it's this particular change?
> 

No, no, i'm not 100% sure. I was doing this to try to find bottleneck:

cvs -q checkout -D "2016-10-XX" -P src

2016-10-06 - 900kpps
2016-10-07 - 900kpps
2016-10-10 - 900kpps
2016-10-11 - 650kpps
2016-10-11 with if_ethersubr.c 1.239 - 900kpps
...
2016-10-14 - 650kpps
2016-10-14 with dlg@ patch - 900kpps
2016-10-14 with dlg@ patch and with if_ethersubr.c 1.239 - 880kpps

2016-10-24 - results are in mail ...

and then i looked at networking diffs from 2016-10-10 and 2016-10-11 and
it seems that if_ethersubr.c is guilty

tests was done over ix only ...

although as you can see with today's plain -current i'm getting 690kpps
and with today's -current with if_ethersubr.c 1.239 i'm getting 910kpps

so i thought that there must be something with if_ethersubr.c

> What kind of traffic are you testing this with?
> I assume small IP or UDP packets, correct?
> 

yes, 64 byte UDP without flowcontrol..

> Actually I'd like to know what causes this.
> 
> So far I've noticed that the code generating ICMP error doesn't
> reserve space for the link header but it's unlikely a culprit.
> (The diff was only compile tested so far...)
> 
> diff --git sys/netinet/ip_icmp.c sys/netinet/ip_icmp.c
> index cdd60aa..b3ddea4 100644
> --- sys/netinet/ip_icmp.c
> +++ sys/netinet/ip_icmp.c
> @@ -208,19 +208,21 @@ icmp_do_error(struct mbuf *n, int type, int code, 
> u_int32_t dest, int destmtu)
>  
>   if (icmplen + ICMP_MINLEN > MCLBYTES)
>   icmplen = MCLBYTES - ICMP_MINLEN - sizeof (struct ip);
>  
>   m = m_gethdr(M_DONTWAIT, MT_HEADER);
> - if (m && (sizeof (struct ip) + icmplen + ICMP_MINLEN > MHLEN)) {
> + if (m && (max_linkhdr + sizeof(struct ip) + icmplen +
> + ICMP_MINLEN > MHLEN)) {
>   MCLGET(m, M_DONTWAIT);
>   if ((m->m_flags & M_EXT) == 0) {
>   m_freem(m);
>   m = NULL;
>   }
>   }
>   if (m == NULL)
>   goto freeit;
> + m->m_data += max_linkhdr;
>   /* keep in same rtable */
>   m->m_pkthdr.ph_rtableid = n->m_pkthdr.ph_rtableid;
>   m->m_len = icmplen + ICMP_MINLEN;
>   if ((m->m_flags & M_EXT) == 0)
>   MH_ALIGN(m, m->m_len);
> 


with -current from few minutes ago and with this diff i'm getting panic

login: panic: pool_do_get: mbufpl free list modified: page
0xff00697e8000; item addr 0xff00697e8800; offset 0x0=
0x384500081c56 != 0xf2a4b1392c5839b2
Stopped at  Debugger+0x9:   leave
   TIDPIDUID PRFLAGS PFLAGS  CPU  COMMAND
*11010  11010 830x100012  02  ntpd
Debugger() at Debugger+0x9
panic() at panic+0xfe
pool_runqueue() at pool_runqueue
pool_get() at pool_get+0xb5
m_get() at m_get+0x28
m_getuio() at m_getuio+0x5c
sosend() at sosend+0x268
sendit() at sendit+0x258
sys_sendmsg() at sys_sendmsg+0xc0
syscall() at syscall+0x27b
--- syscall (number 28) ---
end of kernel
end trace frame: 0x7f7f11f0, count: 5
0xd9f5f7f362a:
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports.  Insufficient info makes it difficult to fi

network performance fun

[Hrvoje Popovski]

Hi all,

OpenBSD box acts as transit router for /8 networks without pf and with
this sysctls

ddb.console=1
kern.pool_debug=0
net.inet.ip.forwarding=1
net.inet.ip.ifq.maxlen=8192

netstat
11/8   192.168.11.2   UGS0 114466419 - 8 ix0
12/8   192.168.12.2   UGS00 - 8 ix1
13/8   192.168.13.2   UGS00 - 8 myx0
14/8   192.168.14.2   UGS00 - 8 myx1
15/8   192.168.15.2   UGS00 - 8 em3
16/8   192.168.16.2   UGS0 89907239 - 8 em2
17/8   192.168.17.2   UGS0 65791508 - 8 bge0
18/8   192.168.18.2   UGS00 - 8 bge1

while testing dlg@ "mcl2k2 mbuf clusters" patch with todays -current i
saw that performance with plain -current drops for about 300Kpps vs
-current from 06.10.2016. by bisecting cvs tree it seems that this
commit is guilty for this

http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/net/if_ethersubr.c?rev=1.240=text/x-cvsweb-markup


-current from 24.10.2016

ix
sendreceive
690Kpps 690Kpps
700Kpps 680Kpps
800Kpps 350Kpps
1.4Mpps 305Kpps
14Mpps  305Kpps

em
sendreceive
690Kpps 690Kpps
700Kpps 680Kpps
800Kpps 700Kpps
1.4Mpps 700Kpps

bge
sendreceive
620Kpps 620Kpps
630Kpps 515Kpps
700Kpps 475Kpps
800Kpps 430Kpps
1.4Mpps 350Kpps


-current with if_ethersubr.c version 1.239

ix
sendreceive
910Kpps 910Kpps
920Kpps 820Kpps
1Mpps   825Kpps
1.4Mpps 700Kpps
14Mpps  700Kpps

em
sendreceive
940Kpps 940Kpps
950Kpps 845Kpps
1Mpps   855Kpps
1.4Mpps 845Kpps

bge
sendreceive
620Kpps 620Kpps
630Kpps 525Kpps
700Kpps 485Kpps
800Kpps 435Kpps
1.4Mpps 350Kpps 


applying dlg@ "mcl2k2 mbuf clusters" patch to todays -current brings
performance back to ix and em ... bge is emotional as always :)

ix
sendreceive
900Kpps 900Kpps
910Kpps 895Kpps
1Mpps   895Kpps
1.4Mpps 810Kpps
14Mpps  815Kpps

em
sendreceive
940Kpps 940Kpps
950Kpps 930Kpps
1Mpps   920Kpps
1.4Mpps 930Kpps

bge
sendreceive
620Kpps 620Kpps
630Kpps 520Kpps
700Kpps 475Kpps
800Kpps 430Kpps
1.4Mpps 366Kpps



i honestly don't know what all that means, i'm just writing my
observation ...



OpenBSD 6.0-current (GENERIC.MP) #5: Mon Oct 24 18:12:28 CEST 2016
r...@x3550m4.my.domain:/usr/src/sys/arch/amd64/compile/GENERIC.MP
RTC BIOS diagnostic error 80
real mem = 34315051008 (32725MB)
avail mem = 33270534144 (31729MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7e67c000 (84 entries)
bios0: vendor IBM version "-[D7E154BUS-2.21]-" date 08/08/2016
bios0: IBM IBM System x3550 M4 Server -[7914T91]-
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP TCPA ERST HEST HPET APIC MCFG OEM0 OEM1 SLIT
SRAT SLIC SSDT SSDT SSDT SSDT DMAR
acpi0: wakeup devices MRP1(S4) DCC0(S4) MRP3(S4) MRP5(S4) EHC2(S5)
PEX0(S5) PEX7(S5) EHC1(S5) IP2P(S3) MRPB(S4) MRPC(S4) MRPD(S4) MRPF(S4)
MRPG(S4) MRPH(S4) MRPI(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.36 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2399.99 MHz
cpu2:

Re: MSI-X support

[Hrvoje Popovski]

On 4.5.2016. 16:32, Mark Kettenis wrote:
>>> This is great, thanks for doing this!  I'm a bit surprised that
>>> we don't need to the same suspend/resume dance in ppb(4) as with
>>> MSI.
>>>
>>
>> That is an excellent point I overlooked. Kettenis, do we?
> 
> Almost certainly.  I committed the diff, but left the bits out that
> start using it in em(4) and xhci(4).
> 

Hi all,

is it a good time to enable msi-x on em(4) ?

Re: bigger mbuf clusters for sosend()

[Hrvoje Popovski]

On 22.8.2016. 8:20, David Gwynne wrote:
> 
>> On 22 Aug 2016, at 03:36, Hrvoje Popovski <hrv...@srce.hr> wrote:
>>
>> On 13.8.2016. 10:59, Claudio Jeker wrote:
>>> This diff refactors the uio to mbuf code to make use of bigger buffers (up
>>> to 64k) and also switches the MCLGET to use M_WAIT like the MGET calls in
>>> the same function. I see no point in not waiting for a cluster and instead
>>> chain lots of mbufs together as a consequence.
>>>
>>> This makes in my opinion the code easier to read and allows for further
>>> optimizations (like using non-DMA reachable mbufs for AF_UNIX sockets).
>>>
>>> This increased the preformance of loopback connections significantly when
>>> I tested this at n2k16.
>>>
>>
>> Hi,
>>
>> it seems that this patch speeds up forwarding for about 40kpps. At least
>> with my test box and with this setup :)
>> Which means that -current can forward full 10Gbps with 1500byte packets :)
> 
> thats kind of nuts cos this shouldnt affect the forwarding path at all.
> 
> im keen for it to go in still though. claudio?
> 
> dlg
> 

yes, you are right ... this is my stupid mistake, i compared results
from 15.08.2016 with results from 21.08.2016 with claudio@ patch without
first testing plain source and then test same setup with claudio@ patch ...

i'm very sorry for making this stupid mistake ...

Re: bigger mbuf clusters for sosend()

[Hrvoje Popovski]

On 13.8.2016. 10:59, Claudio Jeker wrote:
> This diff refactors the uio to mbuf code to make use of bigger buffers (up
> to 64k) and also switches the MCLGET to use M_WAIT like the MGET calls in
> the same function. I see no point in not waiting for a cluster and instead
> chain lots of mbufs together as a consequence.
> 
> This makes in my opinion the code easier to read and allows for further
> optimizations (like using non-DMA reachable mbufs for AF_UNIX sockets).
> 
> This increased the preformance of loopback connections significantly when
> I tested this at n2k16.
> 

Hi,

it seems that this patch speeds up forwarding for about 40kpps. At least
with my test box and with this setup :)
Which means that -current can forward full 10Gbps with 1500byte packets :)

pf=NO
ddb.panic=1
ddb.console=1
kern.pool_debug=0
kern.maxclusters=32768
net.inet.ip.forwarding=1
net.inet.ip.ifq.maxlen=8192


sending from 12.0.0.11 to 11.0.0.11


# netstat -rnf inet | grep ix
11/8   192.168.11.2   UGS0 118844402 - 8 ix0
12/8   192.168.12.2   UGS00 - 8 ix1
192.168.11.0/30192.168.11.1   UC 10 - 4 ix0
192.168.11.1   a0:36:9f:2e:96:a0  UHLl   01 - 1 ix0
192.168.11.2   90:e2:ba:1a:df:85  UHLc   12 - 4 ix0
192.168.11.3   192.168.11.1   UHb00 - 1 ix0
192.168.12.0/30192.168.12.1   UC 00 - 4 ix1
192.168.12.1   a0:36:9f:2e:96:a1  UHLl   00 - 1 ix1
192.168.12.3   192.168.12.1   UHb00 - 1 ix1

without patch
sending receiving
800Kpps 800kpps
840Kpps 840kpps
850Kpps 770kpps
1.4Mpps 690kpps
14Mpps  685kpps

with patch
sending receiving
800kpps 800kpps
880kpps 880kpps
890kpps 790kpps
1.4Mpps 700kpps
14Mpps  700kpps


# netstat -i 1
  em0 inem0 out  total in  total out
 packets  errs  packets  errs colls   packets  errs  packets  errs colls
   2 02 0 0888799 0   888779 0 0
   1 01 0 0889240 0   889124 0 0
   1 01 0 0889296 0   889407 0 0
   1 01 0 0888941 0   888932 0 0
   1 01 0 0889268 0   889291 0 0
   1 01 0 0889095 0   889200 0 0


OpenBSD 6.0-current (GENERIC.MP) #0: Sun Aug 21 18:57:24 CEST 2016
r...@x3550m4.my.domain:/usr/src/sys/arch/amd64/compile/GENERIC.MP
RTC BIOS diagnostic error 80
real mem = 34315051008 (32725MB)
avail mem = 33270591488 (31729MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7e67c000 (84 entries)
bios0: vendor IBM version "-[D7E146CUS-1.82]-" date 04/09/2015
bios0: IBM IBM System x3550 M4 Server -[7914T91]-
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP TCPA ERST HEST HPET APIC MCFG OEM0 OEM1 SLIT
SRAT SLIC SSDT SSDT SSDT SSDT DMAR
acpi0: wakeup devices MRP1(S4) DCC0(S4) MRP3(S4) MRP5(S4) EHC2(S5)
PEX0(S5) PEX7(S5) EHC1(S5) IP2P(S3) MRPB(S4) MRPC(S4) MRPD(S4) MRPF(S4)
MRPG(S4) MRPH(S4) MRPI(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.35 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu2:

Re: MSI-X support

[Hrvoje Popovski]

On 4.5.2016. 8:50, Mark Kettenis wrote:
>> Date: Tue, 3 May 2016 15:23:07 -0700
>> From: Mike Larkin 
>>
>> On Tue, May 03, 2016 at 09:40:28PM +0200, Mark Kettenis wrote:
>>> Today mpi@ reminded me that I had written support for MSI-X some time
>>> ago.  Since he is interested in using multiple vectors, I extended the
>>> code I had a bit to support that feature as well.  This introduces a
>>> new function:
>>>
>>> int pci_intr_map_msix(struct pci_attach_args *, int, pci_intr_handle_t *);
>>>
>>> You use it like pci_intr_map_msi(9) and pci_intr_map(9), buthave to
>>> pass it a vector number as the 2nd argument.  Typically you'll pass 0,
>>> wich will map the 1st vector, which is always there on hardware with
>>> MSI-X support.  Some hardware supports more than 1 vector.  Typical
>>> examples are network cards that support multiple rings.  Please be
>>> aware that on architectures like i386 and amd64, the number of
>>> interrupt vectors at each level is limited.  If you consume too many
>>> of them, devices may fail to attach.
>>>
>>> As for the implementation.  This only adds code for amd64.  I'll
>>> probably add support for sparc64 as well.  I'm hesitant about i386
>>> because it has even less available interrupt vectors than amd64.
>>>
>>> Notice that the amd64 implementation uses _bus_space_map() and
>>> _bus_space_unmap().  That is deliberate.  The MSI-X registers miht
>>> share a BAR with other registers that our drivers already map.  The
>>> underscored versions of the mapping routines make sure that we don't
>>> fail to map things in that case.
>>>
>>> ok?
>>>
>>
>> Did you want to include the if_em and xhci bits too?
> 
> Yes.  They seem to work.  But I can leave them out if people deem this
> too risky.  They'll be separate commits anyway.

Hi,

it seems to work fine here ...

em0 at pci9 dev 0 function 0 "Intel I350" rev 0x01: msix, address
40:f2:e9:10:56:1c
em1 at pci9 dev 0 function 1 "Intel I350" rev 0x01: msix, address
40:f2:e9:10:56:1d
em2 at pci9 dev 0 function 2 "Intel I350" rev 0x01: msix, address
40:f2:e9:10:56:1e
em3 at pci9 dev 0 function 3 "Intel I350" rev 0x01: msix, address
40:f2:e9:10:56:1f


dmesg
OpenBSD 5.9-current (GENERIC.MP) #1: Tue May  3 23:41:02 CEST 2016
r...@x3550m4.my.domain:/usr/src/sys/arch/amd64/compile/GENERIC.MP
RTC BIOS diagnostic error 80
real mem = 34314878976 (32725MB)
avail mem = 33270431744 (31729MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7e67c000 (84 entries)
bios0: vendor IBM version "-[D7E146CUS-1.82]-" date 04/09/2015
bios0: IBM IBM System x3550 M4 Server -[7914T91]-
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP TCPA ERST HEST HPET APIC MCFG OEM0 OEM1 SLIT
SRAT SLIC SSDT SSDT SSDT SSDT SSDT SSDT DMAR
acpi0: wakeup devices MRP1(S4) DCC0(S4) MRP3(S4) MRP5(S4) EHC2(S5)
PEX0(S5) PEX7(S5) EHC1(S5) IP2P(S3) MRPB(S4) MRPC(S4) MRPD(S4) MRPF(S4)
MRPG(S4) MRPH(S4) MRPI(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.40 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu3:

Re: Moving away from softnet interrupts

[Hrvoje Popovski]

On 18.4.2016. 15:31, Hrvoje Popovski wrote:
> On 18.4.2016. 10:50, Martin Pieuchot wrote:
>> The current goal of the Network SMP effort is to have a single CPU
>> process the IP forwarding path in a process context without holding
>> the KERNEL_LOCK().  To achieve this goal we're progressively moving
>> code from the softnet interrupt context to the if_input_task.  In
>> the end we'll completely get rid of this soft-interrupt.
>>
>> So now would be a good time to know if moving all the code currently
>> run in a soft-interrupt context to a task uncovers any bug.  I'm
>> happily running the diff below on amd64 and macppc, it even gives me
>> a small performance boost.
>>
>> I'd appreciate more tests especially on exotic archs.
> 
> Hi,
> 
> i have tested this patch over ix interfaces on box with 12 cores
> Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.39 MHz
> dmesg is below results ..
> 
> 
> pf=NO
> ddb.console=1
> kern.pool_debug=0
> kern.maxclusters=32768
> net.inet.ip.forwarding=1
> net.inet.ip.ifq.maxlen=8192
> 
> while sending traffic i was shutting down physical interfaces and bridge
> and box survived this ...
> 
> routing over ix
> 
> send  receive
> 400kpps   400kpps
> 500kpps   500kpps
> 600kpps   600kpps - after cca 5 min drops down to 490kpps
> 650kpps   650kpps
> 700kpps   640kpps
> 800kpps   640kpps
> 1.4Mpps   600kpps
> 14Mpps600kpps
> 
> 
> bridge over ix
> 
> send  receive
> 400kpps   400kpps
> 500kpps   500kpps - after cca 1min drops down to 450kpps
> 600kpps   600kpps
> 650kpps   640kpps
> 700kpps   610kpps
> 800kpps   560kpps
> 1.4Mpps   370kpps
> 14Mpps370kpps
> 

routing over vlan

sendreceive
400kpps 400kpps
480kpps 480kpps
500kpps 450kpps
600kpps 450kpps
800kpps 450kpps
1.4Mpps 310kpps
14Mpps  310kpps

# ifconfig vlan
vlan111: flags=18843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,MPSAFE> mtu 1500
lladdr a0:36:9f:2e:96:a0
index 14 priority 0
vlan: 111 parent interface: ix0
vnetid: 111
parent: ix0
groups: vlan
status: active
inet 10.113.0.1 netmask 0x broadcast 10.113.255.255
vlan112: flags=18843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,MPSAFE> mtu 1500
lladdr a0:36:9f:2e:96:a1
index 15 priority 0
vlan: 112 parent interface: ix1
vnetid: 112
parent: ix1
groups: vlan
status: active
inet 10.114.0.1 netmask 0x broadcast 10.114.255.255

Re: Moving away from softnet interrupts

[Hrvoje Popovski]

On 18.4.2016. 10:50, Martin Pieuchot wrote:
> The current goal of the Network SMP effort is to have a single CPU
> process the IP forwarding path in a process context without holding
> the KERNEL_LOCK().  To achieve this goal we're progressively moving
> code from the softnet interrupt context to the if_input_task.  In
> the end we'll completely get rid of this soft-interrupt.
> 
> So now would be a good time to know if moving all the code currently
> run in a soft-interrupt context to a task uncovers any bug.  I'm
> happily running the diff below on amd64 and macppc, it even gives me
> a small performance boost.
> 
> I'd appreciate more tests especially on exotic archs.

Hi,

i have tested this patch over ix interfaces on box with 12 cores
Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.39 MHz
dmesg is below results ..


pf=NO
ddb.console=1
kern.pool_debug=0
kern.maxclusters=32768
net.inet.ip.forwarding=1
net.inet.ip.ifq.maxlen=8192

while sending traffic i was shutting down physical interfaces and bridge
and box survived this ...

routing over ix

sendreceive
400kpps 400kpps
500kpps 500kpps
600kpps 600kpps - after cca 5 min drops down to 490kpps
650kpps 650kpps
700kpps 640kpps
800kpps 640kpps
1.4Mpps 600kpps
14Mpps  600kpps


bridge over ix

sendreceive
400kpps 400kpps
500kpps 500kpps - after cca 1min drops down to 450kpps
600kpps 600kpps
650kpps 640kpps
700kpps 610kpps
800kpps 560kpps
1.4Mpps 370kpps
14Mpps  370kpps


# ifconfig ix
ix0:
flags=18b43
mtu 1500
lladdr a0:36:9f:2e:96:a0
description: connected to test1.srce.hr (p2p1)
index 3 priority 0
media: Ethernet autoselect (10GSFP+Cu full-duplex)
status: active
ix1:
flags=18b43
mtu 1500
lladdr a0:36:9f:2e:96:a1
index 4 priority 0
media: Ethernet autoselect (10GSFP+Cu full-duplex)
status: active

# ifconfig bridge0
bridge0: flags=41
index 14
groups: bridge
priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto
rstp
designated: id 00:00:00:00:00:00 priority 0
ix0 flags=3
port 3 ifpriority 0 ifcost 0
ix1 flags=3
port 4 ifpriority 0 ifcost 0
Addresses (max cache: 100, timeout: 240):
90:e2:ba:33:af:ed ix1 1 flags=0<>
90:e2:ba:33:af:ec ix0 1 flags=0<>



OpenBSD 5.9-current (GENERIC.MP) #0: Mon Apr 18 11:15:43 CEST 2016
r...@x3550m4.my.domain:/usr/src/sys/arch/amd64/compile/GENERIC.MP
RTC BIOS diagnostic error 80
real mem = 34315018240 (32725MB)
avail mem = 33270788096 (31729MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7e67c000 (84 entries)
bios0: vendor IBM version "-[D7E146CUS-1.82]-" date 04/09/2015
bios0: IBM IBM System x3550 M4 Server -[7914T91]-
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP TCPA ERST HEST HPET APIC MCFG OEM0 OEM1 SLIT
SRAT SLIC SSDT SSDT SSDT SSDT SSDT SSDT DMAR
acpi0: wakeup devices MRP1(S4) DCC0(S4) MRP3(S4) MRP5(S4) EHC2(S5)
PEX0(S5) PEX7(S5) EHC1(S5) IP2P(S3) MRPB(S4) MRPC(S4) MRPD(S4) MRPF(S4)
MRPG(S4) MRPH(S4) MRPI(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.35 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz, 2400.00 MHz
cpu2:

Re: ix(4): enable checksum offload

[Hrvoje Popovski]

On 9.9.2013. 22:07, Mike Belopuhov wrote:
> On 9 September 2013 21:48, Brad Smith  wrote:
>> Here is a diff to enable the checksum offload support for ix(4).
>>
>> Looking for any testing.
>>
> 
> last time i checked this broke ospf traffic.  please make sure at least
> ip/tcp, ip/udp, ip/icmp, ip/ip, ip/gre, ip/esp, ip/ah and ip/ospf work fine
> with this.
> 

Hi all,

is this still interesting topic? if it is, i have testbed to test csum
for em i350 (i350v2 is on the way) and ix 82599 or x540. and week ago i
ordered
http://www.supermicro.com/products/system/1U/5018/SYS-5018D-FN8T.cfm
with x552 or x557 not sure, i350-am4 and i210 ... lots of em's and ix's :)

Re: New scheduler for OpenBSD

[Hrvoje Popovski]

On 18.3.2016. 20:00, Mark Kettenis wrote:
> One other important case to test is network packet forwarding.  Some
> of our network stack is now running inside a kernel thread.  And any
> changes in the scheduling behaviour have the potential of having a
> significant impact there.

I've done some basic PPS test over ix interface's and numbers are the
same with or without new schedule, ie. 720pps.

routing only
pf=NO
kern.pool_debug=0
net.inet.ip.forwarding=1
net.inet.ip.ifq.maxlen=8192
kern.maxclusters=32768

renesas unknown product on dell r630

[Hrvoje Popovski]

hi,

this patch adds renesas product on dell r630


dmesg on dell r630 without patch:

pci6 at ppb5 bus 6
ppb6 at pci6 dev 0 function 0 vendor "Renesas", unknown product 0x001d
rev 0x00
pci7 at ppb6 bus 7
ppb7 at pci7 dev 0 function 0 vendor "Renesas", unknown product 0x001d
rev 0x00
pci8 at ppb7 bus 8
ppb8 at pci8 dev 0 function 0 vendor "Renesas", unknown product 0x001a
rev 0x00
pci9 at ppb8 bus 9



dmesg on dell r630 with patch:

pci6 at ppb5 bus 6
ppb6 at pci6 dev 0 function 0 "Renesas SH7758 PCIE Switch" rev 0x00
pci7 at ppb6 bus 7
ppb7 at pci7 dev 0 function 0 "Renesas SH7758 PCIE Switch" rev 0x00
pci8 at ppb7 bus 8
ppb8 at pci8 dev 0 function 0 "Renesas SH7758 PCIE-PCI" rev 0x00
pci9 at ppb8 bus 9


this is lspci -nn from linux

06:00.0 PCI bridge [0604]: Renesas Technology Corp. SH7758 PCIe Switch
[PS] [1912:001d]
07:00.0 PCI bridge [0604]: Renesas Technology Corp. SH7758 PCIe Switch
[PS] [1912:001d]
08:00.0 PCI bridge [0604]: Renesas Technology Corp. SH7758 PCIe-PCI
Bridge [PPB] [1912:001a]


whole dmesg:

OpenBSD 5.9 (GENERIC.MP) #0: Sat Feb 20 14:59:33 CET 2016
r...@fw2.my.domain:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34107559936 (32527MB)
avail mem = 33069666304 (31537MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x7ab09000 (74 entries)
bios0: vendor Dell Inc. version "1.5.4" date 10/002/2015
bios0: Dell Inc. PowerEdge R630
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP MCEJ WD__ SLIC HPET APIC MCFG MSCT PMCT SLIT
SRAT SSDT SSDT SSDT PRAD HEST BERT ERST EINJ
acpi0: wakeup devices PCI0(S4) BR1A(S4) BR2A(S4) BR2C(S4) BR3A(S4)
BR3B(S4) BR3C(S4) XHC_(S0) RP02(S4) RP03(S4) RP05(S4) RP08(S4) PCI1(S4)
QRP0(S4) QR1A(S4) QR2A(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2637 v3 @ 3.50GHz, 3600.44 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
cpu1 at mainbus0: apid 16 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2637 v3 @ 3.50GHz, 3600.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 0, package 1
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2637 v3 @ 3.50GHz, 3600.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 18 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2637 v3 @ 3.50GHz, 3600.00 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 1, package 1
cpu4 at mainbus0: apid 8 (application processor)
cpu4: Intel(R) Xeon(R) CPU E5-2637 v3 @ 3.50GHz, 3600.00 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,SENSOR,ARAT
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 0, core 4, package 0
cpu5 at mainbus0: apid 24 (application processor)
cpu5: Intel(R) Xeon(R) CPU E5-2637 v3 @ 3.50GHz, 3600.00 MHz
cpu5:

snapshot from 26-Jan-2016 - pfsync panic

[Hrvoje Popovski]

Hi all,

i have pf,carp,pfsync and dhcpd setup with 2 Dell R610. today i updated
my secondary firewall with latest snapshot from
http://ftp2.eu.openbsd.org/ 
install59.iso   26-Jan-2016 04:00

and after update secondary firewall panic
http://kosjenka.srce.hr/~hrvoje/crash2.jpg

i couldn't type anything in ddb console ..

i disabled pfsync0 and syncdev bnx3 on primary firewall and then
secondary boots up..

would this commit
http://permalink.gmane.org/gmane.os.openbsd.cvs/152882
resolve this problem?


pf.conf
set limit states 20
set skip on { lo bnx3 enc0 }
table  { XXX }

match out on bnx0 from 10.111/16 nat-to XXX/29 port 1024:65535

block
pass quick proto carp keep state (no-sync)
pass in quick on bnx0 from 
pass out log on bnx0
pass in log on bnx1 from 10.111/16
pass inet proto icmp icmp-type { echoreq, unreach code needfrag }



dmesg

OpenBSD 5.9-beta (GENERIC.MP) #1864: Mon Jan 25 19:11:29 MST 2016
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 12854988800 (12259MB)
avail mem = 12461199360 (11883MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xcf49c000 (84 entries)
bios0: vendor Dell Inc. version "6.4.0" date 07/23/2013
bios0: Dell Inc. PowerEdge R610
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC SPCR HPET DM__ MCFG WD__ SLIC ERST HEST
BERT EINJ SRAT TCPA
acpi0: wakeup devices PCI0(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 32 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2660.41 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 1
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 133MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 0 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2660.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 0, package 0
cpu2 at mainbus0: apid 34 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2660.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 1
cpu3 at mainbus0: apid 2 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2660.00 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 1, package 0
cpu4 at mainbus0: apid 50 (application processor)
cpu4: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2660.00 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 0, core 9, package 1
cpu5 at mainbus0: apid 18 (application processor)
cpu5: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2660.00 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 0, core 9, package 0
cpu6 at mainbus0: apid 52 (application processor)
cpu6: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2660.00 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 0, core 10, package 1
cpu7 at mainbus0: apid 20 (application processor)
cpu7: Intel(R) Xeon(R) CPU E5630 @ 2.53GHz, 2660.00 MHz
cpu7:

Re: DDB causing lost keystrokes on Dell iDRAC console (not inside ddb)

[Hrvoje Popovski]

On 23.1.2016. 23:29, Adam McDougall wrote:
> Hello,
> 
> I have a few Dell servers which I've installed OpenBSD for testing
> but ran into a problem with keystroke loss on the console when used
> through the Dell iDRAC remote graphical console.  Surprisingly it
> operates perfectly fine in the installer (thankfully) but when booted
> from a formal install, I lose 25-50% of my keystrokes.  The speed that
> I type does not matter, the keys I type do not matter, they are randomly
> lost.  This is any typing, for example at the login prompt (where it
> becomes very difficult to login).
> 
> Example:
> login: Tisis metypingat he onole
> 
> should have shown: This is me typing at the console
> 
> I was able to determine having "option DDB" in the kernel is a single
> factor leading to keystroke loss.  If I recompile the GENERIC amd64
> 5.8 kernel without DDB, I have no keystroke loss.  Also, if I enter
> DDB while the kernel is running, I have no keystroke loss.
> 
> I have noticed this issue on 5.8 release, a recent 5.9 snapshot,
> and at least the Dell models R420 and R430.  The iDRAC supplies a
> virtual usb keyboard to the OS.  I have not had this issue with other
> OSes.  I should generally be able to perform further tests as requested
> especially this weekend.
> 
> Does anyone have suggestions?  Can I disable DDB without recompiling?
> Thanks.
> 

Hi,

please see...
http://marc.info/?l=openbsd-tech=143700306821021=2


month ago i tried to install snapshot on r630 and fc630 and i manage to
do that but i was hard and painful :)

it seems it works when you keep the key pressed down for one second or
so - then character on the console will be shown. this works for me for
idrac7 and 8 :)