Re: ACPI related performance trouble

2021-02-25 Thread Emmanuel Dreyfus
Brian Buhrow  wrote:

>   hello.  I wonder if you've compared your BIOS settings on both
> machines?  While the BIOS may be the same version, it's possible the
> settings are not identical.  this is strongly suggested by the fact that
> one of your machines shows a serial number in its machdep.dmi output,
> while the other does not.

I double checked. Both machine report it, but the information is only
shown to root. I must have run the sysctl command as an unprivilegied
user on the slow machine.

> Doing that comparison will be tedious, but I think worth it.

Indeed; In an ideal world, there would be a way to get a MIB of BIOS
settings.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
m...@netbsd.org


Re: ACPI related performance trouble

2021-02-25 Thread Brian Buhrow
hello.  I wonder if you've compared your BIOS settings on both 
machines?  While the BIOS
may be the same version, it's possible the settings are not identical.  this is 
strongly
suggested by the fact that one of your machines shows a serial number in its 
machdep.dmi
output, while the other does not.  Doing that comparison will be tedious, but I 
think worth it.
Perhaps the easiest way is to reset the slow machine to factory defaults, then 
do your
comparison, screen by screen with the fast machine.

-Brian

On Feb 25,  2:38pm, Emmanuel Dreyfus wrote:
} Subject: Re: ACPI related performance trouble
} Joerg Sonnenberger  wrote:
} 
} > Have you compared the machdep sysctl?
} 
} Here it is. 
} 
} --- sysctl.glutamine
} +++ sysctl.leucine


Re: ACPI related performance trouble

2021-02-25 Thread Emmanuel Dreyfus
Michael van Elst  wrote:

> As for performance, the common issue was that the ACPI interrupt
> isn't handled and you get several thousand interrupts per second
> that slow down everything.

You spot it:

glutamine# vmstat -iv
interrupt   total rate
TLB shootdown   135950
cpu0 timer3312564   99
ioapic0 pin 9   282417042 8525
msi0 vec 03200
msi1 vec 0  213150
msi2 vec 0  00
msix3 vec 0 00
msi4 vec 0  00
ioapic0 pin 16  00
msi5 vec 0  592771
ioapic0 pin 4   242470
Total   285848360 8628



-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
m...@netbsd.org


Re: ACPI related performance trouble

2021-02-25 Thread Michael van Elst
m...@netbsd.org (Emmanuel Dreyfus) writes:

>Joerg Sonnenberger  wrote:

>> Have you compared the machdep sysctl?

>Here it is. 

>+machdep.dmi.system-serial = 0123456789
>+machdep.dmi.system-uuid = ----ac1f6b747c48

The serial numbers are only shown to root.

As for performance, the common issue was that the ACPI interrupt
isn't handled and you get several thousand interrupts per second
that slow down everything.

E.g. (For a good case):

acpi0: SCI interrupting at int 9

% vmstat -iv
TLB shootdown6572617   16
cpu0 timer  41040041   99
ioapic0 pin 9  00   <-
msix0 vec 000
msix0 vec 1  31115987
msix0 vec 2   9428662
msix0 vec 3   2513640
msix0 vec 4   1872870
msi1 vec 0 00
msi2 vec 0   39273009
ioapic0 pin 16230
msix3 vec 000
msix3 vec 100
msix3 vec 200
msix3 vec 300
msix3 vec 400
ioapic0 pin 23260
msi4 vec 0   6111286   14
ioapic0 pin 18 00
ioapic0 pin 4   19990
Total   62146407  151

% intrctl list
interrupt id  CPU0CPU1CPU2CPU3  device name(s)
ioapic0 pin 90*  0   0   0  acpi SCI   <---
msix0 vec 0  0*  0   0   0  nvme0 adminq
msix0 vec 13111598*  0   0   0  nvme0 ioq1
msix0 vec 2  0  942866*  0   0  nvme0 ioq2
msix0 vec 3  0   0  251364*  0  nvme0 ioq3
msix0 vec 4  0   0   0  187287* nvme0 ioq4
msi1 vec 0   0*  0   0   0  xhci0
msi2 vec 0 3927265*  0   0   0  wm0
ioapic0 pin 16  23*  0   0   0  ehci0, i915drmkms0
msix3 vec 0  0*  0   0   0  wm1TXRX0
msix3 vec 1  0   0*  0   0  wm1TXRX1
msix3 vec 2  0   0   0*  0  wm1TXRX2
msix3 vec 3  0   0   0   0* wm1TXRX3
msix3 vec 4  0*  0   0   0  wm1LINK
ioapic0 pin 23  26*  0   0   0  ehci1
msi4 vec 0 6111264*  0   0   0  ahcisata0
ioapic0 pin 18   0*  0   0   0  ichsmb0
ioapic0 pin 4 1999*  0   0   0  com0


-- 
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."


Re: ACPI related performance trouble

2021-02-25 Thread Emmanuel Dreyfus
Joerg Sonnenberger  wrote:

> Have you compared the machdep sysctl?

Here it is. 

--- sysctl.glutamine
+++ sysctl.leucine
@@ -660,9 +660,9 @@
 hw.wd1.use_ncq_prio = 0
 machdep.biosbasemem = 611
 machdep.biosextmem = 1047552
 machdep.booted_kernel = /netbsd
-machdep.diskinfo: 80:30031872(845/255/63),2 81:1953525168(1023/255/63),2 82:195
3525168(1023/255/63),2  dk0 dk1 dk2 dk3 wd0:81 wd1:82 sd0:80
+machdep.diskinfo: 80:30031872(845/255/63),2 81:1953525168(1023/255/63),2 82:195
3525168(1023/255/63),2  dk0 dk1 dk2 dk3 wd0:81,82 wd1:81,82 sd0:80
 machdep.fpu_present = 1
 machdep.osfxsr = 1
 machdep.sse = 1
 machdep.sse2 = 1
@@ -671,17 +671,21 @@
 machdep.xsave_features = 31
 machdep.dmi.system-vendor = Supermicro
 machdep.dmi.system-product = C7Z370-CG-L
 machdep.dmi.system-version = 0123456789
+machdep.dmi.system-serial = 0123456789
+machdep.dmi.system-uuid = ----ac1f6b747c48
 machdep.dmi.bios-vendor = American Megatrends Inc.
 machdep.dmi.bios-version = 1.2c
 machdep.dmi.bios-date = 20190617
 machdep.dmi.board-vendor = Supermicro
 machdep.dmi.board-product = C7Z370-CG-L
 machdep.dmi.board-version = 1.01
+machdep.dmi.board-serial = ZM188S015949
 machdep.dmi.board-asset-tag = Default string
 machdep.dmi.chassis-vendor = Supermicro
 machdep.dmi.chassis-version = 0123456789
+machdep.dmi.chassis-serial = 0123456789
 machdep.dmi.chassis-asset-tag = Default string
 machdep.dmi.processor-vendor = Intel(R) Corporation
 machdep.dmi.processor-version = Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz
 machdep.dmi.processor-frequency = 2800 MHz

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
m...@netbsd.org


Re: ACPI related performance trouble

2021-02-25 Thread Joerg Sonnenberger
On Thu, Feb 25, 2021 at 11:46:21AM +0100, Emmanuel Dreyfus wrote:
> I just got two identical machines, let us call them glutamine and leucine. I
> run ffmpeg4 to transcode H264 video to webm, and leucine is about 12 times
> faster than glutamine.

Have you compared the machdep sysctl?

Joerg


ACPI related performance trouble

2021-02-25 Thread Emmanuel Dreyfus
Hello

I just got two identical machines, let us call them glutamine and leucine. I
run ffmpeg4 to transcode H264 video to webm, and leucine is about 12 times
faster than glutamine.

The hardware and software are the same, the BIOS revision are identical. The
only difference dmesg tells me about beyond ethernet MAC addresses are the
ACPI table addresses, which are a bit different.

Following the hint about ACPI, I tried booting glutamine with option -2
(disable ACPI). That boosts its performance on par with leucine.

I can live without ACPI, but I still wonder what could cause the problem.
Anyone has an idea? I compared acpidump -d -outputs on the two machines. The
only differences are in dsdt, ssdt2 and ssdt7 and even there, they are scarce.
It looks like if it only about addresses.

--- glutamine/dsdt.dsl  2021-02-25 11:41:36.0 +0100
+++ leucine/dsdt.dsl2021-02-25 11:41:48.0 +0100
@@ -4,15 +4,15 @@
  * Copyright (c) 2000 - 2016 Intel Corporation
  * 
  * Disassembling to symbolic ASL+ operators
  *
- * Disassembly of dsdt.dat, Thu Feb 25 10:41:12 2021
+ * Disassembly of dsdt.dat, Thu Feb 25 11:40:56 2021
  *
  * Original Table Header:
  * Signature"DSDT"
  * Length   0x00029C56 (171094)
  * Revision 0x02
- * Checksum 0x89
+ * Checksum 0xE9
  * OEM ID   "SUPERM"
  * OEM Table ID "SMCI--MB"
  * OEM Revision 0x01072009 (17244169)
  * Compiler ID  "INTL"
@@ -278,9 +278,9 @@
 Name (IOST, 0x4001)
 Name (TOPM, 0x)
 Name (ROMS, 0xFFE0)
 Name (VGAF, One)
-OperationRegion (GNVS, SystemMemory, 0x8C50C000, 0x07C0)
+OperationRegion (GNVS, SystemMemory, 0x8C501000, 0x07C0)
 Field (GNVS, AnyAcc, Lock, Preserve)
 {
 OSYS,   16, 
 SMIF,   8, 
@@ -12121,9 +12121,9 @@
 Notify (\_SB.PCI0.GFX0.DD1F, Arg0)
 }
 }
 
-Name (PNVB, 0x8C53F018)
+Name (PNVB, 0x8C534018)
 Name (PNVL, 0x0287)
 If (ECR1 == One)
 {
 Scope (_SB.PCI0)
--- glutamine/ssdt2.dsl 2021-02-25 11:41:37.0 +0100
+++ leucine/ssdt2.dsl   2021-02-25 11:41:49.0 +0100
@@ -4,15 +4,15 @@
  * Copyright (c) 2000 - 2016 Intel Corporation
  * 
  * Disassembling to symbolic ASL+ operators
  *
- * Disassembly of ssdt2.dat, Thu Feb 25 10:41:12 2021
+ * Disassembly of ssdt2.dat, Thu Feb 25 11:40:56 2021
  *
  * Original Table Header:
  * Signature"SSDT"
  * Length   0x3165 (12645)
  * Revision 0x02
- * Checksum 0x2D
+ * Checksum 0xDD
  * OEM ID   "SaSsdt"
  * OEM Table ID "SaSsdt "
  * OEM Revision 0x3000 (12288)
  * Compiler ID  "INTL"
@@ -60,9 +60,9 @@
 External (OSYS, IntObj)
 External (PNHM, IntObj)
 External (S0ID, UnknownObj)
 
-OperationRegion (SANV, SystemMemory, 0x8C53F358, 0x01F7)
+OperationRegion (SANV, SystemMemory, 0x8C534358, 0x01F7)
 Field (SANV, AnyAcc, Lock, Preserve)
 {
 ASLB,   32, 
 IMON,   8, 
--- glutamine/ssdt7.dsl 2021-02-25 11:41:38.0 +0100
+++ leucine/ssdt7.dsl   2021-02-25 11:41:50.0 +0100
@@ -4,15 +4,15 @@
  * Copyright (c) 2000 - 2016 Intel Corporation
  * 
  * Disassembling to symbolic ASL+ operators
  *
- * Disassembly of ssdt7.dat, Thu Feb 25 10:41:12 2021
+ * Disassembly of ssdt7.dat, Thu Feb 25 11:40:56 2021
  *
  * Original Table Header:
  * Signature"SSDT"
  * Length   0x17AE (6062)
  * Revision 0x02
- * Checksum 0x7B
+ * Checksum 0xDE
  * OEM ID   "CpuRef"
  * OEM Table ID "CpuSsdt"
  * OEM Revision 0x3000 (12288)
  * Compiler ID  "INTL"
@@ -43,27 +43,27 @@
 {
 Name (SSDT, Package (0x15)
 {
 "CPU0IST ", 
-0x8BCD8398, 
+0x8BCCD398, 
 0x0651, 
 "APIST   ", 
-0x8BC7F018, 
+0x8BC74018, 
 0x0D14, 
 "CPU0CST ", 
-0x8BC80898, 
+0x8BC75898, 
 0x03FF, 
 "APCST   ", 
-0x8BC7E018, 
+0x8BC73018, 
 0x030A, 
 "CPU0HWP ", 
-0x8BCD8C18, 
+0x8BCCDC18, 
 0x00BA, 
 "APHWP   ", 
-0x8BC7EBD8, 
+0x8BC73BD8, 
 0x0317, 
 "HWPLVT  ", 
-0x8BC80018, 
+0x8BC75018, 
 0x0628
 })
 Name (\PC00, 0x8000)
 Name (\PC01, 0x8000)
@@ -86,9 +86,9 @@
 
 Scope (\_PR)
 {
 Name (CTPC, Zero)
-OperationRegion (PNVS, SystemMemory, 0x756AD000, 0x006C)
+OperationRegion (PNVS, SystemMemory, 0x7B654000, 0x006C)
 Field (PNVS, AnyAcc, Lock, Preserve)
 {
 PGRV,   8, 
 CFGD,   32, 


-- 
Emmanuel Dreyfus