Re: ACPI related performance trouble
Brian Buhrow wrote: > hello. I wonder if you've compared your BIOS settings on both > machines? While the BIOS may be the same version, it's possible the > settings are not identical. this is strongly suggested by the fact that > one of your machines shows a serial number in its machdep.dmi output, > while the other does not. I double checked. Both machine report it, but the information is only shown to root. I must have run the sysctl command as an unprivilegied user on the slow machine. > Doing that comparison will be tedious, but I think worth it. Indeed; In an ideal world, there would be a way to get a MIB of BIOS settings. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org
Re: ACPI related performance trouble
hello. I wonder if you've compared your BIOS settings on both machines? While the BIOS may be the same version, it's possible the settings are not identical. this is strongly suggested by the fact that one of your machines shows a serial number in its machdep.dmi output, while the other does not. Doing that comparison will be tedious, but I think worth it. Perhaps the easiest way is to reset the slow machine to factory defaults, then do your comparison, screen by screen with the fast machine. -Brian On Feb 25, 2:38pm, Emmanuel Dreyfus wrote: } Subject: Re: ACPI related performance trouble } Joerg Sonnenberger wrote: } } > Have you compared the machdep sysctl? } } Here it is. } } --- sysctl.glutamine } +++ sysctl.leucine
Re: ACPI related performance trouble
Michael van Elst wrote: > As for performance, the common issue was that the ACPI interrupt > isn't handled and you get several thousand interrupts per second > that slow down everything. You spot it: glutamine# vmstat -iv interrupt total rate TLB shootdown 135950 cpu0 timer3312564 99 ioapic0 pin 9 282417042 8525 msi0 vec 03200 msi1 vec 0 213150 msi2 vec 0 00 msix3 vec 0 00 msi4 vec 0 00 ioapic0 pin 16 00 msi5 vec 0 592771 ioapic0 pin 4 242470 Total 285848360 8628 -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org
Re: ACPI related performance trouble
m...@netbsd.org (Emmanuel Dreyfus) writes: >Joerg Sonnenberger wrote: >> Have you compared the machdep sysctl? >Here it is. >+machdep.dmi.system-serial = 0123456789 >+machdep.dmi.system-uuid = ----ac1f6b747c48 The serial numbers are only shown to root. As for performance, the common issue was that the ACPI interrupt isn't handled and you get several thousand interrupts per second that slow down everything. E.g. (For a good case): acpi0: SCI interrupting at int 9 % vmstat -iv TLB shootdown6572617 16 cpu0 timer 41040041 99 ioapic0 pin 9 00 <- msix0 vec 000 msix0 vec 1 31115987 msix0 vec 2 9428662 msix0 vec 3 2513640 msix0 vec 4 1872870 msi1 vec 0 00 msi2 vec 0 39273009 ioapic0 pin 16230 msix3 vec 000 msix3 vec 100 msix3 vec 200 msix3 vec 300 msix3 vec 400 ioapic0 pin 23260 msi4 vec 0 6111286 14 ioapic0 pin 18 00 ioapic0 pin 4 19990 Total 62146407 151 % intrctl list interrupt id CPU0CPU1CPU2CPU3 device name(s) ioapic0 pin 90* 0 0 0 acpi SCI <--- msix0 vec 0 0* 0 0 0 nvme0 adminq msix0 vec 13111598* 0 0 0 nvme0 ioq1 msix0 vec 2 0 942866* 0 0 nvme0 ioq2 msix0 vec 3 0 0 251364* 0 nvme0 ioq3 msix0 vec 4 0 0 0 187287* nvme0 ioq4 msi1 vec 0 0* 0 0 0 xhci0 msi2 vec 0 3927265* 0 0 0 wm0 ioapic0 pin 16 23* 0 0 0 ehci0, i915drmkms0 msix3 vec 0 0* 0 0 0 wm1TXRX0 msix3 vec 1 0 0* 0 0 wm1TXRX1 msix3 vec 2 0 0 0* 0 wm1TXRX2 msix3 vec 3 0 0 0 0* wm1TXRX3 msix3 vec 4 0* 0 0 0 wm1LINK ioapic0 pin 23 26* 0 0 0 ehci1 msi4 vec 0 6111264* 0 0 0 ahcisata0 ioapic0 pin 18 0* 0 0 0 ichsmb0 ioapic0 pin 4 1999* 0 0 0 com0 -- -- Michael van Elst Internet: mlel...@serpens.de "A potential Snark may lurk in every tree."
Re: ACPI related performance trouble
Joerg Sonnenberger wrote: > Have you compared the machdep sysctl? Here it is. --- sysctl.glutamine +++ sysctl.leucine @@ -660,9 +660,9 @@ hw.wd1.use_ncq_prio = 0 machdep.biosbasemem = 611 machdep.biosextmem = 1047552 machdep.booted_kernel = /netbsd -machdep.diskinfo: 80:30031872(845/255/63),2 81:1953525168(1023/255/63),2 82:195 3525168(1023/255/63),2 dk0 dk1 dk2 dk3 wd0:81 wd1:82 sd0:80 +machdep.diskinfo: 80:30031872(845/255/63),2 81:1953525168(1023/255/63),2 82:195 3525168(1023/255/63),2 dk0 dk1 dk2 dk3 wd0:81,82 wd1:81,82 sd0:80 machdep.fpu_present = 1 machdep.osfxsr = 1 machdep.sse = 1 machdep.sse2 = 1 @@ -671,17 +671,21 @@ machdep.xsave_features = 31 machdep.dmi.system-vendor = Supermicro machdep.dmi.system-product = C7Z370-CG-L machdep.dmi.system-version = 0123456789 +machdep.dmi.system-serial = 0123456789 +machdep.dmi.system-uuid = ----ac1f6b747c48 machdep.dmi.bios-vendor = American Megatrends Inc. machdep.dmi.bios-version = 1.2c machdep.dmi.bios-date = 20190617 machdep.dmi.board-vendor = Supermicro machdep.dmi.board-product = C7Z370-CG-L machdep.dmi.board-version = 1.01 +machdep.dmi.board-serial = ZM188S015949 machdep.dmi.board-asset-tag = Default string machdep.dmi.chassis-vendor = Supermicro machdep.dmi.chassis-version = 0123456789 +machdep.dmi.chassis-serial = 0123456789 machdep.dmi.chassis-asset-tag = Default string machdep.dmi.processor-vendor = Intel(R) Corporation machdep.dmi.processor-version = Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz machdep.dmi.processor-frequency = 2800 MHz -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org
Re: ACPI related performance trouble
On Thu, Feb 25, 2021 at 11:46:21AM +0100, Emmanuel Dreyfus wrote: > I just got two identical machines, let us call them glutamine and leucine. I > run ffmpeg4 to transcode H264 video to webm, and leucine is about 12 times > faster than glutamine. Have you compared the machdep sysctl? Joerg
ACPI related performance trouble
Hello I just got two identical machines, let us call them glutamine and leucine. I run ffmpeg4 to transcode H264 video to webm, and leucine is about 12 times faster than glutamine. The hardware and software are the same, the BIOS revision are identical. The only difference dmesg tells me about beyond ethernet MAC addresses are the ACPI table addresses, which are a bit different. Following the hint about ACPI, I tried booting glutamine with option -2 (disable ACPI). That boosts its performance on par with leucine. I can live without ACPI, but I still wonder what could cause the problem. Anyone has an idea? I compared acpidump -d -outputs on the two machines. The only differences are in dsdt, ssdt2 and ssdt7 and even there, they are scarce. It looks like if it only about addresses. --- glutamine/dsdt.dsl 2021-02-25 11:41:36.0 +0100 +++ leucine/dsdt.dsl2021-02-25 11:41:48.0 +0100 @@ -4,15 +4,15 @@ * Copyright (c) 2000 - 2016 Intel Corporation * * Disassembling to symbolic ASL+ operators * - * Disassembly of dsdt.dat, Thu Feb 25 10:41:12 2021 + * Disassembly of dsdt.dat, Thu Feb 25 11:40:56 2021 * * Original Table Header: * Signature"DSDT" * Length 0x00029C56 (171094) * Revision 0x02 - * Checksum 0x89 + * Checksum 0xE9 * OEM ID "SUPERM" * OEM Table ID "SMCI--MB" * OEM Revision 0x01072009 (17244169) * Compiler ID "INTL" @@ -278,9 +278,9 @@ Name (IOST, 0x4001) Name (TOPM, 0x) Name (ROMS, 0xFFE0) Name (VGAF, One) -OperationRegion (GNVS, SystemMemory, 0x8C50C000, 0x07C0) +OperationRegion (GNVS, SystemMemory, 0x8C501000, 0x07C0) Field (GNVS, AnyAcc, Lock, Preserve) { OSYS, 16, SMIF, 8, @@ -12121,9 +12121,9 @@ Notify (\_SB.PCI0.GFX0.DD1F, Arg0) } } -Name (PNVB, 0x8C53F018) +Name (PNVB, 0x8C534018) Name (PNVL, 0x0287) If (ECR1 == One) { Scope (_SB.PCI0) --- glutamine/ssdt2.dsl 2021-02-25 11:41:37.0 +0100 +++ leucine/ssdt2.dsl 2021-02-25 11:41:49.0 +0100 @@ -4,15 +4,15 @@ * Copyright (c) 2000 - 2016 Intel Corporation * * Disassembling to symbolic ASL+ operators * - * Disassembly of ssdt2.dat, Thu Feb 25 10:41:12 2021 + * Disassembly of ssdt2.dat, Thu Feb 25 11:40:56 2021 * * Original Table Header: * Signature"SSDT" * Length 0x3165 (12645) * Revision 0x02 - * Checksum 0x2D + * Checksum 0xDD * OEM ID "SaSsdt" * OEM Table ID "SaSsdt " * OEM Revision 0x3000 (12288) * Compiler ID "INTL" @@ -60,9 +60,9 @@ External (OSYS, IntObj) External (PNHM, IntObj) External (S0ID, UnknownObj) -OperationRegion (SANV, SystemMemory, 0x8C53F358, 0x01F7) +OperationRegion (SANV, SystemMemory, 0x8C534358, 0x01F7) Field (SANV, AnyAcc, Lock, Preserve) { ASLB, 32, IMON, 8, --- glutamine/ssdt7.dsl 2021-02-25 11:41:38.0 +0100 +++ leucine/ssdt7.dsl 2021-02-25 11:41:50.0 +0100 @@ -4,15 +4,15 @@ * Copyright (c) 2000 - 2016 Intel Corporation * * Disassembling to symbolic ASL+ operators * - * Disassembly of ssdt7.dat, Thu Feb 25 10:41:12 2021 + * Disassembly of ssdt7.dat, Thu Feb 25 11:40:56 2021 * * Original Table Header: * Signature"SSDT" * Length 0x17AE (6062) * Revision 0x02 - * Checksum 0x7B + * Checksum 0xDE * OEM ID "CpuRef" * OEM Table ID "CpuSsdt" * OEM Revision 0x3000 (12288) * Compiler ID "INTL" @@ -43,27 +43,27 @@ { Name (SSDT, Package (0x15) { "CPU0IST ", -0x8BCD8398, +0x8BCCD398, 0x0651, "APIST ", -0x8BC7F018, +0x8BC74018, 0x0D14, "CPU0CST ", -0x8BC80898, +0x8BC75898, 0x03FF, "APCST ", -0x8BC7E018, +0x8BC73018, 0x030A, "CPU0HWP ", -0x8BCD8C18, +0x8BCCDC18, 0x00BA, "APHWP ", -0x8BC7EBD8, +0x8BC73BD8, 0x0317, "HWPLVT ", -0x8BC80018, +0x8BC75018, 0x0628 }) Name (\PC00, 0x8000) Name (\PC01, 0x8000) @@ -86,9 +86,9 @@ Scope (\_PR) { Name (CTPC, Zero) -OperationRegion (PNVS, SystemMemory, 0x756AD000, 0x006C) +OperationRegion (PNVS, SystemMemory, 0x7B654000, 0x006C) Field (PNVS, AnyAcc, Lock, Preserve) { PGRV, 8, CFGD, 32, -- Emmanuel Dreyfus