Re: Dell R630 high interrupts on acpi0

2014-12-20 Thread Jonathan Matthew 
On Fri, Dec 19, 2014 at 02:58:13PM +0100, Mark Kettenis wrote:
  Date: Fri, 19 Dec 2014 23:41:56 +1000
  From: Jonathan Matthew jonat...@d14n.org
  
  On Thu, Dec 18, 2014 at 11:14:54PM +0100, Frederic Nowak wrote:
   Hi!
   
   The diff for extracting memory ranges from ACPI was obviously not tested
   with ACPI disabled...so we definitely have to check if the values in
   pcimem_range make some sense. The diff below now uses the old values in
   case ACPI is disabled or does not return valid values.
   
   I hope this is somewhat interesting for someone else as well :)
   Please let me know if this is just noise or if there is anything else I
   am missing.
  
  This looks like the right direction, except I think we only really want to 
  use
  the ACPI range information to add new ranges to the existing one that covers
  the 36-bit address space, rather than relying on it for everything.  No 
  point
  relying on the ACPI stuff being correct if we don't need to.
  
  I hacked this around a bit today and ended up with the diff below.  This 
  only
  deals with ranges outside the 36-bit space, so it makes no difference on 
  most
  machines, and on the r630 that needs it, it adds two ranges,
  0x0300 to 0x033F and 0x0340 to
  0x037F.
 
 This is going in the right direction.  The way I designed things
 though was that the acpi code would build up the complete extent
 purely from information provided by _CRS, and set pcimem_ex before
 pci_init_extents() gets called.

Ah, OK, I'll make it do that instead.

I probably won't be able to work on this for the next week or so, but I'll pick
it up again after that.

Re: Dell R630 high interrupts on acpi0

2014-12-19 Thread Jonathan Matthew 
On Thu, Dec 18, 2014 at 11:14:54PM +0100, Frederic Nowak wrote:
 Hi!
 
 The diff for extracting memory ranges from ACPI was obviously not tested
 with ACPI disabled...so we definitely have to check if the values in
 pcimem_range make some sense. The diff below now uses the old values in
 case ACPI is disabled or does not return valid values.
 
 I hope this is somewhat interesting for someone else as well :)
 Please let me know if this is just noise or if there is anything else I
 am missing.

This looks like the right direction, except I think we only really want to use
the ACPI range information to add new ranges to the existing one that covers
the 36-bit address space, rather than relying on it for everything.  No point
relying on the ACPI stuff being correct if we don't need to.

I hacked this around a bit today and ended up with the diff below.  This only
deals with ranges outside the 36-bit space, so it makes no difference on most
machines, and on the r630 that needs it, it adds two ranges,
0x0300 to 0x033F and 0x0340 to
0x037F.


Index: arch/amd64/pci/pci_machdep.c
===
RCS file: /cvs/src/sys/arch/amd64/pci/pci_machdep.c,v
retrieving revision 1.60
diff -u -p -r1.60 pci_machdep.c
--- arch/amd64/pci/pci_machdep.c16 Dec 2014 23:13:20 -  1.60
+++ arch/amd64/pci/pci_machdep.c19 Dec 2014 12:54:44 -
@@ -90,12 +90,17 @@
 #include dev/pci/ppbreg.h
 
 #include ioapic.h
+#include acpi.h
 
 #if NIOAPIC  0
 #include machine/i82093var.h
 #include machine/mpbiosvar.h
 #endif
 
+#if NACPI  0
+#include dev/acpi/acpivar.h
+#endif
+
 /*
  * Memory Mapped Configuration space access.
  *
@@ -622,17 +627,13 @@ pci_init_extents(void)
 * here.  As long as vendors continue to support
 * 32-bit operating systems, we should never see BARs
 * outside that region.
-*
-* Dell 13G servers have important devices outside the
-* 36-bit address space.  Until we can extract the address
-* ranges from ACPI, expand the allowed range to suit.
 */
pcimem_ex = extent_create(pcimem, 0, 0xUL,
M_DEVBUF, NULL, 0, EX_NOWAIT);
if (pcimem_ex == NULL)
return;
-   extent_alloc_region(pcimem_ex, 0x400UL,
-   0xfc00UL, EX_NOWAIT);
+   extent_alloc_region(pcimem_ex, 0x10UL,
+   0xfff0UL, EX_NOWAIT);
 
for (bmp = bios_memmap; bmp-type != BIOS_MAP_END; bmp++) {
/*
@@ -657,6 +658,14 @@ pci_init_extents(void)
/* Take out the video buffer area and BIOS areas. */
extent_alloc_region(pcimem_ex, IOM_BEGIN, IOM_SIZE,
EX_CONFLICTOK | EX_NOWAIT);
+
+#if NACPI  0
+   /*
+* Free up any regions outside the 36-bit address space
+* specified via ACPI.
+*/
+   acpi_pciroots_extents(pcimem_ex, 0x10UL);
+#endif
}
 
if (pcibus_ex == NULL) {
@@ -665,7 +674,6 @@ pci_init_extents(void)
}
 }
 
-#include acpi.h
 #if NACPI  0
 void acpi_pci_match(struct device *, struct pci_attach_args *);
 pcireg_t acpi_pci_min_powerstate(pci_chipset_tag_t, pcitag_t);
Index: dev/acpi/acpivar.h
===
RCS file: /cvs/src/sys/dev/acpi/acpivar.h,v
retrieving revision 1.79
diff -u -p -r1.79 acpivar.h
--- dev/acpi/acpivar.h  8 Dec 2014 07:12:37 -   1.79
+++ dev/acpi/acpivar.h  19 Dec 2014 12:54:44 -
@@ -47,6 +47,7 @@ extern u_int8_t acpi_lapic_flags[LAPIC_M
 struct klist;
 struct acpiec_softc;
 struct acpipwrres_softc;
+struct extent;
 
 struct acpivideo_softc {
struct device sc_dev;
@@ -357,6 +358,7 @@ int acpi_acquire_glk(uint32_t *);
 intacpi_release_glk(uint32_t *);
 
 void   acpi_pciroots_attach(struct device *, void *, cfprint_t);
+void   acpi_pciroots_extents(struct extent *, u_int64_t);
 
 #endif
 
Index: dev/acpi/amltypes.h
===
RCS file: /cvs/src/sys/dev/acpi/amltypes.h,v
retrieving revision 1.40
diff -u -p -r1.40 amltypes.h
--- dev/acpi/amltypes.h 7 Sep 2012 19:19:59 -   1.40
+++ dev/acpi/amltypes.h 19 Dec 2014 12:54:44 -
@@ -346,6 +346,12 @@ struct aml_value {
 #define aml_pkglen(v)  ((v)-length)
 #define aml_pkgval(v,i)((v)-v_package[(i)])
 
+struct acpi_pci_range {
+   SLIST_ENTRY(acpi_pci_range) next;
+   uint64_tbase;
+   uint64_tlen;
+};
+
 struct acpi_pci {
TAILQ_ENTRY(acpi_pci)   next;
 
@@ -362,6 +368,8 @@ struct acpi_pci {
int _s3w;
int

Re: Dell R630 high interrupts on acpi0

2014-12-19 Thread Mark Kettenis 
 Date: Fri, 19 Dec 2014 23:41:56 +1000
 From: Jonathan Matthew jonat...@d14n.org
 
 On Thu, Dec 18, 2014 at 11:14:54PM +0100, Frederic Nowak wrote:
  Hi!
  
  The diff for extracting memory ranges from ACPI was obviously not tested
  with ACPI disabled...so we definitely have to check if the values in
  pcimem_range make some sense. The diff below now uses the old values in
  case ACPI is disabled or does not return valid values.
  
  I hope this is somewhat interesting for someone else as well :)
  Please let me know if this is just noise or if there is anything else I
  am missing.
 
 This looks like the right direction, except I think we only really want to use
 the ACPI range information to add new ranges to the existing one that covers
 the 36-bit address space, rather than relying on it for everything.  No point
 relying on the ACPI stuff being correct if we don't need to.
 
 I hacked this around a bit today and ended up with the diff below.  This only
 deals with ranges outside the 36-bit space, so it makes no difference on most
 machines, and on the r630 that needs it, it adds two ranges,
 0x0300 to 0x033F and 0x0340 to
 0x037F.

This is going in the right direction.  The way I designed things
though was that the acpi code would build up the complete extent
purely from information provided by _CRS, and set pcimem_ex before
pci_init_extents() gets called.

 Index: arch/amd64/pci/pci_machdep.c
 ===
 RCS file: /cvs/src/sys/arch/amd64/pci/pci_machdep.c,v
 retrieving revision 1.60
 diff -u -p -r1.60 pci_machdep.c
 --- arch/amd64/pci/pci_machdep.c  16 Dec 2014 23:13:20 -  1.60
 +++ arch/amd64/pci/pci_machdep.c  19 Dec 2014 12:54:44 -
 @@ -90,12 +90,17 @@
  #include dev/pci/ppbreg.h
  
  #include ioapic.h
 +#include acpi.h
  
  #if NIOAPIC  0
  #include machine/i82093var.h
  #include machine/mpbiosvar.h
  #endif
  
 +#if NACPI  0
 +#include dev/acpi/acpivar.h
 +#endif
 +
  /*
   * Memory Mapped Configuration space access.
   *
 @@ -622,17 +627,13 @@ pci_init_extents(void)
* here.  As long as vendors continue to support
* 32-bit operating systems, we should never see BARs
* outside that region.
 -  *
 -  * Dell 13G servers have important devices outside the
 -  * 36-bit address space.  Until we can extract the address
 -  * ranges from ACPI, expand the allowed range to suit.
*/
   pcimem_ex = extent_create(pcimem, 0, 0xUL,
   M_DEVBUF, NULL, 0, EX_NOWAIT);
   if (pcimem_ex == NULL)
   return;
 - extent_alloc_region(pcimem_ex, 0x400UL,
 - 0xfc00UL, EX_NOWAIT);
 + extent_alloc_region(pcimem_ex, 0x10UL,
 + 0xfff0UL, EX_NOWAIT);
  
   for (bmp = bios_memmap; bmp-type != BIOS_MAP_END; bmp++) {
   /*
 @@ -657,6 +658,14 @@ pci_init_extents(void)
   /* Take out the video buffer area and BIOS areas. */
   extent_alloc_region(pcimem_ex, IOM_BEGIN, IOM_SIZE,
   EX_CONFLICTOK | EX_NOWAIT);
 +
 +#if NACPI  0
 + /*
 +  * Free up any regions outside the 36-bit address space
 +  * specified via ACPI.
 +  */
 + acpi_pciroots_extents(pcimem_ex, 0x10UL);
 +#endif
   }
  
   if (pcibus_ex == NULL) {
 @@ -665,7 +674,6 @@ pci_init_extents(void)
   }
  }
  
 -#include acpi.h
  #if NACPI  0
  void acpi_pci_match(struct device *, struct pci_attach_args *);
  pcireg_t acpi_pci_min_powerstate(pci_chipset_tag_t, pcitag_t);
 Index: dev/acpi/acpivar.h
 ===
 RCS file: /cvs/src/sys/dev/acpi/acpivar.h,v
 retrieving revision 1.79
 diff -u -p -r1.79 acpivar.h
 --- dev/acpi/acpivar.h8 Dec 2014 07:12:37 -   1.79
 +++ dev/acpi/acpivar.h19 Dec 2014 12:54:44 -
 @@ -47,6 +47,7 @@ extern u_int8_t acpi_lapic_flags[LAPIC_M
  struct klist;
  struct acpiec_softc;
  struct acpipwrres_softc;
 +struct extent;
  
  struct acpivideo_softc {
   struct device sc_dev;
 @@ -357,6 +358,7 @@ int   acpi_acquire_glk(uint32_t *);
  int  acpi_release_glk(uint32_t *);
  
  void acpi_pciroots_attach(struct device *, void *, cfprint_t);
 +void acpi_pciroots_extents(struct extent *, u_int64_t);
  
  #endif
  
 Index: dev/acpi/amltypes.h
 ===
 RCS file: /cvs/src/sys/dev/acpi/amltypes.h,v
 retrieving revision 1.40
 diff -u -p -r1.40 amltypes.h
 --- dev/acpi/amltypes.h   7 Sep 2012 19:19:59 -   1.40
 +++ dev/acpi/amltypes.h   19 Dec 2014 12:54:44 -
 @@ -346,6 +346,12 @@ struct aml_value {
  #define aml_pkglen(v)

Re: Dell R630 high interrupts on acpi0

2014-12-19 Thread Hrvoje Popovski 

On 17.12.2014. 6:34, Philip Guenther wrote:

Uh, ACPI *requires* that C1 exist.  The halt instruction is defined as
entering C1, so not having C1 would mean your CPU lacks a basic
manadatory ia32 instruction.  Hopefully the BIOS docs explain that
you're just disabling deep C-states or something like that.  If not,
yell at the company that made it.

With the exception of C1E, I wouldn't tell a BIOS to disable
C-states unless it was causing the OS to have a problem or you're
actively trying to use the computer to heat your house.  C1E is a
cross between C1 and C3; the issue is that bugs in multiple early
hardware implementations mean it'll behave poorly depending on exactly
how the OS handles it.  This is something to test...and then test
again with each release you install...


Thank you, this is very informative.
I have disabled C states because I have seen ifq.drops and netlivelocks
when I run udp tcpbench over ix interface and C-states are enabled. This 
is between two Dell R620. With Dell R630 I have bigger problems so i 
can't test them right now.


tcpbench client sends around 350kpps for 20s
tcpbench server gets around 212kpps

tcpbench udp server drops:
Without C states and C1E:
net.inet.ip.ifq.maxlen=2048
net.inet.ip.ifq.drops=0
kern.netlivelocks=3

With C states and C1E (DAPC):
net.inet.ip.ifq.maxlen=2048
net.inet.ip.ifq.drops=19143
kern.netlivelocks=86

so I explained to myself that C states are bad for packet bursts and 
these boxes will be uber high speed firewalls :)


and of course after reading you mail I only disable C1E and everything 
is fine.


With C states and without C1E
net.inet.ip.ifq.maxlen=2048
net.inet.ip.ifq.drops=0
kern.netlivelocks=3


Monitor/Mwait - Disabled


I would suggest leaving that on.  We ain't using it *right now*, but,
well, the source tree on my laptop is, and more than ever.  :-)



mwait is enabled :)

Re: Dell R630 high interrupts on acpi0

2014-12-18 Thread Frederic Nowak 
Hi!

The diff for extracting memory ranges from ACPI was obviously not tested
with ACPI disabled...so we definitely have to check if the values in
pcimem_range make some sense. The diff below now uses the old values in
case ACPI is disabled or does not return valid values.

I hope this is somewhat interesting for someone else as well :)
Please let me know if this is just noise or if there is anything else I
am missing.


Index: sys/arch/amd64/include/pci_machdep.h
===
RCS file: /cvs/src/sys/arch/amd64/include/pci_machdep.h,v
retrieving revision 1.22
diff -u -p -b -r1.22 pci_machdep.h
--- sys/arch/amd64/include/pci_machdep.h6 Nov 2013 10:40:36 -   
1.22
+++ sys/arch/amd64/include/pci_machdep.h17 Dec 2014 06:00:01 -
@@ -64,6 +64,7 @@ extern int pci_mcfg_min_bus, pci_mcfg_ma
 
 struct pci_attach_args;
 
+extern uint64_t pcimem_range[2];
 extern struct extent *pciio_ex;
 extern struct extent *pcimem_ex;
 extern struct extent *pcibus_ex;
Index: sys/arch/amd64/pci/pci_machdep.c
===
RCS file: /cvs/src/sys/arch/amd64/pci/pci_machdep.c,v
retrieving revision 1.60
diff -u -p -b -r1.60 pci_machdep.c
--- sys/arch/amd64/pci/pci_machdep.c16 Dec 2014 23:13:20 -  1.60
+++ sys/arch/amd64/pci/pci_machdep.c18 Dec 2014 22:01:29 -
@@ -66,6 +66,7 @@
  */
 
 #include sys/types.h
+#include sys/stdint.h
 #include sys/param.h
 #include sys/time.h
 #include sys/systm.h
@@ -592,6 +593,7 @@ pci_intr_disestablish(pci_chipset_tag_t 
intr_disestablish(cookie);
 }
 
+uint64_t pcimem_range[2] = {UINT64_MAX, 0};
 struct extent *pciio_ex;
 struct extent *pcimem_ex;
 struct extent *pcibus_ex;
@@ -618,31 +620,32 @@ pci_init_extents(void)
 
if (pcimem_ex == NULL) {
/*
-* Cover the 36-bit address space addressable by PAE
-* here.  As long as vendors continue to support
-* 32-bit operating systems, we should never see BARs
-* outside that region.
-*
-* Dell 13G servers have important devices outside the
-* 36-bit address space.  Until we can extract the address
-* ranges from ACPI, expand the allowed range to suit.
+* Cover the address space extracted from
+* ACPI. Fallback to fixed ranges in case ACPI is
+* disabled or reporting an invalid address range.
 */
+   if(pcimem_range[0] = pcimem_range[1]) {
+   pcimem_range[0] = 0;
+   pcimem_range[1] = 0x400UL;
+   }
+
pcimem_ex = extent_create(pcimem, 0, 0xUL,
M_DEVBUF, NULL, 0, EX_NOWAIT);
if (pcimem_ex == NULL)
return;
-   extent_alloc_region(pcimem_ex, 0x400UL,
-   0xfc00UL, EX_NOWAIT);
+   extent_alloc_region(pcimem_ex, pcimem_range[1] + 1,
+   (0xUL - pcimem_range[1]), EX_NOWAIT);
 
for (bmp = bios_memmap; bmp-type != BIOS_MAP_END; bmp++) {
/*
-* Ignore address space beyond 4G.
+* Ignore address space beyond fixed/extracted
+* address range.
 */
-   if (bmp-addr = 0x1ULL)
+   if (bmp-addr  pcimem_range[1])
continue;
size = bmp-size;
-   if (bmp-addr + size = 0x1ULL)
-   size = 0x1ULL - bmp-addr;
+   if (bmp-addr + size  pcimem_range[1])
+   size = pcimem_range[1] - bmp-addr + 1;
 
/* Ignore zero-sized regions. */
if (size == 0)
Index: sys/dev/acpi/acpi.c
===
RCS file: /cvs/src/sys/dev/acpi/acpi.c,v
retrieving revision 1.277
diff -u -p -b -r1.277 acpi.c
--- sys/dev/acpi/acpi.c 9 Dec 2014 06:58:29 -   1.277
+++ sys/dev/acpi/acpi.c 17 Dec 2014 05:59:36 -
@@ -385,0 +385,0 @@ TAILQ_HEAD(, acpi_pci) acpi_pcirootdevs

 int acpi_getpci(struct aml_node *node, void *arg);
 int acpi_getminbus(union acpi_resource *crs, void *arg);
+int acpi_getmemrange(union acpi_resource *crs, void *arg);
+
+int
+acpi_getmemrange(union acpi_resource *crs, void *arg)
+{
+   int typ = AML_CRSTYPE(crs);
+   uint64_t *range = arg;  /* size 2 */
+   uint64_t min, max;
+
+   switch(typ) {
+   case LR_24BIT:
+   min = crs-lr_m24._min;
+   max = crs-lr_m24._max;
+   break;
+   case LR_32BIT:
+   case LR_32BITFIXED:
+   min =

Re: Dell R630 high interrupts on acpi0

2014-12-17 Thread mxb 

 On 16 dec 2014, at 06:40, David Gwynne da...@gwynne.id.au wrote:
 
 others have hit this on r620s as well

I don’t see it on mine.

interrupt   total rate
irq0/clock 9587998940 1599
irq0/ipi136166514   22
irq144/acpi020
irq112/ix029053603446 4847
irq113/ix127844456217 4646
irq96/mfi080725871
irq114/ubsec0  3101629892  517
irq98/ehci0   1120
irq115/em0 4928262870  822
irq116/em1  211437268   35
irq99/ehci1280
irq100/ahci010
Total 7487162787712493


This is a pre-5.6

OpenBSD 5.6-current (GENERIC.MP) #394: Wed Oct  1 12:54:54 MDT 2014
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8492285952 (8098MB)
avail mem = 8257511424 (7874MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xcf42c000 (99 entries)
bios0: vendor Dell Inc. version 1.3.6 date 09/11/2012
bios0: Dell Inc. PowerEdge R620
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC SPCR HPET DMAR MCFG WD__ SLIC ERST HEST BERT EINJ 
TCPA PC__ SRAT SSDT
acpi0: wakeup devices PCI0(S5) PCI1(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.43 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1.2, IBE
cpu1 at mainbus0: apid 32 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 0, package 1
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
cpu2: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 34 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
cpu3: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 1, package 1
cpu4 at mainbus0: apid 4 (application processor)
cpu4: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
cpu4: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 0, core 2, package 0
cpu5 at mainbus0: apid 36 (application processor)
cpu5: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
cpu5: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 0, core 2, package 1
cpu6 at mainbus0: apid 6 (application processor)
cpu6: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
cpu6: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 0, core 3, package 0
cpu7 at mainbus0: apid 38 (application processor)
cpu7: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz,

Re: Dell R630 high interrupts on acpi0

2014-12-17 Thread Mark Kettenis 
 Date: Tue, 16 Dec 2014 21:34:29 -0800
 From: Philip Guenther guent...@gmail.com
 
  R620 have similar settings and can't see C states in dmesg
  acpicpu0 at acpi0
 
 That's either insane, or a bug in our acpicpu code, IMO.

Probably just the effect of the BIOS not advertising C states at all.
Like you said, C1 support is mandatory, so there is no real need to
advertise it unless some of the deeper states are advertised as well.

Re: Dell R630 high interrupts on acpi0

2014-12-17 Thread Mike Larkin 
On Wed, Dec 17, 2014 at 10:44:24AM +0100, mxb wrote:
 
  On 16 dec 2014, at 06:40, David Gwynne da...@gwynne.id.au wrote:
  
  others have hit this on r620s as well
 
 I don???t see it on mine.

Others have hit a similar symptom on r620, caused by a different issue
that was already fixed by kettenis a few months ago.

-ml

 
 interrupt   total rate
 irq0/clock 9587998940 1599
 irq0/ipi136166514   22
 irq144/acpi020
 irq112/ix029053603446 4847
 irq113/ix127844456217 4646
 irq96/mfi080725871
 irq114/ubsec0  3101629892  517
 irq98/ehci0   1120
 irq115/em0 4928262870  822
 irq116/em1  211437268   35
 irq99/ehci1280
 irq100/ahci010
 Total 7487162787712493
 
 
 This is a pre-5.6
 
 OpenBSD 5.6-current (GENERIC.MP) #394: Wed Oct  1 12:54:54 MDT 2014
 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
 real mem = 8492285952 (8098MB)
 avail mem = 8257511424 (7874MB)
 mpath0 at root
 scsibus0 at mpath0: 256 targets
 mainbus0 at root
 bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xcf42c000 (99 entries)
 bios0: vendor Dell Inc. version 1.3.6 date 09/11/2012
 bios0: Dell Inc. PowerEdge R620
 acpi0 at bios0: rev 2
 acpi0: sleep states S0 S4 S5
 acpi0: tables DSDT FACP APIC SPCR HPET DMAR MCFG WD__ SLIC ERST HEST BERT 
 EINJ TCPA PC__ SRAT SSDT
 acpi0: wakeup devices PCI0(S5) PCI1(S5)
 acpitimer0 at acpi0: 3579545 Hz, 24 bits
 acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
 cpu0 at mainbus0: apid 0 (boot processor)
 cpu0: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.43 MHz
 cpu0: 
 FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
 cpu0: 256KB 64b/line 8-way L2 cache
 cpu0: smt 0, core 0, package 0
 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
 cpu0: apic clock running at 99MHz
 cpu0: mwait min=64, max=64, C-substates=0.2.1.1.2, IBE
 cpu1 at mainbus0: apid 32 (application processor)
 cpu1: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
 cpu1: 
 FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
 cpu1: 256KB 64b/line 8-way L2 cache
 cpu1: smt 0, core 0, package 1
 cpu2 at mainbus0: apid 2 (application processor)
 cpu2: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
 cpu2: 
 FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
 cpu2: 256KB 64b/line 8-way L2 cache
 cpu2: smt 0, core 1, package 0
 cpu3 at mainbus0: apid 34 (application processor)
 cpu3: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
 cpu3: 
 FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
 cpu3: 256KB 64b/line 8-way L2 cache
 cpu3: smt 0, core 1, package 1
 cpu4 at mainbus0: apid 4 (application processor)
 cpu4: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
 cpu4: 
 FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
 cpu4: 256KB 64b/line 8-way L2 cache
 cpu4: smt 0, core 2, package 0
 cpu5 at mainbus0: apid 36 (application processor)
 cpu5: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
 cpu5: 
 FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC
 cpu5: 256KB 64b/line 8-way L2 cache
 cpu5: smt 0, core 2, package 1
 cpu6 at mainbus0: apid 6 (application processor)
 cpu6: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz, 3400.00 MHz
 cpu6:

Re: Dell R630 high interrupts on acpi0

2014-12-16 Thread Hrvoje Popovski 

On 16.12.2014. 6:16, Jonathan Matthew wrote:

On Sun, Dec 14, 2014 at 06:22:37PM +0100, Hrvoje Popovski wrote:

Hi all,

I have got two new Dell R630 and have current on them from Sun Dec
14 15:07:17. Installation went great and very fast.
The problem is that I see around 11k interrupts on acpi0. First I
thought that problem is similar to this thread
http://marc.info/?l=openbsd-miscm=140551906923931w=2

But if in dell bios system profile settings is set to performance or
to DAPC there are always interrupts on acpi0.
In links bellow you can find acpidump and dmesg from performance and
DAPC settings in dell bios.


We just got some r630s too, so I spent some time last week figuring out what's
going on here.  Something in the AML wants to talk to the intel MEI device.
Normally this works, but on the new generation of dell machines (we've seen it
on r630s and r730s), it's been moved outside the pci memory range we currently
allow on amd64.  You can see this in your dmesgs:

0:22:0: mem address conflict 0x3303000/0x10
0:22:1: mem address conflict 0x3302000/0x10

The interrupt will keep triggering until it manages to talk to the device,
which will never happen.

kettenis@ says we can get the pci memory range information we need to deal with
this from acpi.  Until that happens, expanding the allowed pci memory range
makes things work properly.

ok?


Index: pci_machdep.c
===
RCS file: /cvs/src/sys/arch/amd64/pci/pci_machdep.c,v
retrieving revision 1.59
diff -u -p -u -p -r1.59 pci_machdep.c
--- pci_machdep.c   19 Apr 2014 11:53:42 -  1.59
+++ pci_machdep.c   16 Dec 2014 04:21:53 -
@@ -622,13 +622,17 @@ pci_init_extents(void)
 * here.  As long as vendors continue to support
 * 32-bit operating systems, we should never see BARs
 * outside that region.
+*
+* Dell 13G servers have important devices outside the
+* 36-bit address space.  Until we can extract the address
+* ranges from acpi, expand the allowed range to suit.
 */
pcimem_ex = extent_create(pcimem, 0, 0xUL,
M_DEVBUF, NULL, 0, EX_NOWAIT);
if (pcimem_ex == NULL)
return;
-   extent_alloc_region(pcimem_ex, 0x10UL,
-   0xfff0UL, EX_NOWAIT);
+   extent_alloc_region(pcimem_ex, 0x400UL,
+   0xfc00UL, EX_NOWAIT);

for (bmp = bios_memmap; bmp-type != BIOS_MAP_END; bmp++) {
/*




Hi,

you patch makes acpi0 calm as overeaten grandad :)

Thank you.

vmstat without your patch
# vmstat -i
interrupt   total rate
irq0/clock  103588029  799
irq0/ipi   1025960
irq144/acpi0   157993005512201
irq96/mfii0  69380
irq114/em0 2702502
irq99/ehci0   1310
irq99/ehci1280
Total  168389802713004

vmstat with your patch
# vmstat -i
interrupt   total rate
irq0/clock 848873  800
irq0/ipi14085   13
irq144/acpi030
irq96/mfii0  22352
irq114/em0   36853
irq99/ehci0560
irq99/ehci1280
Total  868965  819

Re: Dell R630 high interrupts on acpi0

2014-12-16 Thread Mark Kettenis 
 Date: Tue, 16 Dec 2014 15:16:58 +1000
 From: Jonathan Matthew jonat...@d14n.org
 
 On Sun, Dec 14, 2014 at 06:22:37PM +0100, Hrvoje Popovski wrote:
  Hi all,
  
  I have got two new Dell R630 and have current on them from Sun Dec
  14 15:07:17. Installation went great and very fast.
  The problem is that I see around 11k interrupts on acpi0. First I
  thought that problem is similar to this thread
  http://marc.info/?l=openbsd-miscm=140551906923931w=2
  
  But if in dell bios system profile settings is set to performance or
  to DAPC there are always interrupts on acpi0.
  In links bellow you can find acpidump and dmesg from performance and
  DAPC settings in dell bios.
 
 We just got some r630s too, so I spent some time last week figuring out what's
 going on here.  Something in the AML wants to talk to the intel MEI device.
 Normally this works, but on the new generation of dell machines (we've seen it
 on r630s and r730s), it's been moved outside the pci memory range we currently
 allow on amd64.  You can see this in your dmesgs:
 
 0:22:0: mem address conflict 0x3303000/0x10
 0:22:1: mem address conflict 0x3302000/0x10
 
 The interrupt will keep triggering until it manages to talk to the device,
 which will never happen.
 
 kettenis@ says we can get the pci memory range information we need to deal 
 with
 this from acpi.  Until that happens, expanding the allowed pci memory range
 makes things work properly.
 
 ok?

ok kettenis@ (although I'd prefer if you did a s/acpi/ACPI/ in the comment).

 Index: pci_machdep.c
 ===
 RCS file: /cvs/src/sys/arch/amd64/pci/pci_machdep.c,v
 retrieving revision 1.59
 diff -u -p -u -p -r1.59 pci_machdep.c
 --- pci_machdep.c 19 Apr 2014 11:53:42 -  1.59
 +++ pci_machdep.c 16 Dec 2014 04:21:53 -
 @@ -622,13 +622,17 @@ pci_init_extents(void)
* here.  As long as vendors continue to support
* 32-bit operating systems, we should never see BARs
* outside that region.
 +  *
 +  * Dell 13G servers have important devices outside the
 +  * 36-bit address space.  Until we can extract the address
 +  * ranges from acpi, expand the allowed range to suit.
*/
   pcimem_ex = extent_create(pcimem, 0, 0xUL,
   M_DEVBUF, NULL, 0, EX_NOWAIT);
   if (pcimem_ex == NULL)
   return;
 - extent_alloc_region(pcimem_ex, 0x10UL,
 - 0xfff0UL, EX_NOWAIT);
 + extent_alloc_region(pcimem_ex, 0x400UL,
 + 0xfc00UL, EX_NOWAIT);
  
   for (bmp = bios_memmap; bmp-type != BIOS_MAP_END; bmp++) {
   /*

Re: Dell R630 high interrupts on acpi0

2014-12-16 Thread Hrvoje Popovski 

On 16.12.2014. 6:16, Jonathan Matthew wrote:

We just got some r630s too, so I spent some time last week figuring out what's
going on here.  Something in the AML wants to talk to the intel MEI device.
Normally this works, but on the new generation of dell machines (we've seen it
on r630s and r730s), it's been moved outside the pci memory range we currently
allow on amd64.  You can see this in your dmesgs:

0:22:0: mem address conflict 0x3303000/0x10
0:22:1: mem address conflict 0x3302000/0x10

The interrupt will keep triggering until it manages to talk to the device,
which will never happen.

kettenis@ says we can get the pci memory range information we need to deal with
this from acpi.  Until that happens, expanding the allowed pci memory range
makes things work properly.



Hi,

on R630 i have custom bios settings and noticed that even if C states 
are disabled in bios i can see them in dmesg

acpicpu0 at acpi0: C1
acpicpu1 at acpi0: C1
acpicpu2 at acpi0: C1
acpicpu3 at acpi0: C1
acpicpu4 at acpi0: C1
acpicpu5 at acpi0: C1
acpicpu6 at acpi0: C1
acpicpu7 at acpi0: C1

X2Apic is disabled too but in dmesg i see
cpu0: FPU,CPI,...SSE4.1,SSE4.2,x2APIC

This is not good, right?

R630 bios settings

Processor Settings:
Logical Processor - Disabled
QPI Speed - Maximum data rate
Alternate RTID Settings - Disabled
Virtualization Technology - Disabled
Address Translation Service (ATS) - Disabled (gray)
Adjacent Cache Line Prefetch - Enabled
Hardware Prefetcher - Enabled
DCU Streamer Prefetcher - Enabled
DCU IP Prefetcher - Enabled
Execute Disabled - Enabled
Logical Processor Idling - Disabled
Configurable TDP - Nominal
X2Apic Mode - Disabled (gray)
Dell Controlled Turbo - Enabled

System Profile Settings:
System Profile - Custom
CPU Power Management - Maximum Performance
Memory Frequency - Maximum Performance
Turbo Boost - Enabled
Energy Efficient Turbo - Disabled
C1E - Disabled
C states - Disabled
Collaborative CPU Performance Control - Disabled
Memory Patrol Scrub - Standard
Memory Refresh Rate - 1x
Uncore Frequency - Maximum
Energy Efficient Policy - Performance

Full dmesg and acpidump from R630
http://kosjenka.srce.hr/~hrvoje/R630_custom_dmesg.txt
http://kosjenka.srce.hr/~hrvoje/R630_custom.tgz



R620 have similar settings and can't see C states in dmesg
acpicpu0 at acpi0
acpicpu1 at acpi0
acpicpu2 at acpi0
acpicpu3 at acpi0
acpicpu4 at acpi0
acpicpu5 at acpi0

R620 bios settings

Processor Settings:
Logical Processor - Disabled
Alternate RTID Settings - Disabled
Virtualization Technology - Disabled
Adjacent Cache Line Prefetch - Enabled
Hardware Prefetcher - Enabled
DCU Streamer Prefetcher - Enabled
DCU IP Prefetcher - Enabled
Logical Processor Idling - Disabled
Dell Controlled Turbo - Enabled

System Profile Settings:
System Profile - Custom
CPU Power Management - Maximum Performance
Memory Frequency - Maximum Performance
Turbo Boost - Enabled
C1E - Disabled
C states - Disabled
Monitor/Mwait - Disabled
Memory Patrol Scrub - Standard
Memory Refresh Rate - 1x
Memory Operating Voltage - Auto
Collaborative CPU Performance Control - Disabled


Full dmesg and acpidump from R620
http://kosjenka.srce.hr/~hrvoje/R620_custom_dmesg.txt
http://kosjenka.srce.hr/~hrvoje/R620_custom.tgz

Re: Dell R630 high interrupts on acpi0

2014-12-16 Thread Philip Guenther 
On Tue, Dec 16, 2014 at 2:45 PM, Hrvoje Popovski hrv...@srce.hr wrote:
 on R630 i have custom bios settings and noticed that even if C states are
 disabled in bios i can see them in dmesg
 acpicpu0 at acpi0: C1

Uh, ACPI *requires* that C1 exist.  The halt instruction is defined as
entering C1, so not having C1 would mean your CPU lacks a basic
manadatory ia32 instruction.  Hopefully the BIOS docs explain that
you're just disabling deep C-states or something like that.  If not,
yell at the company that made it.

With the exception of C1E, I wouldn't tell a BIOS to disable
C-states unless it was causing the OS to have a problem or you're
actively trying to use the computer to heat your house.  C1E is a
cross between C1 and C3; the issue is that bugs in multiple early
hardware implementations mean it'll behave poorly depending on exactly
how the OS handles it.  This is something to test...and then test
again with each release you install...


 X2Apic is disabled too but in dmesg i see
 cpu0: FPU,CPI,...SSE4.1,SSE4.2,x2APIC

That just means the CPU has the feature bit set in the CPUID sets.
The BIOS is presumably configuring ACPI (which is what OpenBSD pays
attention to) to use the original LAPIC tables instead of the x2APIC
tables for locating CPUs and interrupts.


 R620 have similar settings and can't see C states in dmesg
 acpicpu0 at acpi0

That's either insane, or a bug in our acpicpu code, IMO.


 R620 bios settings
...
 Monitor/Mwait - Disabled

I would suggest leaving that on.  We ain't using it *right now*, but,
well, the source tree on my laptop is, and more than ever.  :-)


Philip Guenther

Re: Dell R630 high interrupts on acpi0

2014-12-16 Thread Frederic Nowak 
Hi!

The below diff extracts the memory range information from ACPI. It looks
up all the memory ranges in _CRS and calculates minimal and maximal
values for pci_machdep.c.

I tested this on two amd64 machines and see no difference in pcidump.

Do you think we need to keep the old method in case the ACPI on some
machines does not report correct memory ranges? A simple check would be
to test if pcimem_range[0]  pcimem_range[1]...


Index: sys/arch/amd64/include/pci_machdep.h
===
RCS file: /cvs/src/sys/arch/amd64/include/pci_machdep.h,v
retrieving revision 1.22
diff -u -p -b -r1.22 pci_machdep.h
--- sys/arch/amd64/include/pci_machdep.h6 Nov 2013 10:40:36 -   
1.22
+++ sys/arch/amd64/include/pci_machdep.h17 Dec 2014 06:00:01 -
@@ -64,6 +64,7 @@ extern int pci_mcfg_min_bus, pci_mcfg_ma
 
 struct pci_attach_args;
 
+extern uint64_t pcimem_range[2];
 extern struct extent *pciio_ex;
 extern struct extent *pcimem_ex;
 extern struct extent *pcibus_ex;
Index: sys/arch/amd64/pci/pci_machdep.c
===
RCS file: /cvs/src/sys/arch/amd64/pci/pci_machdep.c,v
retrieving revision 1.60
diff -u -p -b -r1.60 pci_machdep.c
--- sys/arch/amd64/pci/pci_machdep.c16 Dec 2014 23:13:20 -  1.60
+++ sys/arch/amd64/pci/pci_machdep.c17 Dec 2014 07:16:47 -
@@ -66,6 +66,7 @@
  */
 
 #include sys/types.h
+#include sys/stdint.h
 #include sys/param.h
 #include sys/time.h
 #include sys/systm.h
@@ -592,6 +593,7 @@ pci_intr_disestablish(pci_chipset_tag_t 
intr_disestablish(cookie);
 }
 
+uint64_t pcimem_range[2] = {UINT64_MAX, 0};
 struct extent *pciio_ex;
 struct extent *pcimem_ex;
 struct extent *pcibus_ex;
@@ -618,31 +620,25 @@ pci_init_extents(void)
 
if (pcimem_ex == NULL) {
/*
-* Cover the 36-bit address space addressable by PAE
-* here.  As long as vendors continue to support
-* 32-bit operating systems, we should never see BARs
-* outside that region.
-*
-* Dell 13G servers have important devices outside the
-* 36-bit address space.  Until we can extract the address
-* ranges from ACPI, expand the allowed range to suit.
+* Cover the address space extracted from ACPI.
 */
pcimem_ex = extent_create(pcimem, 0, 0xUL,
M_DEVBUF, NULL, 0, EX_NOWAIT);
if (pcimem_ex == NULL)
return;
-   extent_alloc_region(pcimem_ex, 0x400UL,
-   0xfc00UL, EX_NOWAIT);
+   extent_alloc_region(pcimem_ex, pcimem_range[1] + 1,
+   (0xUL - pcimem_range[1]), EX_NOWAIT);
 
for (bmp = bios_memmap; bmp-type != BIOS_MAP_END; bmp++) {
/*
-* Ignore address space beyond 4G.
+* Ignore address space beyond address range
+* extracted from ACPI.
 */
-   if (bmp-addr = 0x1ULL)
+   if (bmp-addr  pcimem_range[1])
continue;
size = bmp-size;
-   if (bmp-addr + size = 0x1ULL)
-   size = 0x1ULL - bmp-addr;
+   if (bmp-addr + size  pcimem_range[1])
+   size = pcimem_range[1] - bmp-addr + 1;
 
/* Ignore zero-sized regions. */
if (size == 0)
Index: sys/dev/acpi/acpi.c
===
RCS file: /cvs/src/sys/dev/acpi/acpi.c,v
retrieving revision 1.277
diff -u -p -b -r1.277 acpi.c
--- sys/dev/acpi/acpi.c 9 Dec 2014 06:58:29 -   1.277
+++ sys/dev/acpi/acpi.c 17 Dec 2014 05:59:36 -
@@ -385,0 +385,0 @@ TAILQ_HEAD(, acpi_pci) acpi_pcirootdevs

 int acpi_getpci(struct aml_node *node, void *arg);
 int acpi_getminbus(union acpi_resource *crs, void *arg);
+int acpi_getmemrange(union acpi_resource *crs, void *arg);
+
+int
+acpi_getmemrange(union acpi_resource *crs, void *arg)
+{
+   int typ = AML_CRSTYPE(crs);
+   uint64_t *range = arg;  /* size 2 */
+   uint64_t min, max;
+
+   switch(typ) {
+   case LR_24BIT:
+   min = crs-lr_m24._min;
+   max = crs-lr_m24._max;
+   break;
+   case LR_32BIT:
+   case LR_32BITFIXED:
+   min = crs-lr_m32._min;
+   max = crs-lr_m32._max;
+   break;
+   case LR_WORD:
+   min = crs-lr_word._min;
+   max = crs-lr_word._max;
+   break;
+   case LR_DWORD:
+   min = crs-lr_dword._min;
+   max =

Re: Dell R630 high interrupts on acpi0

2014-12-15 Thread Jonathan Matthew 
On Sun, Dec 14, 2014 at 06:22:37PM +0100, Hrvoje Popovski wrote:
 Hi all,
 
 I have got two new Dell R630 and have current on them from Sun Dec
 14 15:07:17. Installation went great and very fast.
 The problem is that I see around 11k interrupts on acpi0. First I
 thought that problem is similar to this thread
 http://marc.info/?l=openbsd-miscm=140551906923931w=2
 
 But if in dell bios system profile settings is set to performance or
 to DAPC there are always interrupts on acpi0.
 In links bellow you can find acpidump and dmesg from performance and
 DAPC settings in dell bios.

We just got some r630s too, so I spent some time last week figuring out what's
going on here.  Something in the AML wants to talk to the intel MEI device.
Normally this works, but on the new generation of dell machines (we've seen it
on r630s and r730s), it's been moved outside the pci memory range we currently
allow on amd64.  You can see this in your dmesgs:

0:22:0: mem address conflict 0x3303000/0x10
0:22:1: mem address conflict 0x3302000/0x10

The interrupt will keep triggering until it manages to talk to the device,
which will never happen.

kettenis@ says we can get the pci memory range information we need to deal with
this from acpi.  Until that happens, expanding the allowed pci memory range
makes things work properly.

ok?


Index: pci_machdep.c
===
RCS file: /cvs/src/sys/arch/amd64/pci/pci_machdep.c,v
retrieving revision 1.59
diff -u -p -u -p -r1.59 pci_machdep.c
--- pci_machdep.c   19 Apr 2014 11:53:42 -  1.59
+++ pci_machdep.c   16 Dec 2014 04:21:53 -
@@ -622,13 +622,17 @@ pci_init_extents(void)
 * here.  As long as vendors continue to support
 * 32-bit operating systems, we should never see BARs
 * outside that region.
+*
+* Dell 13G servers have important devices outside the
+* 36-bit address space.  Until we can extract the address
+* ranges from acpi, expand the allowed range to suit.
 */
pcimem_ex = extent_create(pcimem, 0, 0xUL,
M_DEVBUF, NULL, 0, EX_NOWAIT);
if (pcimem_ex == NULL)
return;
-   extent_alloc_region(pcimem_ex, 0x10UL,
-   0xfff0UL, EX_NOWAIT);
+   extent_alloc_region(pcimem_ex, 0x400UL,
+   0xfc00UL, EX_NOWAIT);
 
for (bmp = bios_memmap; bmp-type != BIOS_MAP_END; bmp++) {
/*

Re: Dell R630 high interrupts on acpi0

2014-12-15 Thread David Gwynne 

 On 16 Dec 2014, at 15:16, Jonathan Matthew jonat...@d14n.org wrote:
 
 On Sun, Dec 14, 2014 at 06:22:37PM +0100, Hrvoje Popovski wrote:
 Hi all,
 
 I have got two new Dell R630 and have current on them from Sun Dec
 14 15:07:17. Installation went great and very fast.
 The problem is that I see around 11k interrupts on acpi0. First I
 thought that problem is similar to this thread
 http://marc.info/?l=openbsd-miscm=140551906923931w=2
 
 But if in dell bios system profile settings is set to performance or
 to DAPC there are always interrupts on acpi0.
 In links bellow you can find acpidump and dmesg from performance and
 DAPC settings in dell bios.
 
 We just got some r630s too, so I spent some time last week figuring out what's
 going on here.  Something in the AML wants to talk to the intel MEI device.
 Normally this works, but on the new generation of dell machines (we've seen it
 on r630s and r730s), it's been moved outside the pci memory range we currently
 allow on amd64.  You can see this in your dmesgs:
 
 0:22:0: mem address conflict 0x3303000/0x10
 0:22:1: mem address conflict 0x3302000/0x10
 
 The interrupt will keep triggering until it manages to talk to the device,
 which will never happen.

we've also experienced this on r720s configured in Performance mode in the BIOS 
too. others have hit this on r620s as well (look for Dell R620 high ACPI 
Interrupt rate on misc@). the diff below fixes them too.

dlg

 
 kettenis@ says we can get the pci memory range information we need to deal 
 with
 this from acpi.  Until that happens, expanding the allowed pci memory range
 makes things work properly.
 
 ok?
 
 
 Index: pci_machdep.c
 ===
 RCS file: /cvs/src/sys/arch/amd64/pci/pci_machdep.c,v
 retrieving revision 1.59
 diff -u -p -u -p -r1.59 pci_machdep.c
 --- pci_machdep.c 19 Apr 2014 11:53:42 -  1.59
 +++ pci_machdep.c 16 Dec 2014 04:21:53 -
 @@ -622,13 +622,17 @@ pci_init_extents(void)
* here.  As long as vendors continue to support
* 32-bit operating systems, we should never see BARs
* outside that region.
 +  *
 +  * Dell 13G servers have important devices outside the
 +  * 36-bit address space.  Until we can extract the address
 +  * ranges from acpi, expand the allowed range to suit.
*/
   pcimem_ex = extent_create(pcimem, 0, 0xUL,
   M_DEVBUF, NULL, 0, EX_NOWAIT);
   if (pcimem_ex == NULL)
   return;
 - extent_alloc_region(pcimem_ex, 0x10UL,
 - 0xfff0UL, EX_NOWAIT);
 + extent_alloc_region(pcimem_ex, 0x400UL,
 + 0xfc00UL, EX_NOWAIT);
 
   for (bmp = bios_memmap; bmp-type != BIOS_MAP_END; bmp++) {
   /*