Re: 11.0 stuck on high network load

[Slawa Olhovchenkov]

On Thu, Sep 15, 2016 at 02:33:07AM +0300, Oleksandr V. Typlyns'kyi wrote:

> Sep 5 Sep 5, 2016 at 00:57 Slawa Olhovchenkov wrote:
> 
> > I am try using 11.0 on Dual E5-2620 (no X2APIC).
> > Under high network load and may be addtional conditional system go to
> > unresponsible state -- no reaction to network and console (USB IPMI
> > emulation). INVARIANTS give to high overhad. Is this exist some way to
> > debug this?
> 
>   Do you have any routing table changes?
>   In my case system totally hangs after route change command invocation.
>   Sequence of commands route del + route add works fine.
>   Maybe it is FIB related, but with -fib option system hangs too.

No, only default exist and don't changed
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 11.0 stuck on high network load

[Oleksandr V. Typlyns'kyi]

Sep 5 Sep 5, 2016 at 00:57 Slawa Olhovchenkov wrote:

> I am try using 11.0 on Dual E5-2620 (no X2APIC).
> Under high network load and may be addtional conditional system go to
> unresponsible state -- no reaction to network and console (USB IPMI
> emulation). INVARIANTS give to high overhad. Is this exist some way to
> debug this?

  Do you have any routing table changes?
  In my case system totally hangs after route change command invocation.
  Sequence of commands route del + route add works fine.
  Maybe it is FIB related, but with -fib option system hangs too.

-- 
WNGS-RIPE
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 11.0 stuck on high network load

[Slawa Olhovchenkov]

On Wed, Sep 14, 2016 at 11:23:20PM +0100, Gary Palmer wrote:

> On Thu, Sep 15, 2016 at 01:13:35AM +0300, Slawa Olhovchenkov wrote:
> > On Wed, Sep 14, 2016 at 03:04:20PM -0700, hiren panchasara wrote:
> > 
> > > On 09/15/16 at 12:57P, Slawa Olhovchenkov wrote:
> > > > On Wed, Sep 14, 2016 at 02:43:06PM -0700, hiren panchasara wrote:
> > > > 
> > > > > On 09/15/16 at 12:35P, Slawa Olhovchenkov wrote:
> > > > > > On Sun, Sep 04, 2016 at 06:46:12PM -0700, hiren panchasara wrote:
> > > > > > 
> > > > > > > On 09/05/16 at 12:57P, Slawa Olhovchenkov wrote:
> > > > > > > > I am try using 11.0 on Dual E5-2620 (no X2APIC).
> > > > > > > > Under high network load and may be addtional conditional system 
> > > > > > > > go to
> > > > > > > > unresponsible state -- no reaction to network and console (USB 
> > > > > > > > IPMI
> > > > > > > > emulation). INVARIANTS give to high overhad. Is this exist some 
> > > > > > > > way to
> > > > > > > > debug this?
> > > > > > > 
> > > > > > > Can you panic it from console to get to db> to get backtrace and 
> > > > > > > other
> > > > > > > info when it goes unresponsive?
> > > > > > 
> > > > > > ipmi console don't respond (chassis power diag don't react)
> > > > > > login on sol console stuck on *tcp.
> > > > > 
> > > > > I assume you tried ~^b (tilda followed by ctrl+b) without success?
> > > > 
> > > > ~B, as in man ipmitool
> > > 
> > > No, not shift-b but ctrl-b.
> > > 
> > > I am not aware of ipmitool reference. On unresponsive console, try
> > > ~^b (tilda followed by ctrl+b)
> > 
> > ipmitool ~B send break, do you talk about break or about special
> > serial console keystroke to enter debuger, distinct from break?
> 
> A lot of old console servers used to send break when they reset or
> on boot, so the FreeBSD console does not break to debugger when
> a break command is received.  That's why the tidla CTRL-b is
> required as it's sufficiently unlikely to be accidentally sent by
> equipment connected to the serial console.

OK, thanks to clarification, I am try ~^B on next attempt.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 11.0 stuck on high network load

[Gary Palmer]

On Thu, Sep 15, 2016 at 01:13:35AM +0300, Slawa Olhovchenkov wrote:
> On Wed, Sep 14, 2016 at 03:04:20PM -0700, hiren panchasara wrote:
> 
> > On 09/15/16 at 12:57P, Slawa Olhovchenkov wrote:
> > > On Wed, Sep 14, 2016 at 02:43:06PM -0700, hiren panchasara wrote:
> > > 
> > > > On 09/15/16 at 12:35P, Slawa Olhovchenkov wrote:
> > > > > On Sun, Sep 04, 2016 at 06:46:12PM -0700, hiren panchasara wrote:
> > > > > 
> > > > > > On 09/05/16 at 12:57P, Slawa Olhovchenkov wrote:
> > > > > > > I am try using 11.0 on Dual E5-2620 (no X2APIC).
> > > > > > > Under high network load and may be addtional conditional system 
> > > > > > > go to
> > > > > > > unresponsible state -- no reaction to network and console (USB 
> > > > > > > IPMI
> > > > > > > emulation). INVARIANTS give to high overhad. Is this exist some 
> > > > > > > way to
> > > > > > > debug this?
> > > > > > 
> > > > > > Can you panic it from console to get to db> to get backtrace and 
> > > > > > other
> > > > > > info when it goes unresponsive?
> > > > > 
> > > > > ipmi console don't respond (chassis power diag don't react)
> > > > > login on sol console stuck on *tcp.
> > > > 
> > > > I assume you tried ~^b (tilda followed by ctrl+b) without success?
> > > 
> > > ~B, as in man ipmitool
> > 
> > No, not shift-b but ctrl-b.
> > 
> > I am not aware of ipmitool reference. On unresponsive console, try
> > ~^b (tilda followed by ctrl+b)
> 
> ipmitool ~B send break, do you talk about break or about special
> serial console keystroke to enter debuger, distinct from break?

A lot of old console servers used to send break when they reset or
on boot, so the FreeBSD console does not break to debugger when
a break command is received.  That's why the tidla CTRL-b is
required as it's sufficiently unlikely to be accidentally sent by
equipment connected to the serial console.

Regards,

Gary
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

11.0 stuck on high network load

[Oliver Pinter]

On Monday, September 5, 2016, Warner Losh > wrote:

> On Mon, Sep 5, 2016 at 1:43 AM, Slawa Olhovchenkov  wrote:
> > On Sun, Sep 04, 2016 at 06:46:12PM -0700, hiren panchasara wrote:
> >
> >> On 09/05/16 at 12:57P, Slawa Olhovchenkov wrote:
> >> > I am try using 11.0 on Dual E5-2620 (no X2APIC).
> >> > Under high network load and may be addtional conditional system go to
> >> > unresponsible state -- no reaction to network and console (USB IPMI
> >> > emulation). INVARIANTS give to high overhad. Is this exist some way to
> >> > debug this?
> >>
> >> Can you panic it from console to get to db> to get backtrace and other
> >> info when it goes unresponsive?
> >
> > no
> > no reaction
>
> So the canonical 'ipmitool chassis power diag' doesn't send an NMI to
> get you to the debugger?
>
> I've seen this at Netflix on one variant of our flash offload box with
> a Intel e5-2697v2 running with the Chelsio driver. We're working
> around it by having fewer receive threads than CPUs in the system. The
> only way the boxes would come back was with watchdog. The load was
> streaming video > ~36Gbps out 4 lagged 10G ports. Console is totally
> unresponsive as well.


> Try to set kern.sched.preempt_thresh sysctl to 224 to get back your
> console..


> This is on our FreeBSD-10 stable based fork.
> From my debugging, we go from totally fine as far as I can tell from
> ps, etc in the moments leading to the hang to being totally wedged. It
> seems a very sudden-onset condition. Sound at all familiar?
>
> Warner
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 11.0 stuck on high network load

[Slawa Olhovchenkov]

On Wed, Sep 14, 2016 at 03:04:20PM -0700, hiren panchasara wrote:

> On 09/15/16 at 12:57P, Slawa Olhovchenkov wrote:
> > On Wed, Sep 14, 2016 at 02:43:06PM -0700, hiren panchasara wrote:
> > 
> > > On 09/15/16 at 12:35P, Slawa Olhovchenkov wrote:
> > > > On Sun, Sep 04, 2016 at 06:46:12PM -0700, hiren panchasara wrote:
> > > > 
> > > > > On 09/05/16 at 12:57P, Slawa Olhovchenkov wrote:
> > > > > > I am try using 11.0 on Dual E5-2620 (no X2APIC).
> > > > > > Under high network load and may be addtional conditional system go 
> > > > > > to
> > > > > > unresponsible state -- no reaction to network and console (USB IPMI
> > > > > > emulation). INVARIANTS give to high overhad. Is this exist some way 
> > > > > > to
> > > > > > debug this?
> > > > > 
> > > > > Can you panic it from console to get to db> to get backtrace and other
> > > > > info when it goes unresponsive?
> > > > 
> > > > ipmi console don't respond (chassis power diag don't react)
> > > > login on sol console stuck on *tcp.
> > > 
> > > I assume you tried ~^b (tilda followed by ctrl+b) without success?
> > 
> > ~B, as in man ipmitool
> 
> No, not shift-b but ctrl-b.
> 
> I am not aware of ipmitool reference. On unresponsive console, try
> ~^b (tilda followed by ctrl+b)

ipmitool ~B send break, do you talk about break or about special
serial console keystroke to enter debuger, distinct from break?

> > 
> > > That usually drops into db>
> > 
> > May be now need some sysctl for enable this?
> 
> There is a sysctl for this too but on console, the keystrokes I said
> should work, imo.
> 
> Cheers,
> Hiren


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 11.0 stuck on high network load

[hiren panchasara]

On 09/15/16 at 12:57P, Slawa Olhovchenkov wrote:
> On Wed, Sep 14, 2016 at 02:43:06PM -0700, hiren panchasara wrote:
> 
> > On 09/15/16 at 12:35P, Slawa Olhovchenkov wrote:
> > > On Sun, Sep 04, 2016 at 06:46:12PM -0700, hiren panchasara wrote:
> > > 
> > > > On 09/05/16 at 12:57P, Slawa Olhovchenkov wrote:
> > > > > I am try using 11.0 on Dual E5-2620 (no X2APIC).
> > > > > Under high network load and may be addtional conditional system go to
> > > > > unresponsible state -- no reaction to network and console (USB IPMI
> > > > > emulation). INVARIANTS give to high overhad. Is this exist some way to
> > > > > debug this?
> > > > 
> > > > Can you panic it from console to get to db> to get backtrace and other
> > > > info when it goes unresponsive?
> > > 
> > > ipmi console don't respond (chassis power diag don't react)
> > > login on sol console stuck on *tcp.
> > 
> > I assume you tried ~^b (tilda followed by ctrl+b) without success?
> 
> ~B, as in man ipmitool

No, not shift-b but ctrl-b.

I am not aware of ipmitool reference. On unresponsive console, try
~^b (tilda followed by ctrl+b)

> 
> > That usually drops into db>
> 
> May be now need some sysctl for enable this?

There is a sysctl for this too but on console, the keystrokes I said
should work, imo.

Cheers,
Hiren


pgpUmm0V7ZAAG.pgp
Description: PGP signature

Re: 11.0 stuck on high network load

[Slawa Olhovchenkov]

On Wed, Sep 14, 2016 at 02:56:57PM -0700, hiren panchasara wrote:

> On 09/15/16 at 12:35P, Slawa Olhovchenkov wrote:
> > On Sun, Sep 04, 2016 at 06:46:12PM -0700, hiren panchasara wrote:
> > 
> > > On 09/05/16 at 12:57P, Slawa Olhovchenkov wrote:
> > > > I am try using 11.0 on Dual E5-2620 (no X2APIC).
> > > > Under high network load and may be addtional conditional system go to
> > > > unresponsible state -- no reaction to network and console (USB IPMI
> > > > emulation). INVARIANTS give to high overhad. Is this exist some way to
> > > > debug this?
> > > 
> > > Can you panic it from console to get to db> to get backtrace and other
> > > info when it goes unresponsive?
> > 
> > ipmi console don't respond (chassis power diag don't react)
> > login on sol console stuck on *tcp.
> 
> Also *tcp means its stuck on lock tcp? if so, that'd be lock on
> V_tcbinfo. I think?
> 
> tcp_subr.c has tcp_init() which calls
> in_pcbinfo_init(_tcbinfo, "tcp", _tcb, hashsize, hashsize,
> "tcp_inpcb", tcp_inpcb_init, NULL, 0, IPI_HASHFIELDS_4TUPLE);
> 
> and "tcp" is the name used to initialise the lock inside
> in_pcbinfo_init() with
> INP_INFO_LOCK_INIT(pcbinfo, name); 
> 
> What exact svn rev are you on?

r305117

> Also do you have any local changes?

only kerberos related.
I am using kerberos and NIS.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 11.0 stuck on high network load

[Slawa Olhovchenkov]

On Wed, Sep 14, 2016 at 02:43:06PM -0700, hiren panchasara wrote:

> On 09/15/16 at 12:35P, Slawa Olhovchenkov wrote:
> > On Sun, Sep 04, 2016 at 06:46:12PM -0700, hiren panchasara wrote:
> > 
> > > On 09/05/16 at 12:57P, Slawa Olhovchenkov wrote:
> > > > I am try using 11.0 on Dual E5-2620 (no X2APIC).
> > > > Under high network load and may be addtional conditional system go to
> > > > unresponsible state -- no reaction to network and console (USB IPMI
> > > > emulation). INVARIANTS give to high overhad. Is this exist some way to
> > > > debug this?
> > > 
> > > Can you panic it from console to get to db> to get backtrace and other
> > > info when it goes unresponsive?
> > 
> > ipmi console don't respond (chassis power diag don't react)
> > login on sol console stuck on *tcp.
> 
> I assume you tried ~^b (tilda followed by ctrl+b) without success?

~B, as in man ipmitool

> That usually drops into db>

May be now need some sysctl for enable this?

> I am also fighting an issue where upon said keystrokes, I see 
> "KDB: enter: Break to debugger" but it doesn't drop to db>
> At that point I have to 'ipmitool blah power reset' the box.
> 
> Cheers,
> Hiren


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 11.0 stuck on high network load

[hiren panchasara]

On 09/15/16 at 12:35P, Slawa Olhovchenkov wrote:
> On Sun, Sep 04, 2016 at 06:46:12PM -0700, hiren panchasara wrote:
> 
> > On 09/05/16 at 12:57P, Slawa Olhovchenkov wrote:
> > > I am try using 11.0 on Dual E5-2620 (no X2APIC).
> > > Under high network load and may be addtional conditional system go to
> > > unresponsible state -- no reaction to network and console (USB IPMI
> > > emulation). INVARIANTS give to high overhad. Is this exist some way to
> > > debug this?
> > 
> > Can you panic it from console to get to db> to get backtrace and other
> > info when it goes unresponsive?
> 
> ipmi console don't respond (chassis power diag don't react)
> login on sol console stuck on *tcp.

Also *tcp means its stuck on lock tcp? if so, that'd be lock on
V_tcbinfo. I think?

tcp_subr.c has tcp_init() which calls
in_pcbinfo_init(_tcbinfo, "tcp", _tcb, hashsize, hashsize,
"tcp_inpcb", tcp_inpcb_init, NULL, 0, IPI_HASHFIELDS_4TUPLE);

and "tcp" is the name used to initialise the lock inside
in_pcbinfo_init() with
INP_INFO_LOCK_INIT(pcbinfo, name); 

What exact svn rev are you on? Also do you have any local changes?

Cheers,
Hiren


pgpU1mnjIQD8n.pgp
Description: PGP signature

Re: 11.0 stuck on high network load

[hiren panchasara]

On 09/15/16 at 12:35P, Slawa Olhovchenkov wrote:
> On Sun, Sep 04, 2016 at 06:46:12PM -0700, hiren panchasara wrote:
> 
> > On 09/05/16 at 12:57P, Slawa Olhovchenkov wrote:
> > > I am try using 11.0 on Dual E5-2620 (no X2APIC).
> > > Under high network load and may be addtional conditional system go to
> > > unresponsible state -- no reaction to network and console (USB IPMI
> > > emulation). INVARIANTS give to high overhad. Is this exist some way to
> > > debug this?
> > 
> > Can you panic it from console to get to db> to get backtrace and other
> > info when it goes unresponsive?
> 
> ipmi console don't respond (chassis power diag don't react)
> login on sol console stuck on *tcp.

I assume you tried ~^b (tilda followed by ctrl+b) without success?

That usually drops into db>

I am also fighting an issue where upon said keystrokes, I see 
"KDB: enter: Break to debugger" but it doesn't drop to db>
At that point I have to 'ipmitool blah power reset' the box.

Cheers,
Hiren


pgphfkL7YoAtv.pgp
Description: PGP signature

Re: 11.0 stuck on high network load

[Slawa Olhovchenkov]

On Sun, Sep 04, 2016 at 06:46:12PM -0700, hiren panchasara wrote:

> On 09/05/16 at 12:57P, Slawa Olhovchenkov wrote:
> > I am try using 11.0 on Dual E5-2620 (no X2APIC).
> > Under high network load and may be addtional conditional system go to
> > unresponsible state -- no reaction to network and console (USB IPMI
> > emulation). INVARIANTS give to high overhad. Is this exist some way to
> > debug this?
> 
> Can you panic it from console to get to db> to get backtrace and other
> info when it goes unresponsive?

ipmi console don't respond (chassis power diag don't react)
login on sol console stuck on *tcp.

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: X2APIC support

[Slawa Olhovchenkov]

On Wed, Sep 14, 2016 at 07:08:02PM +0300, Konstantin Belousov wrote:

> On Wed, Sep 14, 2016 at 05:02:21PM +0300, Andriy Gapon wrote:
> > On 14/09/2016 15:49, Slawa Olhovchenkov wrote:
> > > MSR_APICBASE = 0xfee00d00
> > > x2APIC is prohibited but turned on by BIOS
> > 
> > Kostik, ^
> 
> Well, the following might work, but I have no good idea what to do
> when BIOS does handoff with x2APIC enabled and directs us to not
> enable it.  Switching to xAPIC mode is not an option, I suspect.

CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (2200.05-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x406f1  Family=0x6  Model=0x4f  Stepping=1
  
Features=0xbfebfbff
  
Features2=0x7ffefbff
  AMD Features=0x2c100800
  AMD Features2=0x121
  Structured Extended 
Features=0x21cbfbb
  XSAVE Features=0x1
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
real memory  = 137438953472 (131072 MB)
avail memory = 133407973376 (127227 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: 
boot_cpu_id = 0
FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs
FreeBSD/SMP: 2 package(s) x 12 core(s)
random: unblocking device.
ioapic0  irqs 0-23 on motherboard
ioapic1  irqs 24-47 on motherboard
ioapic2  irqs 48-71 on motherboard
random: entropy device external interface
module_register_init: MOD_LOAD (vesa, 0x807bf2f0, 0) error 19
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
kbd1 at kbdmux0
netmap: loaded module
vtvga0:  on motherboard
cryptosoft0:  on motherboard
acpi0:  on motherboard
acpi0: Power Button (fixed)
[...]

boot OK, w/o any APIC-related messages

> diff --git a/sys/x86/acpica/madt.c b/sys/x86/acpica/madt.c
> index 241a769..3a93fd6 100644
> --- a/sys/x86/acpica/madt.c
> +++ b/sys/x86/acpica/madt.c
> @@ -138,7 +138,6 @@ madt_setup_local(void)
>  
>   madt = pmap_mapbios(madt_physaddr, madt_length);
>   if ((cpu_feature2 & CPUID2_X2APIC) != 0) {
> - x2apic_mode = 1;
>   reason = NULL;
>  
>   /*
> @@ -150,21 +149,17 @@ madt_setup_local(void)
>   if (dmartbl_physaddr != 0) {
>   dmartbl = acpi_map_table(dmartbl_physaddr,
>   ACPI_SIG_DMAR);
> - if ((dmartbl->Flags & ACPI_DMAR_X2APIC_OPT_OUT) != 0) {
> - x2apic_mode = 0;
> + if ((dmartbl->Flags & ACPI_DMAR_X2APIC_OPT_OUT) != 0)
>   reason = "by DMAR table";
> - }
>   acpi_unmap_table(dmartbl);
>   }
>   if (vm_guest == VM_GUEST_VMWARE) {
>   vmware_hvcall(VMW_HVCMD_GETVCPU_INFO, p);
>   if ((p[0] & VMW_VCPUINFO_VCPU_RESERVED) != 0 ||
> - (p[0] & VMW_VCPUINFO_LEGACY_X2APIC) == 0) {
> - x2apic_mode = 0;
> - reason = "inside VMWare without intr redirection";
> - }
> + (p[0] & VMW_VCPUINFO_LEGACY_X2APIC) == 0)
> + reason =
> + "inside VMWare without intr redirection";
>   } else if (vm_guest == VM_GUEST_XEN) {
> - x2apic_mode = 0;
>   reason = "due to running under XEN";
>   } else if (vm_guest == VM_GUEST_NO &&
>   CPUID_TO_FAMILY(cpu_id) == 0x6 &&
> @@ -184,13 +179,21 @@ madt_setup_local(void)
>   if (!strcmp(hw_vendor, "LENOVO") ||
>   !strcmp(hw_vendor,
>   "ASUSTeK Computer Inc.")) {
> - x2apic_mode = 0;
>   reason =
>   "for a suspected SandyBridge BIOS bug";
>   }
>   freeenv(hw_vendor);
>   }
>   }
> + if (reason != NULL && lapic_is_x2apic()) {
> + if (bootverbose)
> + printf("x2APIC should be disabled %s but "
> + "already enabled by BIOS; enabling.\n",
> +  reason);
> + reason = NULL;
> + }
> + if (reason == NULL)
> + x2apic_mode = 1;
>   TUNABLE_INT_FETCH("hw.x2apic_enable", _mode);
>   if 

Re: X2APIC support

[Andriy Gapon]

On 14/09/2016 19:08, Konstantin Belousov wrote:
> Well, the following might work, but I have no good idea what to do
> when BIOS does handoff with x2APIC enabled and directs us to not
> enable it.  Switching to xAPIC mode is not an option, I suspect.

The specification describes a way to transition from x2APIC to xAPIC mode, but
it's not trivial, because it requires checking all CPU IDs and disabling CPUs
with too high ones.

Personally, I am not comfortable with your patch.  First, it seems that the
patch does not cover hw.x2apic_enable=0.  And I don't like that we leave x2APIC
mode enabled even when we otherwise would not enable it just because BIOS.

All in all, if we can't go x2APIC -> xAPIC, then I would prefer a meaningful
panic over a "compromise".

Just my opinion.  The topic probably warrants a wider discussion and a proper
review.

Slawa,
thank you for your persistence and testing.

-- 
Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: X2APIC support

[Konstantin Belousov]

On Wed, Sep 14, 2016 at 05:02:21PM +0300, Andriy Gapon wrote:
> On 14/09/2016 15:49, Slawa Olhovchenkov wrote:
> > MSR_APICBASE = 0xfee00d00
> > x2APIC is prohibited but turned on by BIOS
> 
> Kostik, ^

Well, the following might work, but I have no good idea what to do
when BIOS does handoff with x2APIC enabled and directs us to not
enable it.  Switching to xAPIC mode is not an option, I suspect.

diff --git a/sys/x86/acpica/madt.c b/sys/x86/acpica/madt.c
index 241a769..3a93fd6 100644
--- a/sys/x86/acpica/madt.c
+++ b/sys/x86/acpica/madt.c
@@ -138,7 +138,6 @@ madt_setup_local(void)
 
madt = pmap_mapbios(madt_physaddr, madt_length);
if ((cpu_feature2 & CPUID2_X2APIC) != 0) {
-   x2apic_mode = 1;
reason = NULL;
 
/*
@@ -150,21 +149,17 @@ madt_setup_local(void)
if (dmartbl_physaddr != 0) {
dmartbl = acpi_map_table(dmartbl_physaddr,
ACPI_SIG_DMAR);
-   if ((dmartbl->Flags & ACPI_DMAR_X2APIC_OPT_OUT) != 0) {
-   x2apic_mode = 0;
+   if ((dmartbl->Flags & ACPI_DMAR_X2APIC_OPT_OUT) != 0)
reason = "by DMAR table";
-   }
acpi_unmap_table(dmartbl);
}
if (vm_guest == VM_GUEST_VMWARE) {
vmware_hvcall(VMW_HVCMD_GETVCPU_INFO, p);
if ((p[0] & VMW_VCPUINFO_VCPU_RESERVED) != 0 ||
-   (p[0] & VMW_VCPUINFO_LEGACY_X2APIC) == 0) {
-   x2apic_mode = 0;
-   reason = "inside VMWare without intr redirection";
-   }
+   (p[0] & VMW_VCPUINFO_LEGACY_X2APIC) == 0)
+   reason =
+   "inside VMWare without intr redirection";
} else if (vm_guest == VM_GUEST_XEN) {
-   x2apic_mode = 0;
reason = "due to running under XEN";
} else if (vm_guest == VM_GUEST_NO &&
CPUID_TO_FAMILY(cpu_id) == 0x6 &&
@@ -184,13 +179,21 @@ madt_setup_local(void)
if (!strcmp(hw_vendor, "LENOVO") ||
!strcmp(hw_vendor,
"ASUSTeK Computer Inc.")) {
-   x2apic_mode = 0;
reason =
"for a suspected SandyBridge BIOS bug";
}
freeenv(hw_vendor);
}
}
+   if (reason != NULL && lapic_is_x2apic()) {
+   if (bootverbose)
+   printf("x2APIC should be disabled %s but "
+   "already enabled by BIOS; enabling.\n",
+reason);
+   reason = NULL;
+   }
+   if (reason == NULL)
+   x2apic_mode = 1;
TUNABLE_INT_FETCH("hw.x2apic_enable", _mode);
if (!x2apic_mode && reason != NULL && bootverbose)
printf("x2APIC available but disabled %s\n", reason);
diff --git a/sys/x86/include/apicvar.h b/sys/x86/include/apicvar.h
index 1ddb69e..09c3a63 100644
--- a/sys/x86/include/apicvar.h
+++ b/sys/x86/include/apicvar.h
@@ -206,6 +206,7 @@ struct apic_ops {
void(*create)(u_int, int);
void(*init)(vm_paddr_t);
void(*xapic_mode)(void);
+   bool(*is_x2apic)(void);
void(*setup)(int);
void(*dump)(const char *);
void(*disable)(void);
@@ -268,6 +269,13 @@ lapic_xapic_mode(void)
apic_ops.xapic_mode();
 }
 
+static inline bool
+lapic_is_x2apic(void)
+{
+
+   return (apic_ops.is_x2apic());
+}
+
 static inline void
 lapic_setup(int boot)
 {
diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c
index cd774df..d9a3453 100644
--- a/sys/x86/x86/local_apic.c
+++ b/sys/x86/x86/local_apic.c
@@ -269,6 +269,16 @@ native_lapic_enable_x2apic(void)
wrmsr(MSR_APICBASE, apic_base);
 }
 
+static bool
+native_lapic_is_x2apic(void)
+{
+   uint64_t apic_base;
+
+   apic_base = rdmsr(MSR_APICBASE);
+   return ((apic_base & (APICBASE_X2APIC | APICBASE_ENABLED)) ==
+   (APICBASE_X2APIC | APICBASE_ENABLED));
+}
+
 static voidlapic_enable(void);
 static voidlapic_resume(struct pic *pic, bool suspend_cancelled);
 static voidlapic_timer_oneshot(struct lapic *);
@@ -329,6 +339,7 @@ struct apic_ops apic_ops = {
.create = native_lapic_create,
.init   = native_lapic_init,
.xapic_mode = native_lapic_xapic_mode,
+   .is_x2apic  = 

Re: X2APIC support

[Andriy Gapon]

On 14/09/2016 15:49, Slawa Olhovchenkov wrote:
> MSR_APICBASE = 0xfee00d00
> x2APIC is prohibited but turned on by BIOS

Kostik, ^

P.S. the format string for the value should have been 0x%016jx.

-- 
Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Jenkins build is back to stable : FreeBSD_stable_10 #398

[jenkins-admin]

See 

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: X2APIC support

[Slawa Olhovchenkov]

On Wed, Sep 14, 2016 at 03:35:39PM +0300, Andriy Gapon wrote:

> On 14/09/2016 15:33, Slawa Olhovchenkov wrote:
> > On Wed, Sep 14, 2016 at 03:22:17PM +0300, Andriy Gapon wrote:
> > 
> >> On 14/09/2016 14:36, Konstantin Belousov wrote:
> >>> On Tue, Sep 13, 2016 at 06:52:19PM +0300, Andriy Gapon wrote:
>  On 13/09/2016 18:22, Konstantin Belousov wrote:
> > Any access
> > to the LAPIC registers page in x2APIC mode faults.
> 
>  Is this a fact?
>  I read the following in the specification:
> 
>    In x2APIC mode, the memory mapped interface is not available and any
>    access to the MMIO interface will behave similar to that of a legacy 
>  xAPIC
>    in globally disabled state.
> 
>  But I couldn't find what actually happens for the legacy xAPIC in 
>  globally
>  disabled state.  For AMD processors it is documented that if xAPIC is 
>  disabled
>  then accessing the APIC memory range works the same as accessing the 
>  regular
>  memory.  That can be different for Intel processors, of course.
> >>>
> >>> Look at the native_lapic_init(), very beginning of the function.  If
> >>> x2apic_mode is non-zero, lapic_map is cleared after DMAP page cache
> >>> attributes are changed.
> >>
> >> Right, but we are talking about the case where x2apic_mode *is zero* while 
> >> the
> >> hardware in x2APIC mode.
> >>
> >>> Right now, if BIOS pass control to OS in x2APIC mode, thinks just work
> >>> because no LAPIC accesses are done until native_lapic_enable_x2apic() is
> >>> called. This just happens, I thought about more organized receipt of the
> >>> current LAPIC mode. Issue is that decision for LAPIC mode is performed
> >>> by madt enumerator which pref. should not read LAPIC config (too early).
> >>> And it is not clear at all, what to do if there is reason to use xAPIC,
> >>> as checked by madt_setup_local(), but we are in x2APIC mode already.
> >>
> >> Yes.  As I mentioned earlier we should at least panic with an informative
> >> message.  Maybe we could add some code to so something smarter later.
> >>
> >>> Anyway, returning to the original issue, I do not believe that the
> >>> hand-off on that machine occured in the x2APIC mode when X2APIC_OPT_OUT
> >>> is specified. Kernel would fault in that case.
> >>
> >> I think that it should be easy to check that if Slawa is still willing to 
> >> try
> >> another diagnostic patch.
> > 
> > What combination of BIOS setting need?
> 
> The one that causes the crash.

VT(vga): text 80x25
CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (2200.05-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x406f1  Family=0x6  Model=0x4f  Stepping=1
  
Features=0xbfebfbff
  
Features2=0x7ffefbff
  AMD Features=0x2c100800
  AMD Features2=0x121
  Structured Extended 
Features=0x21cbfbb
  XSAVE Features=0x1
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
real memory  = 137438953472 (131072 MB)
avail memory = 133407973376 (127227 MB)
MSR_APICBASE = 0xfee00d00
x2APIC is prohibited but turned on by BIOS
Event timer "LAPIC" quality 600
ACPI APIC Table: 
boot_cpu_id = 255
kernel trap 12 with interrupts disabled


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = ff
fault virtual address   = 0x0
fault code  = supervisor read data, page not present
instruction pointer = 0x20:0x80537e74
stack pointer   = 0x28:0x814b3a60
frame pointer   = 0x28:0x814b3a70
code segment= base 0x0, limit 0xf, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags= resume, IOPL = 0
current process = 0 ()
trap number = 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0x805272e7 at kdb_backtrace+0x67
#1 0x804dd662 at vpanic+0x182
#2 0x804dd4d3 at panic+0x43
#3 0x807a37a1 at trap_fatal+0x351
#4 0x807a3993 at trap_pfault+0x1e3
#5 0x807a2f1c at trap+0x26c
#6 0x80787ca1 at calltrap+0x8
#7 0x8083b57a at topo_probe+0x61a
#8 0x8078fe93 at cpu_mp_start+0x1c3
#9 0x805382ca at mp_start+0x3a
#10 0x80465cd8 at mi_startup+0x118
#11 0x8028dfac at btext+0x2c
Uptime: 1s

> >> diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c
> >> index 203e9d00e8acc..37ac03fb9d811 100644
> >> --- a/sys/x86/x86/local_apic.c
> >> +++ b/sys/x86/x86/local_apic.c
> >> @@ -429,6 +429,12 @@ native_lapic_init(vm_paddr_t addr)
> >>if 

Re: X2APIC support

[Andriy Gapon]

On 14/09/2016 15:33, Slawa Olhovchenkov wrote:
> On Wed, Sep 14, 2016 at 03:22:17PM +0300, Andriy Gapon wrote:
> 
>> On 14/09/2016 14:36, Konstantin Belousov wrote:
>>> On Tue, Sep 13, 2016 at 06:52:19PM +0300, Andriy Gapon wrote:
 On 13/09/2016 18:22, Konstantin Belousov wrote:
> Any access
> to the LAPIC registers page in x2APIC mode faults.

 Is this a fact?
 I read the following in the specification:

   In x2APIC mode, the memory mapped interface is not available and any
   access to the MMIO interface will behave similar to that of a legacy 
 xAPIC
   in globally disabled state.

 But I couldn't find what actually happens for the legacy xAPIC in globally
 disabled state.  For AMD processors it is documented that if xAPIC is 
 disabled
 then accessing the APIC memory range works the same as accessing the 
 regular
 memory.  That can be different for Intel processors, of course.
>>>
>>> Look at the native_lapic_init(), very beginning of the function.  If
>>> x2apic_mode is non-zero, lapic_map is cleared after DMAP page cache
>>> attributes are changed.
>>
>> Right, but we are talking about the case where x2apic_mode *is zero* while 
>> the
>> hardware in x2APIC mode.
>>
>>> Right now, if BIOS pass control to OS in x2APIC mode, thinks just work
>>> because no LAPIC accesses are done until native_lapic_enable_x2apic() is
>>> called. This just happens, I thought about more organized receipt of the
>>> current LAPIC mode. Issue is that decision for LAPIC mode is performed
>>> by madt enumerator which pref. should not read LAPIC config (too early).
>>> And it is not clear at all, what to do if there is reason to use xAPIC,
>>> as checked by madt_setup_local(), but we are in x2APIC mode already.
>>
>> Yes.  As I mentioned earlier we should at least panic with an informative
>> message.  Maybe we could add some code to so something smarter later.
>>
>>> Anyway, returning to the original issue, I do not believe that the
>>> hand-off on that machine occured in the x2APIC mode when X2APIC_OPT_OUT
>>> is specified. Kernel would fault in that case.
>>
>> I think that it should be easy to check that if Slawa is still willing to try
>> another diagnostic patch.
> 
> What combination of BIOS setting need?

The one that causes the crash.

>> diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c
>> index 203e9d00e8acc..37ac03fb9d811 100644
>> --- a/sys/x86/x86/local_apic.c
>> +++ b/sys/x86/x86/local_apic.c
>> @@ -429,6 +429,12 @@ native_lapic_init(vm_paddr_t addr)
>>  if (x2apic_mode) {
>>  native_lapic_enable_x2apic();
>>  lapic_map = NULL;
>> +} else if ((cpu_feature2 & CPUID2_X2APIC) != 0) {
>> +r = rdmsr(MSR_APICBASE);
>> +printf("MSR_APICBASE = 0x%16jx\n", (uintmax_t)r);
>> +if ((r & (APICBASE_X2APIC | APICBASE_ENABLED)) ==
>> +(APICBASE_X2APIC | APICBASE_ENABLED))
>> +printf("x2APIC is prohibited but turned on by BIOS\n");
>>  }
>>
>>  /* Setup the spurious interrupt handler. */
>>
>>
>>
>> -- 
>> Andriy Gapon


-- 
Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: X2APIC support

[Slawa Olhovchenkov]

On Wed, Sep 14, 2016 at 03:22:17PM +0300, Andriy Gapon wrote:

> On 14/09/2016 14:36, Konstantin Belousov wrote:
> > On Tue, Sep 13, 2016 at 06:52:19PM +0300, Andriy Gapon wrote:
> >> On 13/09/2016 18:22, Konstantin Belousov wrote:
> >>> Any access
> >>> to the LAPIC registers page in x2APIC mode faults.
> >>
> >> Is this a fact?
> >> I read the following in the specification:
> >>
> >>   In x2APIC mode, the memory mapped interface is not available and any
> >>   access to the MMIO interface will behave similar to that of a legacy 
> >> xAPIC
> >>   in globally disabled state.
> >>
> >> But I couldn't find what actually happens for the legacy xAPIC in globally
> >> disabled state.  For AMD processors it is documented that if xAPIC is 
> >> disabled
> >> then accessing the APIC memory range works the same as accessing the 
> >> regular
> >> memory.  That can be different for Intel processors, of course.
> > 
> > Look at the native_lapic_init(), very beginning of the function.  If
> > x2apic_mode is non-zero, lapic_map is cleared after DMAP page cache
> > attributes are changed.
> 
> Right, but we are talking about the case where x2apic_mode *is zero* while the
> hardware in x2APIC mode.
> 
> > Right now, if BIOS pass control to OS in x2APIC mode, thinks just work
> > because no LAPIC accesses are done until native_lapic_enable_x2apic() is
> > called. This just happens, I thought about more organized receipt of the
> > current LAPIC mode. Issue is that decision for LAPIC mode is performed
> > by madt enumerator which pref. should not read LAPIC config (too early).
> > And it is not clear at all, what to do if there is reason to use xAPIC,
> > as checked by madt_setup_local(), but we are in x2APIC mode already.
> 
> Yes.  As I mentioned earlier we should at least panic with an informative
> message.  Maybe we could add some code to so something smarter later.
> 
> > Anyway, returning to the original issue, I do not believe that the
> > hand-off on that machine occured in the x2APIC mode when X2APIC_OPT_OUT
> > is specified. Kernel would fault in that case.
> 
> I think that it should be easy to check that if Slawa is still willing to try
> another diagnostic patch.

What combination of BIOS setting need?

> diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c
> index 203e9d00e8acc..37ac03fb9d811 100644
> --- a/sys/x86/x86/local_apic.c
> +++ b/sys/x86/x86/local_apic.c
> @@ -429,6 +429,12 @@ native_lapic_init(vm_paddr_t addr)
>   if (x2apic_mode) {
>   native_lapic_enable_x2apic();
>   lapic_map = NULL;
> + } else if ((cpu_feature2 & CPUID2_X2APIC) != 0) {
> + r = rdmsr(MSR_APICBASE);
> + printf("MSR_APICBASE = 0x%16jx\n", (uintmax_t)r);
> + if ((r & (APICBASE_X2APIC | APICBASE_ENABLED)) ==
> + (APICBASE_X2APIC | APICBASE_ENABLED))
> + printf("x2APIC is prohibited but turned on by BIOS\n");
>   }
> 
>   /* Setup the spurious interrupt handler. */
> 
> 
> 
> -- 
> Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: X2APIC support

[Andriy Gapon]

On 13/09/2016 17:59, Slawa Olhovchenkov wrote:
> Goggling on X2APIC_OPT_OUT take same result: other OS in this case
> downgrade to xAPIC mode.

It still does not make any sense for BIOS to turn on x2APIC and then instruct
the OS that x2APIC must not be used.

-- 
Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: X2APIC support

[Andriy Gapon]

On 14/09/2016 14:36, Konstantin Belousov wrote:
> On Tue, Sep 13, 2016 at 06:52:19PM +0300, Andriy Gapon wrote:
>> On 13/09/2016 18:22, Konstantin Belousov wrote:
>>> Any access
>>> to the LAPIC registers page in x2APIC mode faults.
>>
>> Is this a fact?
>> I read the following in the specification:
>>
>>   In x2APIC mode, the memory mapped interface is not available and any
>>   access to the MMIO interface will behave similar to that of a legacy xAPIC
>>   in globally disabled state.
>>
>> But I couldn't find what actually happens for the legacy xAPIC in globally
>> disabled state.  For AMD processors it is documented that if xAPIC is 
>> disabled
>> then accessing the APIC memory range works the same as accessing the regular
>> memory.  That can be different for Intel processors, of course.
> 
> Look at the native_lapic_init(), very beginning of the function.  If
> x2apic_mode is non-zero, lapic_map is cleared after DMAP page cache
> attributes are changed.

Right, but we are talking about the case where x2apic_mode *is zero* while the
hardware in x2APIC mode.

> Right now, if BIOS pass control to OS in x2APIC mode, thinks just work
> because no LAPIC accesses are done until native_lapic_enable_x2apic() is
> called. This just happens, I thought about more organized receipt of the
> current LAPIC mode. Issue is that decision for LAPIC mode is performed
> by madt enumerator which pref. should not read LAPIC config (too early).
> And it is not clear at all, what to do if there is reason to use xAPIC,
> as checked by madt_setup_local(), but we are in x2APIC mode already.

Yes.  As I mentioned earlier we should at least panic with an informative
message.  Maybe we could add some code to so something smarter later.

> Anyway, returning to the original issue, I do not believe that the
> hand-off on that machine occured in the x2APIC mode when X2APIC_OPT_OUT
> is specified. Kernel would fault in that case.

I think that it should be easy to check that if Slawa is still willing to try
another diagnostic patch.

diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c
index 203e9d00e8acc..37ac03fb9d811 100644
--- a/sys/x86/x86/local_apic.c
+++ b/sys/x86/x86/local_apic.c
@@ -429,6 +429,12 @@ native_lapic_init(vm_paddr_t addr)
if (x2apic_mode) {
native_lapic_enable_x2apic();
lapic_map = NULL;
+   } else if ((cpu_feature2 & CPUID2_X2APIC) != 0) {
+   r = rdmsr(MSR_APICBASE);
+   printf("MSR_APICBASE = 0x%16jx\n", (uintmax_t)r);
+   if ((r & (APICBASE_X2APIC | APICBASE_ENABLED)) ==
+   (APICBASE_X2APIC | APICBASE_ENABLED))
+   printf("x2APIC is prohibited but turned on by BIOS\n");
}

/* Setup the spurious interrupt handler. */



-- 
Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: X2APIC support

[Slawa Olhovchenkov]

On Wed, Sep 14, 2016 at 02:36:34PM +0300, Konstantin Belousov wrote:

> On Tue, Sep 13, 2016 at 06:52:19PM +0300, Andriy Gapon wrote:
> > On 13/09/2016 18:22, Konstantin Belousov wrote:
> > > Any access
> > > to the LAPIC registers page in x2APIC mode faults.
> > 
> > Is this a fact?
> > I read the following in the specification:
> > 
> >   In x2APIC mode, the memory mapped interface is not available and any
> >   access to the MMIO interface will behave similar to that of a legacy xAPIC
> >   in globally disabled state.
> > 
> > But I couldn't find what actually happens for the legacy xAPIC in globally
> > disabled state.  For AMD processors it is documented that if xAPIC is 
> > disabled
> > then accessing the APIC memory range works the same as accessing the regular
> > memory.  That can be different for Intel processors, of course.
> 
> Look at the native_lapic_init(), very beginning of the function.  If
> x2apic_mode is non-zero, lapic_map is cleared after DMAP page cache
> attributes are changed.
> 
> Right now, if BIOS pass control to OS in x2APIC mode, thinks just work
> because no LAPIC accesses are done until native_lapic_enable_x2apic() is
> called. This just happens, I thought about more organized receipt of the
> current LAPIC mode. Issue is that decision for LAPIC mode is performed
> by madt enumerator which pref. should not read LAPIC config (too early).
> And it is not clear at all, what to do if there is reason to use xAPIC,
> as checked by madt_setup_local(), but we are in x2APIC mode already.
> 
> Anyway, returning to the original issue, I do not believe that the
> hand-off on that machine occured in the x2APIC mode when X2APIC_OPT_OUT
> is specified. Kernel would fault in that case.

As I assume, other OS don't fault in this case.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: X2APIC support

[Konstantin Belousov]

On Tue, Sep 13, 2016 at 06:52:19PM +0300, Andriy Gapon wrote:
> On 13/09/2016 18:22, Konstantin Belousov wrote:
> > Any access
> > to the LAPIC registers page in x2APIC mode faults.
> 
> Is this a fact?
> I read the following in the specification:
> 
>   In x2APIC mode, the memory mapped interface is not available and any
>   access to the MMIO interface will behave similar to that of a legacy xAPIC
>   in globally disabled state.
> 
> But I couldn't find what actually happens for the legacy xAPIC in globally
> disabled state.  For AMD processors it is documented that if xAPIC is disabled
> then accessing the APIC memory range works the same as accessing the regular
> memory.  That can be different for Intel processors, of course.

Look at the native_lapic_init(), very beginning of the function.  If
x2apic_mode is non-zero, lapic_map is cleared after DMAP page cache
attributes are changed.

Right now, if BIOS pass control to OS in x2APIC mode, thinks just work
because no LAPIC accesses are done until native_lapic_enable_x2apic() is
called. This just happens, I thought about more organized receipt of the
current LAPIC mode. Issue is that decision for LAPIC mode is performed
by madt enumerator which pref. should not read LAPIC config (too early).
And it is not clear at all, what to do if there is reason to use xAPIC,
as checked by madt_setup_local(), but we are in x2APIC mode already.

Anyway, returning to the original issue, I do not believe that the
hand-off on that machine occured in the x2APIC mode when X2APIC_OPT_OUT
is specified. Kernel would fault in that case.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"