Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR->x2apic enablement bit ?
On 12/20/2012 08:46 AM, Yinghai Lu wrote: > On Thu, Dec 20, 2012 at 2:38 AM, Zhang, Lin-Bao (Linux Kernel R) > wrote: >> I think current core argument is our BIOS did following judgement , >> but Linux kernel did not(Linux seems just judge BSP supports x2apic mode or >> not by CPUID method) > > kernel checks that, if the BSP in x2apic mode, kernel will check if > intr-remapping can be enabled, otherwise > it will switch back to xapic mode. > >> Our BSP really supports x2apic mode, however,as this spec, our BIOS think >> some of configuration of server >> could not meet x2apic conditions ,So didn't place local xAPICs into x2apic >> mode. >> But linux kernel indeed enable x2apic mode(just believe BSP, didn't do above >> judgement) . so this is the conflict. > > so your BIOS is wrong, it does not enable intr-remapping. > FWIW, I bought a *server* today and it didn't have interrupt remapping and x2APIC, I would consider it defective and return it. -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR->x2apic enablement bit ?
On Thu, Dec 20, 2012 at 2:38 AM, Zhang, Lin-Bao (Linux Kernel R) wrote: > I think current core argument is our BIOS did following judgement , > but Linux kernel did not(Linux seems just judge BSP supports x2apic mode or > not by CPUID method) kernel checks that, if the BSP in x2apic mode, kernel will check if intr-remapping can be enabled, otherwise it will switch back to xapic mode. > Our BSP really supports x2apic mode, however,as this spec, our BIOS think > some of configuration of server > could not meet x2apic conditions ,So didn't place local xAPICs into x2apic > mode. > But linux kernel indeed enable x2apic mode(just believe BSP, didn't do above > judgement) . so this is the conflict. so your BIOS is wrong, it does not enable intr-remapping. Yinghai -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR->x2apic enablement bit ?
Hi Yinghai,
Maybe two days ago, mis-understand the usage of MSR.
Today, I carefully researched Intel? 64 Architecture x2APIC Specification which
is open for all people.
I think current core argument is our BIOS did following judgement ,
but Linux kernel did not(Linux seems just judge BSP supports x2apic mode or not
by CPUID method)
2.9 SYSTEM TRANSITIONS
This section describes implications for the x2APIC across system state
transitions -
specifically initialization and booting.
The default will be for the BIOS to pass the control to the OS with the local
x2APICs
in xAPIC mode if all x2APIC IDs reported by CPUID.0BH:EDX are less than 255,
and
in x2APIC mode if there are any logical processor reporting its x2APIC ID at
255 or
greater.
Our BSP really supports x2apic mode, however,as this spec, our BIOS think some
of configuration of server
could not meet x2apic conditions ,So didn't place local xAPICs into x2apic
mode.
But linux kernel indeed enable x2apic mode(just believe BSP, didn't do above
judgement) . so this is the conflict.
I carefully read Linux kernel source code, I didn't find related code.
BIOS and OS both can configure and enable local xAPICs into x2APIC mode ,but
they should follow
the same spec . from the spec , we should consider all cores or logical CPUs .
if I lost some source code , sorry first , could you point me these code ?
thanks very much!
Following are some clarifications for old questions. Thanks very much for
Yinghai and others' help.
From spec:
2.2 DETECTING AND ENABLING X 2APIC
A processor’s support to operate its local APIC in the x2APIC mode can be
detected
by querying the extended feature flag information reported by CPUID. When CPUID
is executed with EAX = 1, the returned value in ECX[Bit 21] indicates
processor’s
support for the x2APIC mode. If CPUID.(EAX=01H):ECX[Bit 21] is set, then the
local
APIC in the processor supports the x2APIC capability and can be placed into the
x2APIC mode. This bit is set only when the x2APIC hardware is present.
? System software can place the local APIC in the x2APIC mode by setting the
x2APIC mode enable bit (bit 10) in the IA32_APIC_BASE MSR at MSR address
01BH. The layout for the IA32_APIC_BASE MSR is shown in Figure 2-1.
So ,as above words:
1, CPUID is just executed to query if CPU has supported x2apic mode.
It is read only. It is decided by CPU vendor.
In kernel source , cpu_has_x2apic() should leverage CPUID interface.
2, bit 11 and 10 of IA32_APIC_BASE MSR are used to switch to x2apic (EN=1,
EXTD=1).
So I begin to understand that , OS certainly can control MSR to make CPU enter
x2apic mode.
BIOS also can , OS can also read it and write it as we need.
> -Original Message-
> From: yhlu.ker...@gmail.com [mailto:yhlu.ker...@gmail.com] On Behalf Of
> Yinghai Lu
> Sent: 2012年12月20日 0:06
>
> how does BIOS claim?
>
Sorry for words , BIOS didn't claim , it just didn't set MSR(en=1 , EXTD=1). OS
will read it first.
This just can indicate that before BIOS passing control to OS, local x2APICs
running in xAPIC mode.
> disable that in CPUID?
>
No , CPUID is read only , "When CPUID is executed with EAX = 1, the returned
value in ECX[Bit 21] indicates processor’s
support for the x2APIC mode."
BIOS just will query ,can set. OS also did like this.
N�Р骒r��yb�X�肚�v�^�)藓{.n�+�伐�{��赙zXФ�≤�}��财�z�:+v�����赙zZ+��+zf＂�h���~i���z� �wア�?�ㄨ��&�)撷f��^j谦y�m��@A�a囤�
0鹅h���i
RE: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR-x2apic enablement bit ?
Hi Yinghai,
Maybe two days ago, mis-understand the usage of MSR.
Today, I carefully researched Intel? 64 Architecture x2APIC Specification which
is open for all people.
I think current core argument is our BIOS did following judgement ,
but Linux kernel did not(Linux seems just judge BSP supports x2apic mode or not
by CPUID method)
2.9 SYSTEM TRANSITIONS
This section describes implications for the x2APIC across system state
transitions -
specifically initialization and booting.
The default will be for the BIOS to pass the control to the OS with the local
x2APICs
in xAPIC mode if all x2APIC IDs reported by CPUID.0BH:EDX are less than 255,
and
in x2APIC mode if there are any logical processor reporting its x2APIC ID at
255 or
greater.
Our BSP really supports x2apic mode, however,as this spec, our BIOS think some
of configuration of server
could not meet x2apic conditions ,So didn't place local xAPICs into x2apic
mode.
But linux kernel indeed enable x2apic mode(just believe BSP, didn't do above
judgement) . so this is the conflict.
I carefully read Linux kernel source code, I didn't find related code.
BIOS and OS both can configure and enable local xAPICs into x2APIC mode ,but
they should follow
the same spec . from the spec , we should consider all cores or logical CPUs .
if I lost some source code , sorry first , could you point me these code ?
thanks very much!
Following are some clarifications for old questions. Thanks very much for
Yinghai and others' help.
From spec:
2.2 DETECTING AND ENABLING X 2APIC
A processor’s support to operate its local APIC in the x2APIC mode can be
detected
by querying the extended feature flag information reported by CPUID. When CPUID
is executed with EAX = 1, the returned value in ECX[Bit 21] indicates
processor’s
support for the x2APIC mode. If CPUID.(EAX=01H):ECX[Bit 21] is set, then the
local
APIC in the processor supports the x2APIC capability and can be placed into the
x2APIC mode. This bit is set only when the x2APIC hardware is present.
? System software can place the local APIC in the x2APIC mode by setting the
x2APIC mode enable bit (bit 10) in the IA32_APIC_BASE MSR at MSR address
01BH. The layout for the IA32_APIC_BASE MSR is shown in Figure 2-1.
So ,as above words:
1, CPUID is just executed to query if CPU has supported x2apic mode.
It is read only. It is decided by CPU vendor.
In kernel source , cpu_has_x2apic() should leverage CPUID interface.
2, bit 11 and 10 of IA32_APIC_BASE MSR are used to switch to x2apic (EN=1,
EXTD=1).
So I begin to understand that , OS certainly can control MSR to make CPU enter
x2apic mode.
BIOS also can , OS can also read it and write it as we need.
-Original Message-
From: yhlu.ker...@gmail.com [mailto:yhlu.ker...@gmail.com] On Behalf Of
Yinghai Lu
Sent: 2012年12月20日 0:06
how does BIOS claim?
Sorry for words , BIOS didn't claim , it just didn't set MSR(en=1 , EXTD=1). OS
will read it first.
This just can indicate that before BIOS passing control to OS, local x2APICs
running in xAPIC mode.
disable that in CPUID?
No , CPUID is read only , When CPUID is executed with EAX = 1, the returned
value in ECX[Bit 21] indicates processor’s
support for the x2APIC mode.
BIOS just will query ,can set. OS also did like this.
N�Р骒r��yb�X�肚�v�^�)藓{.n�+�伐�{��赙zXФ�≤�}��财�z�j:+v�����赙zZ+��+zf＂�h���~i���z� �wア�?�ㄨ���)撷f��^j谦y�m��@A�a囤�
0鹅h���i
Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR-x2apic enablement bit ?
On Thu, Dec 20, 2012 at 2:38 AM, Zhang, Lin-Bao (Linux Kernel RD) linbao.zh...@hp.com wrote: I think current core argument is our BIOS did following judgement , but Linux kernel did not(Linux seems just judge BSP supports x2apic mode or not by CPUID method) kernel checks that, if the BSP in x2apic mode, kernel will check if intr-remapping can be enabled, otherwise it will switch back to xapic mode. Our BSP really supports x2apic mode, however,as this spec, our BIOS think some of configuration of server could not meet x2apic conditions ,So didn't place local xAPICs into x2apic mode. But linux kernel indeed enable x2apic mode(just believe BSP, didn't do above judgement) . so this is the conflict. so your BIOS is wrong, it does not enable intr-remapping. Yinghai -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR-x2apic enablement bit ?
On 12/20/2012 08:46 AM, Yinghai Lu wrote: On Thu, Dec 20, 2012 at 2:38 AM, Zhang, Lin-Bao (Linux Kernel RD) linbao.zh...@hp.com wrote: I think current core argument is our BIOS did following judgement , but Linux kernel did not(Linux seems just judge BSP supports x2apic mode or not by CPUID method) kernel checks that, if the BSP in x2apic mode, kernel will check if intr-remapping can be enabled, otherwise it will switch back to xapic mode. Our BSP really supports x2apic mode, however,as this spec, our BIOS think some of configuration of server could not meet x2apic conditions ,So didn't place local xAPICs into x2apic mode. But linux kernel indeed enable x2apic mode(just believe BSP, didn't do above judgement) . so this is the conflict. so your BIOS is wrong, it does not enable intr-remapping. FWIW, I bought a *server* today and it didn't have interrupt remapping and x2APIC, I would consider it defective and return it. -hpa -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR->x2apic enablement bit ?
On 12/18/2012 09:49 AM, Yinghai Lu wrote: > > So you need to make cpuid show does not support x2apic --- check with > intel they have way to do that. > No, but there is a flag in ACPI that "this BIOS is broken with x2APIC". Setting that flag as opposed to fixing the bugs, though is very much not recommended, however; the lack of interrupt routing means there are ways for a rogue device to compromise the security provided by VT-d, see: http://www.invisiblethingslab.com/resources/2011/Software%20Attacks%20on%20Intel%20VT-d.pdf -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR->x2apic enablement bit ?
On Wed, Dec 19, 2012 at 2:36 AM, Zhang, Lin-Bao (Linux Kernel R) wrote: > Hi Yinghai , > > Actually ,my question like this: > OS can really enable x2apic correctly without BIOS' help in this case : > BIOS has claimed definitely it doesn't enable x2apic ,but OS feel it can > enable x2apic(based on BSP supports x2apic) ). how does BIOS cleaim? disable that in CPUID? > > If Linux don't need BIOS' help , how did linux kernel reach this action? need DMAR table to enable intr remapping. --- ioapic only have 8 bits for dst. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR->x2apic enablement bit ?
On Wed, Dec 19, 2012 at 2:13 AM, Zhang, Lin-Bao (Linux Kernel R) wrote: >> > b) If BIOS feel the system doesn't meet x2apic conditions , it will not set >> x2apic enablement bit in MSR ,and pass control to OS with xapic mode. >> >> kernel will check if cpuid support x2apic, if it supports x2apic, it will >> check if >> DMAR/intr-remapping could be enabled, if so kernel will switch to x2apic. >> otherwise it will stay with xapic. >> >> So you need to make cpuid show does not support x2apic --- check with intel >> they have way to do that. > My concerns is BIOS creates different ACPI tables in xAPIC mode and X2APIC > mode. > BIOS will auto detect if the system meets x2APIC condition , if no , it will > not create right ACPI tables for this mode and don't set MSR bit. > For this case (BIOS didn't set MSR , but BSP supports x2apic ) , I want to > know > 1, For Linux kernel, which ACPI tables will be used for x2apic and XAPIC ? two MADT? why two? according to Intel, if your apic id < 255, even cpu is in x2apic state already, you only have xapic entry in MADT. > 2, why Linux kernel will set x2apic enablement bit in CPU msr ? generally > speaking , BIOS set it and OS get it. > If OS get it , who will read it ? BIOS ? I consult BIOS designer , he said , > their BIOS calling service(just like UEFI run time service) > will also judge MSR again , so I don't know why Linux kernel will set MSR bit > , unless OS has special target . because os does not trust BIOS. BIOS ONLY SHOULD only do thing it should do: 1. initialize memory. 2. assign some resource to pci devices bar 3. provide memory mem-map to bootloader and os. If it don't want OS to use x2apic, disable that in CPUID. > >> or you can pass "nox2apic" in boot command line. > Yeah, I think this would be fine if we really want to use xAPIC mode instead > of x2apic mode. > But I am puzzled why Linux kernel will force to enter x2apic just depends on > BSP has supported x2apic ? > "BSP supports x2apic" condition is enough for judging if OS can enable x2apic > ? if system support x2apic, os will try use x2apic at first, unless BIOS mess it up. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR->x2apic enablement bit ?
Hi Yinghai , Actually ,my question like this: OS can really enable x2apic correctly without BIOS' help in this case : BIOS has claimed definitely it doesn't enable x2apic ,but OS feel it can enable x2apic(based on BSP supports x2apic) ). If Linux don't need BIOS' help , how did linux kernel reach this action? I am reading this related code, I just see Linux kernel judge cpu_has_x2apic() ,if yes, it will set MSR, certainly , IR works right. We can think BIOS' supporting is not necessary ? Thanks very much . -- Bob(LinBao Zhang) HP linux kernel enginner -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR->x2apic enablement bit ?
Hi Yinghai , Thanks very much for your reply. I also checked some other previous emails about x2apic patches. > From: yhlu.ker...@gmail.com [mailto:yhlu.ker...@gmail.com] On Behalf Of > Yinghai Lu I just worry about this case, > > b) If BIOS feel the system doesn't meet x2apic conditions , it will not set > x2apic enablement bit in MSR ,and pass control to OS with xapic mode. > > kernel will check if cpuid support x2apic, if it supports x2apic, it will > check if > DMAR/intr-remapping could be enabled, if so kernel will switch to x2apic. > otherwise it will stay with xapic. > > So you need to make cpuid show does not support x2apic --- check with intel > they have way to do that. My concerns is BIOS creates different ACPI tables in xAPIC mode and X2APIC mode. BIOS will auto detect if the system meets x2APIC condition , if no , it will not create right ACPI tables for this mode and don't set MSR bit. For this case (BIOS didn't set MSR , but BSP supports x2apic ) , I want to know 1, For Linux kernel, which ACPI tables will be used for x2apic and XAPIC ? 2, why Linux kernel will set x2apic enablement bit in CPU msr ? generally speaking , BIOS set it and OS get it. If OS get it , who will read it ? BIOS ? I consult BIOS designer , he said , their BIOS calling service(just like UEFI run time service) will also judge MSR again , so I don't know why Linux kernel will set MSR bit , unless OS has special target . > or you can pass "nox2apic" in boot command line. Yeah, I think this would be fine if we really want to use xAPIC mode instead of x2apic mode. But I am puzzled why Linux kernel will force to enter x2apic just depends on BSP has supported x2apic ? "BSP supports x2apic" condition is enough for judging if OS can enable x2apic ? Thanks very much ! -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR-x2apic enablement bit ?
Hi Yinghai , Thanks very much for your reply. I also checked some other previous emails about x2apic patches. From: yhlu.ker...@gmail.com [mailto:yhlu.ker...@gmail.com] On Behalf Of Yinghai Lu I just worry about this case, b) If BIOS feel the system doesn't meet x2apic conditions , it will not set x2apic enablement bit in MSR ,and pass control to OS with xapic mode. kernel will check if cpuid support x2apic, if it supports x2apic, it will check if DMAR/intr-remapping could be enabled, if so kernel will switch to x2apic. otherwise it will stay with xapic. So you need to make cpuid show does not support x2apic --- check with intel they have way to do that. My concerns is BIOS creates different ACPI tables in xAPIC mode and X2APIC mode. BIOS will auto detect if the system meets x2APIC condition , if no , it will not create right ACPI tables for this mode and don't set MSR bit. For this case (BIOS didn't set MSR , but BSP supports x2apic ) , I want to know 1, For Linux kernel, which ACPI tables will be used for x2apic and XAPIC ? 2, why Linux kernel will set x2apic enablement bit in CPU msr ? generally speaking , BIOS set it and OS get it. If OS get it , who will read it ? BIOS ? I consult BIOS designer , he said , their BIOS calling service(just like UEFI run time service) will also judge MSR again , so I don't know why Linux kernel will set MSR bit , unless OS has special target . or you can pass nox2apic in boot command line. Yeah, I think this would be fine if we really want to use xAPIC mode instead of x2apic mode. But I am puzzled why Linux kernel will force to enter x2apic just depends on BSP has supported x2apic ? BSP supports x2apic condition is enough for judging if OS can enable x2apic ? Thanks very much ! -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR-x2apic enablement bit ?
Hi Yinghai , Actually ,my question like this: OS can really enable x2apic correctly without BIOS' help in this case : BIOS has claimed definitely it doesn't enable x2apic ,but OS feel it can enable x2apic(based on BSP supports x2apic) ). If Linux don't need BIOS' help , how did linux kernel reach this action? I am reading this related code, I just see Linux kernel judge cpu_has_x2apic() ,if yes, it will set MSR, certainly , IR works right. We can think BIOS' supporting is not necessary ? Thanks very much . -- Bob(LinBao Zhang) HP linux kernel enginner -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR-x2apic enablement bit ?
On Wed, Dec 19, 2012 at 2:13 AM, Zhang, Lin-Bao (Linux Kernel RD) linbao.zh...@hp.com wrote: b) If BIOS feel the system doesn't meet x2apic conditions , it will not set x2apic enablement bit in MSR ,and pass control to OS with xapic mode. kernel will check if cpuid support x2apic, if it supports x2apic, it will check if DMAR/intr-remapping could be enabled, if so kernel will switch to x2apic. otherwise it will stay with xapic. So you need to make cpuid show does not support x2apic --- check with intel they have way to do that. My concerns is BIOS creates different ACPI tables in xAPIC mode and X2APIC mode. BIOS will auto detect if the system meets x2APIC condition , if no , it will not create right ACPI tables for this mode and don't set MSR bit. For this case (BIOS didn't set MSR , but BSP supports x2apic ) , I want to know 1, For Linux kernel, which ACPI tables will be used for x2apic and XAPIC ? two MADT? why two? according to Intel, if your apic id 255, even cpu is in x2apic state already, you only have xapic entry in MADT. 2, why Linux kernel will set x2apic enablement bit in CPU msr ? generally speaking , BIOS set it and OS get it. If OS get it , who will read it ? BIOS ? I consult BIOS designer , he said , their BIOS calling service(just like UEFI run time service) will also judge MSR again , so I don't know why Linux kernel will set MSR bit , unless OS has special target . because os does not trust BIOS. BIOS ONLY SHOULD only do thing it should do: 1. initialize memory. 2. assign some resource to pci devices bar 3. provide memory mem-map to bootloader and os. If it don't want OS to use x2apic, disable that in CPUID. or you can pass nox2apic in boot command line. Yeah, I think this would be fine if we really want to use xAPIC mode instead of x2apic mode. But I am puzzled why Linux kernel will force to enter x2apic just depends on BSP has supported x2apic ? BSP supports x2apic condition is enough for judging if OS can enable x2apic ? if system support x2apic, os will try use x2apic at first, unless BIOS mess it up. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR-x2apic enablement bit ?
On Wed, Dec 19, 2012 at 2:36 AM, Zhang, Lin-Bao (Linux Kernel RD) linbao.zh...@hp.com wrote: Hi Yinghai , Actually ,my question like this: OS can really enable x2apic correctly without BIOS' help in this case : BIOS has claimed definitely it doesn't enable x2apic ,but OS feel it can enable x2apic(based on BSP supports x2apic) ). how does BIOS cleaim? disable that in CPUID? If Linux don't need BIOS' help , how did linux kernel reach this action? need DMAR table to enable intr remapping. --- ioapic only have 8 bits for dst. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR-x2apic enablement bit ?
On 12/18/2012 09:49 AM, Yinghai Lu wrote: So you need to make cpuid show does not support x2apic --- check with intel they have way to do that. No, but there is a flag in ACPI that this BIOS is broken with x2APIC. Setting that flag as opposed to fixing the bugs, though is very much not recommended, however; the lack of interrupt routing means there are ways for a rogue device to compromise the security provided by VT-d, see: http://www.invisiblethingslab.com/resources/2011/Software%20Attacks%20on%20Intel%20VT-d.pdf -hpa -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR->x2apic enablement bit ?
On Tue, Dec 18, 2012 at 1:50 AM, Zhang, Lin-Bao (Linux Kernel R) wrote: > Hi Suresh and other guys , > > In 3.4.4/3.6.6 ,I found a x2apic issue . if I am wrong , sorry first > , and welcome your correction . thanks for your forwarding other maintainers. > I am testing a server , its BIOS is like this: > a) If BIOS think the system is of x2apic , it will set x2apic enablement bit > in MSR and create x2apic ACPI tables and pass control to OS with x2apic mode kernel will check if DMAR table is right or not, if intr-remapping could be enabled, kernel will stay with x2apic. if intr-remapping can not be enabled (bad dmar table), kernel will switch back to xapic mode, but if BSP's X2APIC is bigger than 255, then you will get kernel panic. > b) If BIOS feel the system doesn't meet x2apic conditions , it will not set > x2apic enablement bit in MSR ,and pass control to OS with xapic mode. kernel will check if cpuid support x2apic, if it supports x2apic, it will check if DMAR/intr-remapping could be enabled, if so kernel will switch to x2apic. otherwise it will stay with xapic. So you need to make cpuid show does not support x2apic --- check with intel they have way to do that. or you can pass "nox2apic" in boot command line. Yinghai -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR->x2apic enablement bit ?
Hi Suresh and other guys ,
In 3.4.4/3.6.6 ,I found a x2apic issue . if I am wrong , sorry first ,
and welcome your correction . thanks for your forwarding other maintainers.
I am testing a server , its BIOS is like this:
a) If BIOS think the system is of x2apic , it will set x2apic enablement bit in
MSR and create x2apic ACPI tables and pass control to OS with x2apic mode
b) If BIOS feel the system doesn't meet x2apic conditions , it will not set
x2apic enablement bit in MSR ,and pass control to OS with xapic mode.
It seems that MSR should be interface to OS to use. OS should leverage this,
but it seems not.
In linux kernel source code ,
We see:
void check_x2apic(void)
{
if (x2apic_enabled()) { // this depends on 2 conditions : cpu has
supported x2apic or not, bios setting MSR bit or not .
pr_info("x2apic enabled by BIOS, switching to x2apic ops\n");
x2apic_preenabled = x2apic_mode = 1;
}
It seems that Linux kernel will follow BIOS' result in early booting.
But I found that in other source : enable_IR_x2apic() , linux kernel obviously
discard BIOS' result like this:
#define x2apic_supported() (cpu_has_x2apic) // just depends on if
booting CPU has supported x2apic
Enable_IR_x2apic {
x2apic_enabled = 1;
if (x2apic_supported() && !x2apic_mode) { //x2apic_mode depends on
BIOS's MSR.
x2apic_mode = 1;
enable_x2apic();// OS
will write MSR->x2apic enablement bit and print
pr_info("Enabled x2apic\n");
}
..
}
void enable_x2apic(void)
{
u64 msr;
rdmsrl(MSR_IA32_APICBASE, msr);
if (x2apic_disabled) {
__disable_x2apic(msr);
return;
}
if (!x2apic_mode)
return;
if (!(msr & X2APIC_ENABLE)) {
printk_once(KERN_INFO "Enabling x2apic\n");
// linux kernel will write MSR->x2apic_enable ,work around BIOS
, this is reasonable ?
wrmsrl(MSR_IA32_APICBASE, msr | X2APIC_ENABLE);
}
}
I am very surprised for this ,why BIOS has claimed that BIOS didn't support
x2apic , but Linux kernel will force enabling x2apic
(although booting CPU has supported x2apic) ?
I have one machine , in booting log ,
There is no " x2apic enabled by BIOS, switching to x2apic ops "
But later , linux kernel will print
" [0.464649] parse_iosapics: ecap f0207e
[0.469728] IOAPIC id 8 under DRHD base 0xa800 IU 0
[0.576731] IOAPIC id 0 under DRHD base 0xa800 IOMMU 0
[0.584065] parse_io_apic: io_apic 2 nr 2
[0.589590] Enabled IRQ remapping in x2apic mode
[0.595663] Enabling x2apic
[0.599337] Enabled x2apic
[0.603007] Switched APIC routing to cluster x2apic." ( I can add
x2apic_phys to force using physical mode).
1, I quite worry about this , try to think , BIOS didn't enable x2apic , it
just did enable xAPIC mode ,
it didn't create ACPI tables of x2apic. In this case, how Linux kernel can run
normally in x2apic mode ?
in fact, in this machine , Linux kernel didn't run into some issues ,but I
still think this is a potential defect.
2, why Linux kernel first depends on MSR and later will discard MSR ? is there
any history ?
Linux kernel wants to completely discard MSR checking in future ?
3, anybody test machines which support x2apic ? do you have similar problems ?
in these machines , will you check MSR bit ?
-- Bob(LinBao Zhang)
HP linux kernel enginner
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR-x2apic enablement bit ?
Hi Suresh and other guys ,
In 3.4.4/3.6.6 ,I found a x2apic issue . if I am wrong , sorry first ,
and welcome your correction . thanks for your forwarding other maintainers.
I am testing a server , its BIOS is like this:
a) If BIOS think the system is of x2apic , it will set x2apic enablement bit in
MSR and create x2apic ACPI tables and pass control to OS with x2apic mode
b) If BIOS feel the system doesn't meet x2apic conditions , it will not set
x2apic enablement bit in MSR ,and pass control to OS with xapic mode.
It seems that MSR should be interface to OS to use. OS should leverage this,
but it seems not.
In linux kernel source code ,
We see:
void check_x2apic(void)
{
if (x2apic_enabled()) { // this depends on 2 conditions : cpu has
supported x2apic or not, bios setting MSR bit or not .
pr_info(x2apic enabled by BIOS, switching to x2apic ops\n);
x2apic_preenabled = x2apic_mode = 1;
}
It seems that Linux kernel will follow BIOS' result in early booting.
But I found that in other source : enable_IR_x2apic() , linux kernel obviously
discard BIOS' result like this:
#define x2apic_supported() (cpu_has_x2apic) // just depends on if
booting CPU has supported x2apic
Enable_IR_x2apic {
x2apic_enabled = 1;
if (x2apic_supported() !x2apic_mode) { //x2apic_mode depends on
BIOS's MSR.
x2apic_mode = 1;
enable_x2apic();// OS
will write MSR-x2apic enablement bit and print
pr_info(Enabled x2apic\n);
}
..
}
void enable_x2apic(void)
{
u64 msr;
rdmsrl(MSR_IA32_APICBASE, msr);
if (x2apic_disabled) {
__disable_x2apic(msr);
return;
}
if (!x2apic_mode)
return;
if (!(msr X2APIC_ENABLE)) {
printk_once(KERN_INFO Enabling x2apic\n);
// linux kernel will write MSR-x2apic_enable ,work around BIOS
, this is reasonable ?
wrmsrl(MSR_IA32_APICBASE, msr | X2APIC_ENABLE);
}
}
I am very surprised for this ,why BIOS has claimed that BIOS didn't support
x2apic , but Linux kernel will force enabling x2apic
(although booting CPU has supported x2apic) ?
I have one machine , in booting log ,
There is no x2apic enabled by BIOS, switching to x2apic ops
But later , linux kernel will print
[0.464649] parse_iosapics: ecap f0207e
[0.469728] IOAPIC id 8 under DRHD base 0xa800 IU 0
[0.576731] IOAPIC id 0 under DRHD base 0xa800 IOMMU 0
[0.584065] parse_io_apic: io_apic 2 nr 2
[0.589590] Enabled IRQ remapping in x2apic mode
[0.595663] Enabling x2apic
[0.599337] Enabled x2apic
[0.603007] Switched APIC routing to cluster x2apic. ( I can add
x2apic_phys to force using physical mode).
1, I quite worry about this , try to think , BIOS didn't enable x2apic , it
just did enable xAPIC mode ,
it didn't create ACPI tables of x2apic. In this case, how Linux kernel can run
normally in x2apic mode ?
in fact, in this machine , Linux kernel didn't run into some issues ,but I
still think this is a potential defect.
2, why Linux kernel first depends on MSR and later will discard MSR ? is there
any history ?
Linux kernel wants to completely discard MSR checking in future ?
3, anybody test machines which support x2apic ? do you have similar problems ?
in these machines , will you check MSR bit ?
-- Bob(LinBao Zhang)
HP linux kernel enginner
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Why Linux kernel forced to enter X2APIC mode( just because of booting cpu has supported x2apic) without depending on BIOS' setting in MSR-x2apic enablement bit ?
On Tue, Dec 18, 2012 at 1:50 AM, Zhang, Lin-Bao (Linux Kernel RD) linbao.zh...@hp.com wrote: Hi Suresh and other guys , In 3.4.4/3.6.6 ,I found a x2apic issue . if I am wrong , sorry first , and welcome your correction . thanks for your forwarding other maintainers. I am testing a server , its BIOS is like this: a) If BIOS think the system is of x2apic , it will set x2apic enablement bit in MSR and create x2apic ACPI tables and pass control to OS with x2apic mode kernel will check if DMAR table is right or not, if intr-remapping could be enabled, kernel will stay with x2apic. if intr-remapping can not be enabled (bad dmar table), kernel will switch back to xapic mode, but if BSP's X2APIC is bigger than 255, then you will get kernel panic. b) If BIOS feel the system doesn't meet x2apic conditions , it will not set x2apic enablement bit in MSR ,and pass control to OS with xapic mode. kernel will check if cpuid support x2apic, if it supports x2apic, it will check if DMAR/intr-remapping could be enabled, if so kernel will switch to x2apic. otherwise it will stay with xapic. So you need to make cpuid show does not support x2apic --- check with intel they have way to do that. or you can pass nox2apic in boot command line. Yinghai -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
