Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
Bjorn, You have the following registers to consider (for P4/Core): #define MSR_IA32_PEBS_ENABLE0x03f1 #define MSR_CORE_PERF_FIXED_CTR00x0309 #define MSR_CORE_PERF_FIXED_CTR10x030a #define MSR_CORE_PERF_FIXED_CTR20x030b #define MSR_CORE_PERF_FIXED_CTR_CTRL0x038d #define MSR_CORE_PERF_GLOBAL_STATUS 0x038e #define MSR_CORE_PERF_GLOBAL_CTRL 0x038f #define MSR_CORE_PERF_GLOBAL_OVF_CTRL 0x0390 With Barcelona, you'll also have to consider: #define MSR_AMD64_IBSFETCHCTL 0xc0011030 #define MSR_AMD64_IBSFETCHLINAD 0xc0011031 #define MSR_AMD64_IBSFETCHPHYSAD0xc0011032 #define MSR_AMD64_IBSOPCTL 0xc0011033 #define MSR_AMD64_IBSOPRIP 0xc0011034 #define MSR_AMD64_IBSOPDATA 0xc0011035 #define MSR_AMD64_IBSOPDATA20xc0011036 #define MSR_AMD64_IBSOPDATA30xc0011037 #define MSR_AMD64_IBSDCLINAD0xc0011038 #define MSR_AMD64_IBSDCPHYSAD 0xc0011039 #define MSR_AMD64_IBSCTL0xc001103A I think you could organize by groups with a bitmap per group and some sorted linked list to keep track of all of them. On Fri, Jun 22, 2007 at 09:13:54AM +0200, Bj?rn Steinbrink wrote: > Hi Stephane, > > On 2007.06.21 01:36:45 -0700, Stephane Eranian wrote: > > Bjorn, > > > > > > On Wed, Jun 20, 2007 at 02:59:33PM -0700, Stephane Eranian wrote: > > > Bjorn, > > > > > > I ran into one issue related with the new allocator. > > Should be the same with 2.6.21 and earlier, the "new" allocator should > do exactly the samething, it just fixes the breakage introduced in the > post-2.6.21 cleanup. > > > > In the case of a Core 2 Duo processor, the PMU implements more > > > than just basic counters. In particular it supports fixed counters > > > and PEBS where both use another set of MSRs. Those are not within > > > a 66 bit distance from MSR_ARCH_PERFMON_EVNTSEL0. Thus the allocator > > > fails with an assertion. > > How far away are they? > > > > > > > I do know that perfmon is the only consumer of those extended > > > features TODAY. Yet I think we need to define the allocator such > > > that it can work with other "distant" MSRs as well. > > > > > > > I think that a workaround for this issue could be for the allocator > > to grant the requests for registers outside of the range, i.e., register > > that it does not see/manage. > > That would also allow multiple subsystems to use them at the same time. > And whoever adds the second user of those MSRs might not be aware of the > just-grant-it policy of the allocator. And bugs that arise due to such > problems will probably become a real PITA to track down. > > Unfortunately, I don't see any elegant solution to this atm, and of > course making your code simply circumvent the allocator isn't an option > either. > > Thanks, > Björn > > - > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > ___ > oprofile-list mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/oprofile-list -- -Stephane - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
Hi Stephane, On 2007.06.21 01:36:45 -0700, Stephane Eranian wrote: > Bjorn, > > > On Wed, Jun 20, 2007 at 02:59:33PM -0700, Stephane Eranian wrote: > > Bjorn, > > > > I ran into one issue related with the new allocator. Should be the same with 2.6.21 and earlier, the "new" allocator should do exactly the samething, it just fixes the breakage introduced in the post-2.6.21 cleanup. > > In the case of a Core 2 Duo processor, the PMU implements more > > than just basic counters. In particular it supports fixed counters > > and PEBS where both use another set of MSRs. Those are not within > > a 66 bit distance from MSR_ARCH_PERFMON_EVNTSEL0. Thus the allocator > > fails with an assertion. How far away are they? > > > > I do know that perfmon is the only consumer of those extended > > features TODAY. Yet I think we need to define the allocator such > > that it can work with other "distant" MSRs as well. > > > > I think that a workaround for this issue could be for the allocator > to grant the requests for registers outside of the range, i.e., register > that it does not see/manage. That would also allow multiple subsystems to use them at the same time. And whoever adds the second user of those MSRs might not be aware of the just-grant-it policy of the allocator. And bugs that arise due to such problems will probably become a real PITA to track down. Unfortunately, I don't see any elegant solution to this atm, and of course making your code simply circumvent the allocator isn't an option either. Thanks, Björn - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
Hi Stephane, On 2007.06.21 01:36:45 -0700, Stephane Eranian wrote: Bjorn, On Wed, Jun 20, 2007 at 02:59:33PM -0700, Stephane Eranian wrote: Bjorn, I ran into one issue related with the new allocator. Should be the same with 2.6.21 and earlier, the new allocator should do exactly the samething, it just fixes the breakage introduced in the post-2.6.21 cleanup. In the case of a Core 2 Duo processor, the PMU implements more than just basic counters. In particular it supports fixed counters and PEBS where both use another set of MSRs. Those are not within a 66 bit distance from MSR_ARCH_PERFMON_EVNTSEL0. Thus the allocator fails with an assertion. How far away are they? I do know that perfmon is the only consumer of those extended features TODAY. Yet I think we need to define the allocator such that it can work with other distant MSRs as well. I think that a workaround for this issue could be for the allocator to grant the requests for registers outside of the range, i.e., register that it does not see/manage. That would also allow multiple subsystems to use them at the same time. And whoever adds the second user of those MSRs might not be aware of the just-grant-it policy of the allocator. And bugs that arise due to such problems will probably become a real PITA to track down. Unfortunately, I don't see any elegant solution to this atm, and of course making your code simply circumvent the allocator isn't an option either. Thanks, Björn - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
Bjorn, You have the following registers to consider (for P4/Core): #define MSR_IA32_PEBS_ENABLE0x03f1 #define MSR_CORE_PERF_FIXED_CTR00x0309 #define MSR_CORE_PERF_FIXED_CTR10x030a #define MSR_CORE_PERF_FIXED_CTR20x030b #define MSR_CORE_PERF_FIXED_CTR_CTRL0x038d #define MSR_CORE_PERF_GLOBAL_STATUS 0x038e #define MSR_CORE_PERF_GLOBAL_CTRL 0x038f #define MSR_CORE_PERF_GLOBAL_OVF_CTRL 0x0390 With Barcelona, you'll also have to consider: #define MSR_AMD64_IBSFETCHCTL 0xc0011030 #define MSR_AMD64_IBSFETCHLINAD 0xc0011031 #define MSR_AMD64_IBSFETCHPHYSAD0xc0011032 #define MSR_AMD64_IBSOPCTL 0xc0011033 #define MSR_AMD64_IBSOPRIP 0xc0011034 #define MSR_AMD64_IBSOPDATA 0xc0011035 #define MSR_AMD64_IBSOPDATA20xc0011036 #define MSR_AMD64_IBSOPDATA30xc0011037 #define MSR_AMD64_IBSDCLINAD0xc0011038 #define MSR_AMD64_IBSDCPHYSAD 0xc0011039 #define MSR_AMD64_IBSCTL0xc001103A I think you could organize by groups with a bitmap per group and some sorted linked list to keep track of all of them. On Fri, Jun 22, 2007 at 09:13:54AM +0200, Bj?rn Steinbrink wrote: Hi Stephane, On 2007.06.21 01:36:45 -0700, Stephane Eranian wrote: Bjorn, On Wed, Jun 20, 2007 at 02:59:33PM -0700, Stephane Eranian wrote: Bjorn, I ran into one issue related with the new allocator. Should be the same with 2.6.21 and earlier, the new allocator should do exactly the samething, it just fixes the breakage introduced in the post-2.6.21 cleanup. In the case of a Core 2 Duo processor, the PMU implements more than just basic counters. In particular it supports fixed counters and PEBS where both use another set of MSRs. Those are not within a 66 bit distance from MSR_ARCH_PERFMON_EVNTSEL0. Thus the allocator fails with an assertion. How far away are they? I do know that perfmon is the only consumer of those extended features TODAY. Yet I think we need to define the allocator such that it can work with other distant MSRs as well. I think that a workaround for this issue could be for the allocator to grant the requests for registers outside of the range, i.e., register that it does not see/manage. That would also allow multiple subsystems to use them at the same time. And whoever adds the second user of those MSRs might not be aware of the just-grant-it policy of the allocator. And bugs that arise due to such problems will probably become a real PITA to track down. Unfortunately, I don't see any elegant solution to this atm, and of course making your code simply circumvent the allocator isn't an option either. Thanks, Björn - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ oprofile-list mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/oprofile-list -- -Stephane - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
Bjorn, On Wed, Jun 20, 2007 at 02:59:33PM -0700, Stephane Eranian wrote: > Bjorn, > > I ran into one issue related with the new allocator. > > In the case of a Core 2 Duo processor, the PMU implements more > than just basic counters. In particular it supports fixed counters > and PEBS where both use another set of MSRs. Those are not within > a 66 bit distance from MSR_ARCH_PERFMON_EVNTSEL0. Thus the allocator > fails with an assertion. > > I do know that perfmon is the only consumer of those extended > features TODAY. Yet I think we need to define the allocator such > that it can work with other "distant" MSRs as well. > I think that a workaround for this issue could be for the allocator to grant the requests for registers outside of the range, i.e., register that it does not see/manage. > On Wed, Jun 20, 2007 at 08:33:15PM +0200, Bj?rn Steinbrink wrote: > > On 2007.06.20 15:01:02 +0200, Andi Kleen wrote: > > > > > > > Once this is fixed (which is what Bjorn did), then I will agree with > > > > you. > > > > For this, the allocator needs to be able to probe the CPU and initialize > > > > its own data structures. > > > > > > Ok that sounds reasonable. Please someone send a patch that does only > > > that. > > > > OK, here come the bugfixes without any restructuring. The first patch > > enables unconditional probing of the watchdog. The second makes the > > perfmon nmi watchdog reserve the correct perfctr/evntsel. > > > > Björn > > > > - > > This SF.net email is sponsored by DB2 Express > > Download DB2 Express C - the FREE version of DB2 express and take > > control of your XML. No limits. Just data. Click to get it now. > > http://sourceforge.net/powerbar/db2/ > > ___ > > oprofile-list mailing list > > [EMAIL PROTECTED] > > https://lists.sourceforge.net/lists/listinfo/oprofile-list > > -- > > -Stephane > ___ > perfmon mailing list > [EMAIL PROTECTED] > http://www.hpl.hp.com/hosted/linux/mail-archives/perfmon/ -- -Stephane - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
Bjorn, On Wed, Jun 20, 2007 at 02:59:33PM -0700, Stephane Eranian wrote: Bjorn, I ran into one issue related with the new allocator. In the case of a Core 2 Duo processor, the PMU implements more than just basic counters. In particular it supports fixed counters and PEBS where both use another set of MSRs. Those are not within a 66 bit distance from MSR_ARCH_PERFMON_EVNTSEL0. Thus the allocator fails with an assertion. I do know that perfmon is the only consumer of those extended features TODAY. Yet I think we need to define the allocator such that it can work with other distant MSRs as well. I think that a workaround for this issue could be for the allocator to grant the requests for registers outside of the range, i.e., register that it does not see/manage. On Wed, Jun 20, 2007 at 08:33:15PM +0200, Bj?rn Steinbrink wrote: On 2007.06.20 15:01:02 +0200, Andi Kleen wrote: Once this is fixed (which is what Bjorn did), then I will agree with you. For this, the allocator needs to be able to probe the CPU and initialize its own data structures. Ok that sounds reasonable. Please someone send a patch that does only that. OK, here come the bugfixes without any restructuring. The first patch enables unconditional probing of the watchdog. The second makes the perfmon nmi watchdog reserve the correct perfctr/evntsel. Björn - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ oprofile-list mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/oprofile-list -- -Stephane ___ perfmon mailing list [EMAIL PROTECTED] http://www.hpl.hp.com/hosted/linux/mail-archives/perfmon/ -- -Stephane - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
Bjorn, I ran into one issue related with the new allocator. In the case of a Core 2 Duo processor, the PMU implements more than just basic counters. In particular it supports fixed counters and PEBS where both use another set of MSRs. Those are not within a 66 bit distance from MSR_ARCH_PERFMON_EVNTSEL0. Thus the allocator fails with an assertion. I do know that perfmon is the only consumer of those extended features TODAY. Yet I think we need to define the allocator such that it can work with other "distant" MSRs as well. On Wed, Jun 20, 2007 at 08:33:15PM +0200, Bj?rn Steinbrink wrote: > On 2007.06.20 15:01:02 +0200, Andi Kleen wrote: > > > > > Once this is fixed (which is what Bjorn did), then I will agree with you. > > > For this, the allocator needs to be able to probe the CPU and initialize > > > its own data structures. > > > > Ok that sounds reasonable. Please someone send a patch that does only > > that. > > OK, here come the bugfixes without any restructuring. The first patch > enables unconditional probing of the watchdog. The second makes the > perfmon nmi watchdog reserve the correct perfctr/evntsel. > > Björn > > - > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > ___ > oprofile-list mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/oprofile-list -- -Stephane - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
On 2007.06.20 15:01:02 +0200, Andi Kleen wrote: > > > Once this is fixed (which is what Bjorn did), then I will agree with you. > > For this, the allocator needs to be able to probe the CPU and initialize > > its own data structures. > > Ok that sounds reasonable. Please someone send a patch that does only > that. OK, here come the bugfixes without any restructuring. The first patch enables unconditional probing of the watchdog. The second makes the perfmon nmi watchdog reserve the correct perfctr/evntsel. Björn - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
> Once this is fixed (which is what Bjorn did), then I will agree with you. > For this, the allocator needs to be able to probe the CPU and initialize > its own data structures. Ok that sounds reasonable. Please someone send a patch that does only that. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
Andi,
On Wed, Jun 20, 2007 at 02:31:43PM +0200, Andi Kleen wrote:
> On Wednesday 20 June 2007 12:35:56 Björn Steinbrink wrote:
> > The performance counter allocator is tied to the LAPIC NMI watchdog,
>
> That's not true. It's completely independent. It just happens to be in
> the same file, but it has no direct functional ties to the watchdog.
>
I agree with you that the allocator is functionally independent of the
watchdog. That is how I'd like to see it and I think we all agree on
that.
Yet in the current implementation, there is a link between the two which
causes the issues I ran into. If you look at:
static inline unsigned int nmi_evntsel_msr_to_bit(unsigned int msr)
{
return wd_ops ? msr - wd_ops->evntsel : 0;
}
int reserve_evntsel_nmi(unsigned int msr)
{
unsigned int counter;
counter = nmi_evntsel_msr_to_bit(msr);
BUG_ON(counter > NMI_MAX_COUNTER_BITS);
}
You see that if the wd_ops (a watchdog structure) is not initialized
then the allocator collapses all MSRs onto one bit.
Once this is fixed (which is what Bjorn did), then I will agree with you.
For this, the allocator needs to be able to probe the CPU and initialize
its own data structures.
--
-Stephane
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
Andi,
On Wed, Jun 20, 2007 at 02:31:43PM +0200, Andi Kleen wrote:
On Wednesday 20 June 2007 12:35:56 Björn Steinbrink wrote:
The performance counter allocator is tied to the LAPIC NMI watchdog,
That's not true. It's completely independent. It just happens to be in
the same file, but it has no direct functional ties to the watchdog.
I agree with you that the allocator is functionally independent of the
watchdog. That is how I'd like to see it and I think we all agree on
that.
Yet in the current implementation, there is a link between the two which
causes the issues I ran into. If you look at:
static inline unsigned int nmi_evntsel_msr_to_bit(unsigned int msr)
{
return wd_ops ? msr - wd_ops-evntsel : 0;
}
int reserve_evntsel_nmi(unsigned int msr)
{
unsigned int counter;
counter = nmi_evntsel_msr_to_bit(msr);
BUG_ON(counter NMI_MAX_COUNTER_BITS);
}
You see that if the wd_ops (a watchdog structure) is not initialized
then the allocator collapses all MSRs onto one bit.
Once this is fixed (which is what Bjorn did), then I will agree with you.
For this, the allocator needs to be able to probe the CPU and initialize
its own data structures.
--
-Stephane
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
Once this is fixed (which is what Bjorn did), then I will agree with you. For this, the allocator needs to be able to probe the CPU and initialize its own data structures. Ok that sounds reasonable. Please someone send a patch that does only that. -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
On 2007.06.20 15:01:02 +0200, Andi Kleen wrote: Once this is fixed (which is what Bjorn did), then I will agree with you. For this, the allocator needs to be able to probe the CPU and initialize its own data structures. Ok that sounds reasonable. Please someone send a patch that does only that. OK, here come the bugfixes without any restructuring. The first patch enables unconditional probing of the watchdog. The second makes the perfmon nmi watchdog reserve the correct perfctr/evntsel. Björn - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [perfmon] Re: [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog
Bjorn, I ran into one issue related with the new allocator. In the case of a Core 2 Duo processor, the PMU implements more than just basic counters. In particular it supports fixed counters and PEBS where both use another set of MSRs. Those are not within a 66 bit distance from MSR_ARCH_PERFMON_EVNTSEL0. Thus the allocator fails with an assertion. I do know that perfmon is the only consumer of those extended features TODAY. Yet I think we need to define the allocator such that it can work with other distant MSRs as well. On Wed, Jun 20, 2007 at 08:33:15PM +0200, Bj?rn Steinbrink wrote: On 2007.06.20 15:01:02 +0200, Andi Kleen wrote: Once this is fixed (which is what Bjorn did), then I will agree with you. For this, the allocator needs to be able to probe the CPU and initialize its own data structures. Ok that sounds reasonable. Please someone send a patch that does only that. OK, here come the bugfixes without any restructuring. The first patch enables unconditional probing of the watchdog. The second makes the perfmon nmi watchdog reserve the correct perfctr/evntsel. Björn - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ oprofile-list mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/oprofile-list -- -Stephane - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
