Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread Paul Mackerras
David Miller writes: From: Paul Mackerras [EMAIL PROTECTED] Date: Thu, 15 Nov 2007 08:50:22 +1100 I'd prefer to have a transaction() system call like I suggested to Nick rather than overloading read() like this. So much for getting rid of the extra system calls... *I* never had a

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread David Miller
From: Paul Mackerras [EMAIL PROTECTED] Date: Thu, 15 Nov 2007 10:12:22 +1100 *I* never had a problem with a few extra system calls. I don't understand why you (apparently) do. We're stuck with them forever, they are hard to version and extend cleanly. Those are my main objections. - To

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread Nick Piggin
On Thursday 15 November 2007 09:56, Chuck Ebbert wrote: On 11/14/2007 05:17 AM, Nick Piggin wrote: But in general, for special files, I guess the response is usually some structured data (that is not visible at the syscall layer). So I don't see a big problem to have a similarly arbitrarily

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread David Miller
From: Paul Mackerras [EMAIL PROTECTED] Date: Thu, 15 Nov 2007 08:50:22 +1100 I'd prefer to have a transaction() system call like I suggested to Nick rather than overloading read() like this. So much for getting rid of the extra system calls... - To unsubscribe from this list: send the line

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread Chuck Ebbert
On 11/14/2007 05:17 AM, Nick Piggin wrote: But in general, for special files, I guess the response is usually some structured data (that is not visible at the syscall layer). So I don't see a big problem to have a similarly arbitrarily structured request. IOW, an ioctl. - To unsubscribe

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread Nick Piggin
On Thursday 15 November 2007 08:30, Paul Mackerras wrote: Nick Piggin writes: What I really mean is a readv-like syscall, but one that also vectorises the file offset. Maybe this is useful enough as a generic syscall that also helps Paul's example... I've sometimes thought it would be

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread Stephane Eranian
Andi, On Wed, Nov 14, 2007 at 03:24:11PM +0100, Andi Kleen wrote: On Wed, Nov 14, 2007 at 05:09:09AM -0800, Stephane Eranian wrote: Partially true. The file descriptor becomes really useful when you sample. You leverage the file descriptor to receive notifications of counter overflows

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread Paul Mackerras
Andi Kleen writes: This only works when counting (not sampling) and only for self-monitoring. It works for global monitoring too. How would you provide access to the counters of another process? Through an extension to ptrace perhaps? Paul. - To unsubscribe from this list: send the line

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread Paul Mackerras
David Miller writes: From: Paul Mackerras [EMAIL PROTECTED] Date: Thu, 15 Nov 2007 10:12:22 +1100 *I* never had a problem with a few extra system calls. I don't understand why you (apparently) do. We're stuck with them forever, they are hard to version and extend cleanly. Those

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread David Miller
From: Paul Mackerras [EMAIL PROTECTED] Date: Thu, 15 Nov 2007 12:11:10 +1100 The third (hard to extend cleanly) is a good point, and is a valid criticism of the current set of perfmon2 system calls, I think. However, the goal of being able to extend the interface tends to be in opposition to

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread Paul Mackerras
David Miller writes: From: Paul Mackerras [EMAIL PROTECTED] Date: Thu, 15 Nov 2007 12:11:10 +1100 The third (hard to extend cleanly) is a good point, and is a valid criticism of the current set of perfmon2 system calls, I think. However, the goal of being able to extend the interface

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread dean gaudet
On Wed, 14 Nov 2007, Andi Kleen wrote: Later a syscall might be needed with event multiplexing, but that seems more like a far away non essential feature. actually multiplexing is the main feature i am in need of. there are an insufficient number of counters (even on k8 with 4 counters) to do

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread Paul Mackerras
dean gaudet writes: actually multiplexing is the main feature i am in need of. there are an insufficient number of counters (even on k8 with 4 counters) to do complete stall accounting or to get a general overview of L1d/L1i/L2 cache hit rates, average miss latency, time spent in various

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-14 Thread dean gaudet
On Thu, 15 Nov 2007, Paul Mackerras wrote: dean gaudet writes: actually multiplexing is the main feature i am in need of. there are an insufficient number of counters (even on k8 with 4 counters) to do complete stall accounting or to get a general overview of L1d/L1i/L2 cache hit

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andrew Morton
On Wed, 14 Nov 2007 18:24:36 +1100 Paul Mackerras <[EMAIL PROTECTED]> wrote: > Andrew Morton writes: > > > I was hoping that after the round of release-and-review which Stephane, > > Andi and I did about twelve months ago that we were on track to merge the > > perfmon codebase as-offered. But

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Paul Mackerras
Andrew Morton writes: > I was hoping that after the round of release-and-review which Stephane, > Andi and I did about twelve months ago that we were on track to merge the > perfmon codebase as-offered. But now it turns out that the sentiment is > that the code simply has too many

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
[dropped all these bouncing email lists. Adding closed lists to public cc lists is just a bad idea] > int > main(int argc, char **argv) > { > int ctx_fd; > pfarg_pmd_t pd[1]; > pfarg_pmc_t pc[1]; > pfarg_ctx_t ctx; > pfarg_load_t load_args; > > memset(, 0,

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
On Tue, Nov 13, 2007 at 04:28:52PM -0800, Philip Mucci wrote: > I know you don't want to hear this, but we actually use all of the > features of perfmon, because a) we wanted to use the best methods That is hard to believe. But let's go for it temporarily for the argument. Can you instead

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Philip Mucci
Hi Andi, pfmon is a single tool and fairly low level, the HPC folks don't use it so much because it isn't parallel aware and is meant for power- users. It is not representative of the tools used in HPC at all. Our community uses tools built on the infrastructure provided by libpfm and

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Andi, On Tue, Nov 13, 2007 at 11:25:34PM +0100, Andi Kleen wrote: > On Tue, Nov 13, 2007 at 02:22:34PM -0800, Stephane Eranian wrote: > > Andi, > > On Tue, Nov 13, 2007 at 10:50:56PM +0100, Andi Kleen wrote: > > > > Yes, horribly more complicated because of locking issues within perfmon. > > > >

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Christoph Hellwig
What about investing some effort to do a proper performance counter infrastructure or turning the mess perfom is into one instead of this useless rant? Code is not getting any better by your complain ccing gazillions of useless list. - To unsubscribe from this list: send the line "unsubscribe

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
On Tue, Nov 13, 2007 at 02:22:34PM -0800, Stephane Eranian wrote: > Andi, > On Tue, Nov 13, 2007 at 10:50:56PM +0100, Andi Kleen wrote: > > > Yes, horribly more complicated because of locking issues within perfmon. > > > As soon as you expose a file descriptor, you need some locking to prevent > >

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Andi, On Tue, Nov 13, 2007 at 10:50:56PM +0100, Andi Kleen wrote: > > Yes, horribly more complicated because of locking issues within perfmon. > > As soon as you expose a file descriptor, you need some locking to prevent > > multiple user threads (malicious or not) to compete to access the PMU

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
> Yes, horribly more complicated because of locking issues within perfmon. > As soon as you expose a file descriptor, you need some locking to prevent > multiple user threads (malicious or not) to compete to access the PMU state. Why do you need the file descriptor? One of the main problems

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Andi. On Tue, Nov 13, 2007 at 10:29:02PM +0100, Andi Kleen wrote: > On Tue, Nov 13, 2007 at 01:13:45PM -0800, Stephane Eranian wrote: > > Oprofile does not setup the PMU interrupt. It builds on top of the NMI > > watchdog > > setup. > > Oprofile works without the NMI watchdog too, but it just

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Greg KH
On Tue, Nov 13, 2007 at 01:33:13PM -0800, Stephane Eranian wrote: > I think I understand your concerns. I will work on this. I think it is > possible to > refactor. It will certainly be painful (for me), but I think it can be done > within > some reasonable delay. Of course, it would be help if

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Greg, On Tue, Nov 13, 2007 at 10:59:24AM -0800, Greg KH wrote: > On Tue, Nov 13, 2007 at 10:47:45AM -0800, Philip Mucci wrote: > > Hi folks, > > > > Well, I can say the mood here at supercomputing'07 is pretty somber in > > regards to the latest exchange of messages regarding the perfmon

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
On Tue, Nov 13, 2007 at 01:13:45PM -0800, Stephane Eranian wrote: > Oprofile does not setup the PMU interrupt. It builds on top of the NMI > watchdog > setup. Oprofile works without the NMI watchdog too, but it just happens to be another NMI user. > It uses the register_die() mechanism, Not

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Will, On Tue, Nov 13, 2007 at 01:33:55PM -0500, William Cohen wrote: > > The oprofile module can setup a handler for PMU interrupts. This is done in > archi/x86/oprofile/nmi_int:nmi_cpu_setup(). Other modules could do the > same. However, it bumps what ever was using the nmi/pmu off, then

Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
> In the past I suggested that it might be useful to have a version of > perfmon2 that only set up the perfmon on a global basis. That would allow Context switch is imho the main differentiating feature of perfmon over oprofile. Not sure it makes sense to take that one out. I don't think the

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
> He speaks for quite a few people - they have serious need for this feature Most likely they have serious need for a very small subset of perfmon2. The point of my proposal was to get this very small subset in quickly. Phil, how many of the command line options of pfmon do you actually use? How

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Greg KH
On Tue, Nov 13, 2007 at 12:07:28PM -0800, Andrew Morton wrote: > > So. If what I am saying is correct then the best course of action would be > for Stephane to help us all to understand what these features are and why > we need them. The ideal way in which to do this is > > [patch] perfmon:

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andrew Morton
On Tue, 13 Nov 2007 10:59:24 -0800 Greg KH <[EMAIL PROTECTED]> wrote: > On Tue, Nov 13, 2007 at 10:47:45AM -0800, Philip Mucci wrote: > > Hi folks, > > > > Well, I can say the mood here at supercomputing'07 is pretty somber in > > regards to the latest exchange of messages regarding the perfmon

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Philip Mucci
Hi folks, Well, I can say the mood here at supercomputing'07 is pretty somber in regards to the latest exchange of messages regarding the perfmon patches. Our community has been the largest user of both the PerfCtr and the Perfmon patches, the former being regularly installed by vendors

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Greg KH
On Tue, Nov 13, 2007 at 10:47:45AM -0800, Philip Mucci wrote: > Hi folks, > > Well, I can say the mood here at supercomputing'07 is pretty somber in > regards to the latest exchange of messages regarding the perfmon patches. "somber"? Why? We (a number of the kernel developers) want to see

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread William Cohen
Stephane Eranian wrote: Hello, On Tue, Nov 13, 2007 at 10:35:11AM -0500, William Cohen wrote: Robert Richter wrote: On 10.11.07 21:32:39, Andi Kleen wrote: It would be really good to extract a core perfmon and start with that and then add stuff as it makes sense. e.g. core perfmon could be

Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Hello, On Tue, Nov 13, 2007 at 10:35:11AM -0500, William Cohen wrote: > Robert Richter wrote: > > On 10.11.07 21:32:39, Andi Kleen wrote: > >> It would be really good to extract a core perfmon and start with > >> that and then add stuff as it makes sense. > >> > >> e.g. core perfmon could be

Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread William Cohen
Robert Richter wrote: On 10.11.07 21:32:39, Andi Kleen wrote: It would be really good to extract a core perfmon and start with that and then add stuff as it makes sense. e.g. core perfmon could be something simple like just support to context switch state and initialize counters in a basic way

Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread William Cohen
Robert Richter wrote: On 10.11.07 21:32:39, Andi Kleen wrote: It would be really good to extract a core perfmon and start with that and then add stuff as it makes sense. e.g. core perfmon could be something simple like just support to context switch state and initialize counters in a basic way

Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Hello, On Tue, Nov 13, 2007 at 10:35:11AM -0500, William Cohen wrote: Robert Richter wrote: On 10.11.07 21:32:39, Andi Kleen wrote: It would be really good to extract a core perfmon and start with that and then add stuff as it makes sense. e.g. core perfmon could be something simple

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread William Cohen
Stephane Eranian wrote: Hello, On Tue, Nov 13, 2007 at 10:35:11AM -0500, William Cohen wrote: Robert Richter wrote: On 10.11.07 21:32:39, Andi Kleen wrote: It would be really good to extract a core perfmon and start with that and then add stuff as it makes sense. e.g. core perfmon could be

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Greg KH
On Tue, Nov 13, 2007 at 10:47:45AM -0800, Philip Mucci wrote: Hi folks, Well, I can say the mood here at supercomputing'07 is pretty somber in regards to the latest exchange of messages regarding the perfmon patches. somber? Why? We (a number of the kernel developers) want to see the

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Philip Mucci
Hi folks, Well, I can say the mood here at supercomputing'07 is pretty somber in regards to the latest exchange of messages regarding the perfmon patches. Our community has been the largest user of both the PerfCtr and the Perfmon patches, the former being regularly installed by vendors

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andrew Morton
On Tue, 13 Nov 2007 10:59:24 -0800 Greg KH [EMAIL PROTECTED] wrote: On Tue, Nov 13, 2007 at 10:47:45AM -0800, Philip Mucci wrote: Hi folks, Well, I can say the mood here at supercomputing'07 is pretty somber in regards to the latest exchange of messages regarding the perfmon patches.

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Greg KH
On Tue, Nov 13, 2007 at 12:07:28PM -0800, Andrew Morton wrote: So. If what I am saying is correct then the best course of action would be for Stephane to help us all to understand what these features are and why we need them. The ideal way in which to do this is [patch] perfmon: core

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
He speaks for quite a few people - they have serious need for this feature Most likely they have serious need for a very small subset of perfmon2. The point of my proposal was to get this very small subset in quickly. Phil, how many of the command line options of pfmon do you actually use? How

Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
In the past I suggested that it might be useful to have a version of perfmon2 that only set up the perfmon on a global basis. That would allow Context switch is imho the main differentiating feature of perfmon over oprofile. Not sure it makes sense to take that one out. I don't think the

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Will, On Tue, Nov 13, 2007 at 01:33:55PM -0500, William Cohen wrote: The oprofile module can setup a handler for PMU interrupts. This is done in archi/x86/oprofile/nmi_int:nmi_cpu_setup(). Other modules could do the same. However, it bumps what ever was using the nmi/pmu off, then

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
On Tue, Nov 13, 2007 at 01:13:45PM -0800, Stephane Eranian wrote: Oprofile does not setup the PMU interrupt. It builds on top of the NMI watchdog setup. Oprofile works without the NMI watchdog too, but it just happens to be another NMI user. It uses the register_die() mechanism, Not

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Greg, On Tue, Nov 13, 2007 at 10:59:24AM -0800, Greg KH wrote: On Tue, Nov 13, 2007 at 10:47:45AM -0800, Philip Mucci wrote: Hi folks, Well, I can say the mood here at supercomputing'07 is pretty somber in regards to the latest exchange of messages regarding the perfmon patches.

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Greg KH
On Tue, Nov 13, 2007 at 01:33:13PM -0800, Stephane Eranian wrote: I think I understand your concerns. I will work on this. I think it is possible to refactor. It will certainly be painful (for me), but I think it can be done within some reasonable delay. Of course, it would be help if you

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Andi. On Tue, Nov 13, 2007 at 10:29:02PM +0100, Andi Kleen wrote: On Tue, Nov 13, 2007 at 01:13:45PM -0800, Stephane Eranian wrote: Oprofile does not setup the PMU interrupt. It builds on top of the NMI watchdog setup. Oprofile works without the NMI watchdog too, but it just happens to

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
Yes, horribly more complicated because of locking issues within perfmon. As soon as you expose a file descriptor, you need some locking to prevent multiple user threads (malicious or not) to compete to access the PMU state. Why do you need the file descriptor? One of the main problems with

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Andi, On Tue, Nov 13, 2007 at 10:50:56PM +0100, Andi Kleen wrote: Yes, horribly more complicated because of locking issues within perfmon. As soon as you expose a file descriptor, you need some locking to prevent multiple user threads (malicious or not) to compete to access the PMU state.

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
On Tue, Nov 13, 2007 at 02:22:34PM -0800, Stephane Eranian wrote: Andi, On Tue, Nov 13, 2007 at 10:50:56PM +0100, Andi Kleen wrote: Yes, horribly more complicated because of locking issues within perfmon. As soon as you expose a file descriptor, you need some locking to prevent multiple

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Christoph Hellwig
stupid bullshitting snipped What about investing some effort to do a proper performance counter infrastructure or turning the mess perfom is into one instead of this useless rant? Code is not getting any better by your complain ccing gazillions of useless list. - To unsubscribe from this list:

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Stephane Eranian
Andi, On Tue, Nov 13, 2007 at 11:25:34PM +0100, Andi Kleen wrote: On Tue, Nov 13, 2007 at 02:22:34PM -0800, Stephane Eranian wrote: Andi, On Tue, Nov 13, 2007 at 10:50:56PM +0100, Andi Kleen wrote: Yes, horribly more complicated because of locking issues within perfmon. As soon as

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Philip Mucci
Hi Andi, pfmon is a single tool and fairly low level, the HPC folks don't use it so much because it isn't parallel aware and is meant for power- users. It is not representative of the tools used in HPC at all. Our community uses tools built on the infrastructure provided by libpfm and

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
On Tue, Nov 13, 2007 at 04:28:52PM -0800, Philip Mucci wrote: I know you don't want to hear this, but we actually use all of the features of perfmon, because a) we wanted to use the best methods That is hard to believe. But let's go for it temporarily for the argument. Can you instead

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andi Kleen
[dropped all these bouncing email lists. Adding closed lists to public cc lists is just a bad idea] int main(int argc, char **argv) { int ctx_fd; pfarg_pmd_t pd[1]; pfarg_pmc_t pc[1]; pfarg_ctx_t ctx; pfarg_load_t load_args; memset(ctx, 0,

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Paul Mackerras
Andrew Morton writes: I was hoping that after the round of release-and-review which Stephane, Andi and I did about twelve months ago that we were on track to merge the perfmon codebase as-offered. But now it turns out that the sentiment is that the code simply has too many bells-and-whistles

Re: [perfmon] Re: [perfmon2] perfmon2 merge news

2007-11-13 Thread Andrew Morton
On Wed, 14 Nov 2007 18:24:36 +1100 Paul Mackerras [EMAIL PROTECTED] wrote: Andrew Morton writes: I was hoping that after the round of release-and-review which Stephane, Andi and I did about twelve months ago that we were on track to merge the perfmon codebase as-offered. But now it

<    1   2