Stephane, Thanks a lot for such a prompt reply ! Please see my inlined comments.
On Monday 03 September 2007 12:28, Stephane Eranian wrote: > Vivek, > > On Mon, Sep 03, 2007 at 11:49:11AM -0400, [EMAIL PROTECTED] wrote: > > Hi ! > > > > I am trying to use perfmon on Itanium 2 server (HP's testdrive td178 > > machine ). I have a multi-threaded program but each thread can be pinned > > to a different CPU (total 16 cores here). Each thread is going to monitor > > its own overflow (i.e. receive its own overflow notification). For > > configuring this, do i need to set PFM_FL_SYSTEM_WIDE while creating > > PFM_CONTEXT ? > > In system-wide perfmon requires that you create as many contexts and you > have CPU core to monitor. That each context is loaded onto the right CPU > core via pfm_load_context(). On IA-64 currently, you are using the v2.0 > interface which has the PFM_LOAD_CONTEXT command. On this version, perfmon > will pick the CPU used to mke the PFM_LOAD_CONTEXT call as the CPU to > monitor. This means you need to ensure that you have pinned the caller > thread to the right CPU core prior to making the call. Each context needs > to be created with the PFM_FL_SYSTEM_WIDE flag set. > Setting the PFM_FL_SYSTEM_WIDE flag at the context creation time gives me the following error at the time I write to PMCs. PFM_WRITE_PMCS:: Invalid argument perfmonctl error PFM_WRITE_PMCS errno 22 The flags I am passing with PFM_WRITE_PMCS are : /* * indicate we want notification when buffer is full */ pc[0].reg_flags |= PFM_REGFL_OVFL_NOTIFY; /* * indicate which PMD to include in the sample */ pc[0].reg_smpl_pmds[0] = DEAR_REGS_MASK; This code works if I remove PFM_FL_SYSTEM_WIDE flag at the time of context creation. > > Currently, I am not setting this flag. I get multiple contexts fd's - > > each per thread, for each fd i use fcntl with F_SETOWN to give the > > creator thread the ownership of that fd. Then i make each fd's > > asynchronous (using fcntl with O_ASYNC) and use Linux's sideaffect > > mechanism of F_SETSIG to get individual signals to each thread (i.e. for > > a file descriptor fd(x) created by thread(x), thread(x) receives the > > SIGIO signal signalling the overflow). > > The rest of the control flow remains identical to per-thread monitoring. > So if you want to receive asynchronous notification with signal, you have > to do as you described. > > > However, I observe that when running with 4 threads on 4 cores , one > > thread hardly receives overflow notifications. Similarly, running with > > higher number of threads(eg 16), only a few threads get overflow > > notifications and the others get very little. What could be wrong here ? > > Is there anything i need to do to program perfmon ? > > Check that you are running on the right CPU before calling > PFM_LOAD_CONTEXT. yes, I checked them using sched_getaffinity(gettid(),...) call. Affinities are correct ! > > PS : This mechanism used to work well when I was running with > > LinuxThreads (1 thread = 1 pid model). I then always used getpid() > > instead of gettid(). My current system however doesn't support > > LD_ASSUME_KERNEL environment variable. So, I have to run it with NPTL > > threads now. > > With NPTL, you need to use gettid() for the F_SETOWN call. Yes, I am using gettid() with F_SETOWN. > > > I have pfmon version 3.2 and i am using libpfm-3.1. kernel is 2.6.18 > > Your library seems a bit old. Why not use libpfm-3.2? -- Okay, I have migrated to version 3.2 but that didn't help! Regards, Vivek (MS-Thesis) CS Department, NCSU _______________________________________________ perfmon mailing list [email protected] http://www.hpl.hp.com/hosted/linux/mail-archives/perfmon/
