Meanwhile, Stephane, can you push this single line fix into libpfm?
It ties the cell_support.num_cnt value to this check instead of hard-coding
it. This gives me a workaround for PAPI until the issue is more
fundamentally resolved.
Thanks,
- d
diff B C:/Documents and Settings/terpstra/Local
Settings/Temp/pfmlib_cell_1.1.1.3_21.c
C:/papiHEAD/src/libpfm-3.y/lib/pfmlib_cell.c
577c577,578
< if (cnt != PFMLIB_CNT_FIRST && cnt > 2) {
---
> // if (cnt != PFMLIB_CNT_FIRST && cnt > 2) {
> if (cnt != PFMLIB_CNT_FIRST && cnt > cell_support.num_cnt) {
> -----Original Message-----
> From: stephane eranian [mailto:[EMAIL PROTECTED]
> Sent: Monday, June 09, 2008 4:03 PM
> To: Carl Love
> Cc: Dan Terpstra; perfmon2-devel@lists.sourceforge.net
> Subject: Re: [perfmon2] perfmon2 on Cell
>
> Carl,
>
> Concerning Tohisba, I think you should ask Takayuki Uchikawa
> <[EMAIL PROTECTED]>.
>
>
> On Mon, Jun 9, 2008 at 9:53 PM, Carl Love <[EMAIL PROTECTED]> wrote:
> >
> >
> > Dan and Stephane:
> >
> > I looked into this a bit more. I worked on the kernel side only of the
> > perfmon2 for Cell. I asked someone that I know who has done some work
> > on the Power libpfm code. It looks like the pfm_get_num_counters() is a
> > libpfm call. We looked at the libpfm code a little bit for CELL. The
> > function returns the #define for PMU_CELL_NUM_COUNTERS which is set to
> > 8.
> >
> > I do not know who wrote the libpfm support for cell. There is a Toshiba
> > copyright listed at the top of the CELL libpfm page. Stephane do you
> > know who contributed the libpfm code?
> >
> > The Cell kernel support implementation only supports four 32 bit
> > counters not eight 16 bit counters. Maybe the libpfm person assumed
> > that the kernel supported eight counters. I don't know. We need to
> > have a discussion with the person who implemented the libpfm stuff to
> > get to the bottom of all this. I can help explain the kernel side of
> > things but need someone to do the user side.
> >
> > I went back and looked at the kernel code again. I don't remember all
> > of the details off the top of my head. It looks like what we did was to
> > define each of the eight 16 bit counters so there would be names and all
> > there so in the future if we did implement variable sized counters
> > things would be there. However, in include/asm-powerpc/cell-pmu.h we
> > actually state that there are 4 physical 32 bit registers, the code is:
> >
> > /* The Cell PMU has four hardware performance counters, which can be
> > * configured as four 32-bit counters or eight 16-bit counters.
> > */
> > #define NR_PHYS_CTRS 4
> > #define NR_CTRS (NR_PHYS_CTRS * 2)
> >
> > The sizing of the various arrays and for loops are all based on NR_CTRS.
> >
> > If you look in arch/powerpc/perfmon/perfmon-cell.c in function
> > pfm_cell_write_pmc() you will see if the counter number is less then
> > NR_CTRS, you call write_pm07_control() to do the actual write. Hence
> > the code is only supporting writing to the first four counters.
> >
> > The function write_pm07() gets mapped to cbe_write_pm07_control() which
> > is defined in arch/powerpc/platforms/cell/pmu.c.
> >
> > It looks like it was left up to the call to write the control registers
> > to set the bit indicating that the registers are configured as 32 or 16
> > bit counters.
> >
> > I don't remember the details but it looks like some of the kernel code
> > has been changed. It looks like the function pfm_cell_get_cntr_width()
> > is checking to determine if the pm_control register bit is set to
> > combine two 16 bit registers into a 32 bit register. The function
> > returns the actual size of the counters. I think this function was
> > changed but I can't say for sure. Clearly Toshiba has made changes to
> > the file.
> >
> > Since NR_CTRS is still set to 4 for CELL, you can only use the first
> > four counters. The size of the counters is controlled by bits 7 to 10
> > (where IBM defines MSB to be bit 0) of the 32 bit pm_control register.
> > The mapping is bit 7 controls pm0 and pm4, bit 8 controls pm1 and pm5,
> > etc. If the bit is set, then pmi and pmi+4 are 16 bit counters.
> > Otherwise the two 16 bit counters are combined into a 32 bit counter.
> >
> > I am assuming pfm_cell_get_event_code() is also a libpfm call. I have
> > not dug into the details of this code.
> >
> > Carl Love
> >
> > On Mon, 2008-06-09 at 11:23 -0400, Dan Terpstra wrote:
> >> More...
> >>
> >> pfm_pmu_support_t cell_support={
> >> ...
> >> .num_cnt = PMU_CELL_NUM_COUNTERS,
> >> ...
> >>
> >> #define PMU_CELL_NUM_COUNTERS 8 /* total number of EvtSel/EvtCtr */
> >>
> >> This again seems to imply 8 counters (PMC/PMD pairs) for Cell.
> >> The question is still why does pfm_cell_get_event_code hardcode this as
> 2,
> >> and where is the limit of 4 32 bit counters defined/enforced?
> >> - d
> >>
> >> > -----Original Message-----
> >> > From: Dan Terpstra [mailto:[EMAIL PROTECTED]
> >> > Sent: Sunday, June 08, 2008 4:16 PM
> >> > To: '[EMAIL PROTECTED]'; 'Carl Love'
> >> > Cc: 'perfmon2-devel@lists.sourceforge.net'
> >> > Subject: RE: [perfmon2] perfmon2 on Cell
> >> >
> >> > Guys -
> >> > I'm perfectly happy with 4 32-bit counters. Given the options, that's
> the
> >> > trade-off I would have made as well. My confusion lies in the fact
> that
> >> > pfm_num_counters is returning 8, and pfm_cell_get_event_code was
> erroring
> >> > out after (hard-coded) 2. Carl, two nodes doesn't work for me,
> because
> >> > each node is independent. A blade should report the same answer as a
> >> > Playstation. Intel Core 2 Duos don't report 10 counters because they
> have
> >> > 2 cores; just 5 counters because each core is programmed
> independently. I
> >> > think the hard-coded 2 is a simple programming bug, but I sure would
> like
> >> > some confirmation of that.
> >> > - dan
> >> >
> >> > > -----Original Message-----
> >> > > From: stephane eranian [mailto:[EMAIL PROTECTED]
> >> > > Sent: Friday, June 06, 2008 4:54 PM
> >> > > To: Carl Love
> >> > > Cc: Dan Terpstra; perfmon2-devel@lists.sourceforge.net
> >> > > Subject: Re: [perfmon2] perfmon2 on Cell
> >> > >
> >> > > Hello,
> >> > >
> >> > >
> >> > > On Fri, Jun 6, 2008 at 9:27 PM, Carl Love <[EMAIL PROTECTED]> wrote:
> >> > > > Dan:
> >> > > >
> >> > > > Perfmon2 does not support variable sized counters. It makes the
> >> > > > assumption that counter size is a known, fixed size. The
> >> > implementation
> >> > >
> >> > > Adding support for variable size counters is on my list of things
> to do
> >> > > soon.
> >> > > By variable it will mean that not all counters will have to have
> the
> >> > same
> >> > > size
> >> > > but that will be coded in the PMU description module. This is not
> going
> >> > to
> >> > > be totally dynamic, i.e., from one application to the other.
> >> > >
> >> > > > for cell was fixed to four 32 bit counters per node due to the
> lack of
> >> > > > support for a variable size. We did not choose to go with eight
> 16
> >> > bit
> >> > > > counters per node because the feeling was the overhead of having
> to
> >> > > > invoke the interrupt handler on a 16 bit counter would be very
> high.
> >> > > > Typically, a user will be measuring either cycles or instructions
> >> > along
> >> > > > with another event. Causing an interrupt every 2^16 cycles would
> be
> >> > > > very expensive.
> >> > > >
> >> > > > I would have to double code but does pfm_get_num_counters()
> return the
> >> > > > total number of counters in a system? On a cell system there are
> two
> >> > > > nodes. Each node has a performance counter unit with four 32 bit
> >> > > > counters.
> >> > > >
> >> > > > Supporting both 16 and 32 bit counters gets very complicated.
> >> > > > Specifically in the code for handling 64 bit virtual counters.
> We
> >> > > > talked about providing support at some future time but felt there
> were
> >> > > > other higher priority things to do first. Also, the perfmon2
> >> > interface
> >> > > > is already complex. This is one of the reason it is has not
> gotten
> >> > > > accepted into the kernel. We felt adding variable sized counter
> >> > support
> >> > > > now would only add additional complexity that would further
> hinder
> >> > > > getting it accepted. So best to try and keep things simple for
> now,
> >> > get
> >> > > > it accepted then incrementally add the complexity later.
> >> > > >
> >> > > > Carl Love
> >> > > >
> >> > > >
> >> > > > On Fri, 2008-06-06 at 15:07 -0400, Dan Terpstra wrote:
> >> > > >> I'm working on implementing PAPI on Cell.
> >> > > >> And I'm confused.
> >> > > >> Can someone tell me why pfm_get_num_counters() returns 8 for
> Cell (I
> >> > > think
> >> > > >> this is # of 16-bit counters?) when pfm_cell_get_event_code()
> >> > compares
> >> > > >> against a hard-coded limit of 2 (see below)? Could this limit
> simply
> >> > > have
> >> > > >> been inappropriately inherited from another platform?
> >> > > >> Also, how does one distinguish between using these counters in
> 16-bit
> >> > > mode
> >> > > >> vs. 32-bit mode?
> >> > > >> - d
> >> > > >>
> >> > > >> static int
> >> > > >> pfm_cell_get_event_code(unsigned int i, unsigned int cnt, int
> *code)
> >> > > >> {
> >> > > >> if (cnt != PFMLIB_CNT_FIRST && cnt > 2) {
> >> > > >> return PFMLIB_ERR_INVAL;
> >> > > >> }
> >> > > >>
> >> > > >> *code = cell_pe[i].pme_code;
> >> > > >>
> >> > > >> return PFMLIB_SUCCESS;
> >> > > >> }
> >> > > >>
> >> > > >>
> >> > > >> ----------------------------------------------------------------
> -----
> >> > --
> >> > > --
> >> > > >> Check out the new SourceForge.net Marketplace.
> >> > > >> It's the best place to buy or sell services for
> >> > > >> just about anything Open Source.
> >> > > >> http://sourceforge.net/services/buy/index.php
> >> > > >> _______________________________________________
> >> > > >> perfmon2-devel mailing list
> >> > > >> perfmon2-devel@lists.sourceforge.net
> >> > > >> https://lists.sourceforge.net/lists/listinfo/perfmon2-devel
> >> > > >
> >> > > >
> >> > > > -----------------------------------------------------------------
> -----
> >> > --
> >> > > -
> >> > > > Check out the new SourceForge.net Marketplace.
> >> > > > It's the best place to buy or sell services for
> >> > > > just about anything Open Source.
> >> > > > http://sourceforge.net/services/buy/index.php
> >> > > > _______________________________________________
> >> > > > perfmon2-devel mailing list
> >> > > > perfmon2-devel@lists.sourceforge.net
> >> > > > https://lists.sourceforge.net/lists/listinfo/perfmon2-devel
> >> > > >
> >>
> >
> >
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
perfmon2-devel mailing list
perfmon2-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/perfmon2-devel
