Dan and Stephane:
I looked into this a bit more. I worked on the kernel side only of the
perfmon2 for Cell. I asked someone that I know who has done some work
on the Power libpfm code. It looks like the pfm_get_num_counters() is a
libpfm call. We looked at the libpfm code a little bit for CELL. The
function returns the #define for PMU_CELL_NUM_COUNTERS which is set to
8.
I do not know who wrote the libpfm support for cell. There is a Toshiba
copyright listed at the top of the CELL libpfm page. Stephane do you
know who contributed the libpfm code?
The Cell kernel support implementation only supports four 32 bit
counters not eight 16 bit counters. Maybe the libpfm person assumed
that the kernel supported eight counters. I don't know. We need to
have a discussion with the person who implemented the libpfm stuff to
get to the bottom of all this. I can help explain the kernel side of
things but need someone to do the user side.
I went back and looked at the kernel code again. I don't remember all
of the details off the top of my head. It looks like what we did was to
define each of the eight 16 bit counters so there would be names and all
there so in the future if we did implement variable sized counters
things would be there. However, in include/asm-powerpc/cell-pmu.h we
actually state that there are 4 physical 32 bit registers, the code is:
/* The Cell PMU has four hardware performance counters, which can be
* configured as four 32-bit counters or eight 16-bit counters.
*/
#define NR_PHYS_CTRS 4
#define NR_CTRS (NR_PHYS_CTRS * 2)
The sizing of the various arrays and for loops are all based on NR_CTRS.
If you look in arch/powerpc/perfmon/perfmon-cell.c in function
pfm_cell_write_pmc() you will see if the counter number is less then
NR_CTRS, you call write_pm07_control() to do the actual write. Hence
the code is only supporting writing to the first four counters.
The function write_pm07() gets mapped to cbe_write_pm07_control() which
is defined in arch/powerpc/platforms/cell/pmu.c.
It looks like it was left up to the call to write the control registers
to set the bit indicating that the registers are configured as 32 or 16
bit counters.
I don't remember the details but it looks like some of the kernel code
has been changed. It looks like the function pfm_cell_get_cntr_width()
is checking to determine if the pm_control register bit is set to
combine two 16 bit registers into a 32 bit register. The function
returns the actual size of the counters. I think this function was
changed but I can't say for sure. Clearly Toshiba has made changes to
the file.
Since NR_CTRS is still set to 4 for CELL, you can only use the first
four counters. The size of the counters is controlled by bits 7 to 10
(where IBM defines MSB to be bit 0) of the 32 bit pm_control register.
The mapping is bit 7 controls pm0 and pm4, bit 8 controls pm1 and pm5,
etc. If the bit is set, then pmi and pmi+4 are 16 bit counters.
Otherwise the two 16 bit counters are combined into a 32 bit counter.
I am assuming pfm_cell_get_event_code() is also a libpfm call. I have
not dug into the details of this code.
Carl Love
On Mon, 2008-06-09 at 11:23 -0400, Dan Terpstra wrote:
> More...
>
> pfm_pmu_support_t cell_support={
> ...
> .num_cnt = PMU_CELL_NUM_COUNTERS,
> ...
>
> #define PMU_CELL_NUM_COUNTERS 8 /* total number of EvtSel/EvtCtr */
>
> This again seems to imply 8 counters (PMC/PMD pairs) for Cell.
> The question is still why does pfm_cell_get_event_code hardcode this as 2,
> and where is the limit of 4 32 bit counters defined/enforced?
> - d
>
> > -----Original Message-----
> > From: Dan Terpstra [mailto:[EMAIL PROTECTED]
> > Sent: Sunday, June 08, 2008 4:16 PM
> > To: '[EMAIL PROTECTED]'; 'Carl Love'
> > Cc: 'perfmon2-devel@lists.sourceforge.net'
> > Subject: RE: [perfmon2] perfmon2 on Cell
> >
> > Guys -
> > I'm perfectly happy with 4 32-bit counters. Given the options, that's the
> > trade-off I would have made as well. My confusion lies in the fact that
> > pfm_num_counters is returning 8, and pfm_cell_get_event_code was erroring
> > out after (hard-coded) 2. Carl, two nodes doesn't work for me, because
> > each node is independent. A blade should report the same answer as a
> > Playstation. Intel Core 2 Duos don't report 10 counters because they have
> > 2 cores; just 5 counters because each core is programmed independently. I
> > think the hard-coded 2 is a simple programming bug, but I sure would like
> > some confirmation of that.
> > - dan
> >
> > > -----Original Message-----
> > > From: stephane eranian [mailto:[EMAIL PROTECTED]
> > > Sent: Friday, June 06, 2008 4:54 PM
> > > To: Carl Love
> > > Cc: Dan Terpstra; perfmon2-devel@lists.sourceforge.net
> > > Subject: Re: [perfmon2] perfmon2 on Cell
> > >
> > > Hello,
> > >
> > >
> > > On Fri, Jun 6, 2008 at 9:27 PM, Carl Love <[EMAIL PROTECTED]> wrote:
> > > > Dan:
> > > >
> > > > Perfmon2 does not support variable sized counters. It makes the
> > > > assumption that counter size is a known, fixed size. The
> > implementation
> > >
> > > Adding support for variable size counters is on my list of things to do
> > > soon.
> > > By variable it will mean that not all counters will have to have the
> > same
> > > size
> > > but that will be coded in the PMU description module. This is not going
> > to
> > > be totally dynamic, i.e., from one application to the other.
> > >
> > > > for cell was fixed to four 32 bit counters per node due to the lack of
> > > > support for a variable size. We did not choose to go with eight 16
> > bit
> > > > counters per node because the feeling was the overhead of having to
> > > > invoke the interrupt handler on a 16 bit counter would be very high.
> > > > Typically, a user will be measuring either cycles or instructions
> > along
> > > > with another event. Causing an interrupt every 2^16 cycles would be
> > > > very expensive.
> > > >
> > > > I would have to double code but does pfm_get_num_counters() return the
> > > > total number of counters in a system? On a cell system there are two
> > > > nodes. Each node has a performance counter unit with four 32 bit
> > > > counters.
> > > >
> > > > Supporting both 16 and 32 bit counters gets very complicated.
> > > > Specifically in the code for handling 64 bit virtual counters. We
> > > > talked about providing support at some future time but felt there were
> > > > other higher priority things to do first. Also, the perfmon2
> > interface
> > > > is already complex. This is one of the reason it is has not gotten
> > > > accepted into the kernel. We felt adding variable sized counter
> > support
> > > > now would only add additional complexity that would further hinder
> > > > getting it accepted. So best to try and keep things simple for now,
> > get
> > > > it accepted then incrementally add the complexity later.
> > > >
> > > > Carl Love
> > > >
> > > >
> > > > On Fri, 2008-06-06 at 15:07 -0400, Dan Terpstra wrote:
> > > >> I'm working on implementing PAPI on Cell.
> > > >> And I'm confused.
> > > >> Can someone tell me why pfm_get_num_counters() returns 8 for Cell (I
> > > think
> > > >> this is # of 16-bit counters?) when pfm_cell_get_event_code()
> > compares
> > > >> against a hard-coded limit of 2 (see below)? Could this limit simply
> > > have
> > > >> been inappropriately inherited from another platform?
> > > >> Also, how does one distinguish between using these counters in 16-bit
> > > mode
> > > >> vs. 32-bit mode?
> > > >> - d
> > > >>
> > > >> static int
> > > >> pfm_cell_get_event_code(unsigned int i, unsigned int cnt, int *code)
> > > >> {
> > > >> if (cnt != PFMLIB_CNT_FIRST && cnt > 2) {
> > > >> return PFMLIB_ERR_INVAL;
> > > >> }
> > > >>
> > > >> *code = cell_pe[i].pme_code;
> > > >>
> > > >> return PFMLIB_SUCCESS;
> > > >> }
> > > >>
> > > >>
> > > >> ---------------------------------------------------------------------
> > --
> > > --
> > > >> Check out the new SourceForge.net Marketplace.
> > > >> It's the best place to buy or sell services for
> > > >> just about anything Open Source.
> > > >> http://sourceforge.net/services/buy/index.php
> > > >> _______________________________________________
> > > >> perfmon2-devel mailing list
> > > >> perfmon2-devel@lists.sourceforge.net
> > > >> https://lists.sourceforge.net/lists/listinfo/perfmon2-devel
> > > >
> > > >
> > > > ----------------------------------------------------------------------
> > --
> > > -
> > > > Check out the new SourceForge.net Marketplace.
> > > > It's the best place to buy or sell services for
> > > > just about anything Open Source.
> > > > http://sourceforge.net/services/buy/index.php
> > > > _______________________________________________
> > > > perfmon2-devel mailing list
> > > > perfmon2-devel@lists.sourceforge.net
> > > > https://lists.sourceforge.net/lists/listinfo/perfmon2-devel
> > > >
>
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
perfmon2-devel mailing list
perfmon2-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/perfmon2-devel
