Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-27 Thread Alexandre Montplaisir
Hi François, Very interesting! Some more comments below On 13-02-26 04:07 PM, François Rajotte wrote: > [...] > > By using the kernel state system, we have effectively halved the amount of > information we have to store in the statistics state system. > By doing so, we have also heavily coupled t

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-26 Thread François Rajotte
Hello all, I have implemented some of the ideas discussed here, mostly as a proof of concept. The idea was to see where were the major hurdles in implementing more advanced statistics in the current framework. For simplicity, I have only implemented the cpu usage of every TID for each CPU. For s

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-15 Thread Michel Dagenais
> suppose we want to calculate per - cpu CPU utilization of > each process > I n the meantime , suppose we are also interested to have a > reverse statistics: per - process CPU utilization for each CPU > By using the current organization of the attribute tree , we may > need to duplicate the

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-14 Thread Naser Ezzati
Hello I want to mention a general case, in which we want to support "group by" queries for different attributes and resources. Regarding the problem mentioned by François, suppose we want to calculate * per*-*cpu CPU utilization of each process* (select CPU usages for each CPU separately, group

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-12 Thread Alexandre Montplaisir
On 13-02-12 04:45 PM, François Rajotte wrote: > > I believe the main concern was if we want the information about _many_ > processes at a given timestamp. > If only one query is necessary (Michel's proposal), then we are sure > to have all the information in one query. (constant in the number of >

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-12 Thread François Rajotte
On Wed, Feb 13, 2013 at 9:26 AM, Alexandre Montplaisir < alexmon...@voxpopuli.im> wrote: > > >> this method only requires a single "attribute" to store the cumulative > time and whether or not the process is in an active interval. Its value is > changed at schedule in and at schedule out. However,

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-12 Thread Michel Dagenais
> If you fall on a "real" interval, there is no additional queries. If > you fall on a null interval, you have to do one additional query, to get the > previous interval. If you want to query a range, you have to do 2 > queries already, by definition. So in the worst case, you end up doing 4 > que

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-12 Thread Alexandre Montplaisir
>> this method only requires a single "attribute" to store the cumulative time >> and whether or not the process is in an active interval. Its value is >> changed at schedule in and at schedule out. However, you need to do >> additional queries. For instance, if you want to show the CPU usage p

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-12 Thread Matthew Khouzam
On 13-02-12 08:18 AM, Michel Dagenais wrote: >> - If you fall on a "real" interval, you know you are in a time where the >> process was active on the CPU. You will need to do the interpolation >> between the value (which represents the cumulative amount at the >> start of the interval), the length

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-12 Thread Michel Dagenais
> - If you fall on a "real" interval, you know you are in a time where the > process was active on the CPU. You will need to do the interpolation > between the value (which represents the cumulative amount at the > start of the interval), the length of the interval, and the current > position in t

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-11 Thread Alexandre Montplaisir
Some other considerations that came to mind after writing the whole thing: 1 - In fact that state history would NOT be storable in a partial history (aka "partializable"). The partial history only gives us the values at the queried times, not the whole intervals. It can give us the start time, but

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-11 Thread Alexandre Montplaisir
Hi François, This would indeed be a nice addition. It would require storing more data in the state system compared to what we have now (and I'm not sure if it would "partializable", probably but we would have to check for corner cases). We did an experiment about this a couple weeks back, it's a

Re: [linuxtools-dev] [TMF] Advanced statistics view

2013-02-11 Thread Michel Dagenais
> The structure could look like this : > > Process #1 : 25% total > -> CPU0 : 20% > -> CPU1 : 5% I suppose that you first need to compute the CPU usage per process while building the state history, summing it up each time it is scheduled out. The usage up to t0 then becomes the sum at t0 (