Re: [gentoo-user] [OT] Calculating power consumption of a running program?

2017-08-23 Thread R0b0t1
On Wed, Aug 23, 2017 at 5:49 PM, Alan McKinnon  wrote:
> On 24/08/2017 00:29, R0b0t1 wrote:
>> As an example, I am interested in characterizing the power consumption
>> of rendering a PDF document. I would hopefully only need to run the
>> renderer once.
>>
>> I can use PowerTOP, but it seems to be limited to rough measurements
>> on the order of tenths of a watt. This measurement can be divided
>> among the wakeup events in an attempt to calculate software power
>> consumption but it seems imperfect if I want to monitor a single
>> process that may be competing relatively equally for resources with
>> the kernel and other user processes.
>>
>> PMBus is a spinoff of SMBus which is a spinoff of I2C which is found
>> on many motherboards. PMBus is supposed to be the interface which
>> controls and reports power supply activity. Besides the main kW power
>> supply, there is usually a power supply near your processor that steps
>> down 3.3V or 5V to 2.8V, 1.8V, or lower (I've seen as low as 0.8V, but
>> not on a desktop). I was not aware these had a visible interface.
>>
>> Apparently you can talk to these, but my searches can only find code
>> which seems highly experimental. The other replies seem to be for
>> embedded Linux systems running on FPGAs and perhaps Cortex-A parts.
>>
>> If I were using a microcontroller I could get uA or nA draw per MHz
>> and I know my operating voltage and operating time. However, desktop
>> processors are much more complex, and I am not sure if they have been
>> entirely characterized. The most advanced tool I can find is PowerTOP
>> and it does not seem very accurate.
>>
>> Does anyone have any suggestions? Should I start reading source code
>> or post on the forums?
>
> Both of these sound good
>
>> Or perhaps someone has used PowerTOP and found
>> it to be reasonably accurate?
>
> No not this. PowerTOP was designed to find badly-behaving programs like
> pidgin that woke up and polled it's queue every 1ms or so. It's not for
> what you want at all, not even close.
>

That's kind of unfortunate. It's the only software I was able to find.
There's another program, LatencyTOP, that might use API that is
relevant for my purposes (interruptible states are important for power
management), but it looks like development on it has stalled.


PoisonBL, I think my response to Alec touches on some of your points as well.

On Wed, Aug 23, 2017 at 8:39 PM, Alec Ten Harmsel
 wrote:
> On Wed, Aug 23, 2017 at 05:29:13PM -0500, R0b0t1 wrote:
>> As an example, I am interested in characterizing the power consumption
>> of rendering a PDF document. I would hopefully only need to run the
>> renderer once.
>
> I remember in college some interesting work in security-related stuff
> that was about characterizing power draw in order to crack passwords.
> Potentially that could be useful, as you could record the waveforms
> going into your power supply while rendering a PDF, but it sure sounds
> complicated.
>

That is an additional application. It's actually amazing how course
the information needed is - it was possible to listen to a switch mode
power supply's audible frequencies to recover the (asymmetric?) key
being used to encrypt data.

>> Apparently you can talk to these, but my searches can only find code
>> which seems highly experimental. The other replies seem to be for
>> embedded Linux systems running on FPGAs and perhaps Cortex-A parts.
>
> Even if you could talk to these, taking periodic measurements of
> current and voltage, then integrating over time would probably give
> results with large errors.
>

Right. However as above, my situation is a poor one, any information
is better than what I am looking at now. There are embedded (ranging
up to Cortex-A parts) boards that have very detailed power reporting
capabilities, but it adds a noticeable amount to the bill of
materials. If I am lucky the drive for lower power consumption in
laptops and desktops means these things exists in those products. I am
not sure I can access them if they do, power management in Linux still
lags behind proprietary OSes, unfortunately.

I am hoping they offer data streaming or the ability to set maximum
power output. If I can cap the power output I could compare
performance between two setpoints, although this suffers from a lot of
variability as well.

I'm not sure I can get data on or from any regulators that exist
inside the processor, or any power management facilities that are on
the processor die (a popular setup is to control a buck/boost
converter directly from the processor, as in it is integrated directly
with the processor's power management facilities; modern processors
likely have much more than that). If I could do that, that would give
me the best data I might be able to get.

>> If I were using a microcontroller I could get uA or nA draw per MHz
>> and I know my operating voltage and operating time. However, desktop
>> processors are much 

Re: [gentoo-user] [OT] Calculating power consumption of a running program?

2017-08-23 Thread Alec Ten Harmsel
On Wed, Aug 23, 2017 at 05:29:13PM -0500, R0b0t1 wrote:
> As an example, I am interested in characterizing the power consumption
> of rendering a PDF document. I would hopefully only need to run the
> renderer once.

I remember in college some interesting work in security-related stuff
that was about characterizing power draw in order to crack passwords.
Potentially that could be useful, as you could record the waveforms
going into your power supply while rendering a PDF, but it sure sounds
complicated.

> Apparently you can talk to these, but my searches can only find code
> which seems highly experimental. The other replies seem to be for
> embedded Linux systems running on FPGAs and perhaps Cortex-A parts.

Even if you could talk to these, taking periodic measurements of
current and voltage, then integrating over time would probably give
results with large errors.

> If I were using a microcontroller I could get uA or nA draw per MHz
> and I know my operating voltage and operating time. However, desktop
> processors are much more complex, and I am not sure if they have been
> entirely characterized. The most advanced tool I can find is PowerTOP
> and it does not seem very accurate.

Due to modern processors having things like pipelines and all of the
caching and memory pre-fetching going on behind the scenes, it's probably
not easy to measure this with any degree of accuracy.

Have fun,

Alec



Re: [gentoo-user] [OT] Calculating power consumption of a running program?

2017-08-23 Thread Poison BL.
On Wed, Aug 23, 2017 at 7:07 PM, Poison BL.  wrote:

> On Wed, Aug 23, 2017 at 6:29 PM, R0b0t1  wrote:
>
>> As an example, I am interested in characterizing the power consumption
>> of rendering a PDF document. I would hopefully only need to run the
>> renderer once.
>
>
> ... That's potentially useful for profiling during
> optimization of a very specific workflow in controlled conditions, but it's
> really of questionable usefulness beyond that.
>
> --
> Joshua M. Murphy
>

Also, I meant to note, the same issues that apply to power profiling of
rendering PDFs really also apply to nearly everything we tend to use
desktop systems for on a regular basis. Our datasets change in complexity
too much, many of the processes working with them are heavily impacted by
user interactivity, and background tasks and other variable factors further
complicate things so much that, aside from the extreme anomalies (like the
pidgin issue Alan noted), it's rare that a coherent power usage can be
accurately attributed to any one process.

-- 
Joshua M. Murphy


Re: [gentoo-user] [OT] Calculating power consumption of a running program?

2017-08-23 Thread Poison BL.
On Wed, Aug 23, 2017 at 6:29 PM, R0b0t1  wrote:

> As an example, I am interested in characterizing the power consumption
> of rendering a PDF document. I would hopefully only need to run the
> renderer once.


The catch with that goal is that a) rendering a PDF is likely as much of a
ram and disk intensive process as it is cpu intensive, b) the computational
complexity of it is *completely* dependent on the source's complexity, and
c) the actual power draw of the system will even vary based on external
environmental factors. For a specific document, you could measure it by
pre-loading everything into ram (so disk and ram i/o for the executable and
data to process aren't factors), ensure the system is truly idle (i.e. no
background processes, including scheduled tasks), take a power usage
measurement (preferably with an external, physical, power meter) as a
baseline, then run the process N times in a loop, and take a measurement of
both the total run time and the change in power draw. From there, you can
divide the run time by N, the multiply by the change in power draw, to get
the result. That'll tell you roughly how much power you would draw every
time you rendered that particular document, from ram, on that system, with
those exact settings, in a room at roughly that temperature and humidity.
That's potentially useful for profiling during optimization of a very
specific workflow in controlled conditions, but it's really of questionable
usefulness beyond that.

-- 
Joshua M. Murphy


Re: [gentoo-user] [OT] Calculating power consumption of a running program?

2017-08-23 Thread Alan McKinnon
On 24/08/2017 00:29, R0b0t1 wrote:
> As an example, I am interested in characterizing the power consumption
> of rendering a PDF document. I would hopefully only need to run the
> renderer once.
> 
> I can use PowerTOP, but it seems to be limited to rough measurements
> on the order of tenths of a watt. This measurement can be divided
> among the wakeup events in an attempt to calculate software power
> consumption but it seems imperfect if I want to monitor a single
> process that may be competing relatively equally for resources with
> the kernel and other user processes.
> 
> PMBus is a spinoff of SMBus which is a spinoff of I2C which is found
> on many motherboards. PMBus is supposed to be the interface which
> controls and reports power supply activity. Besides the main kW power
> supply, there is usually a power supply near your processor that steps
> down 3.3V or 5V to 2.8V, 1.8V, or lower (I've seen as low as 0.8V, but
> not on a desktop). I was not aware these had a visible interface.
> 
> Apparently you can talk to these, but my searches can only find code
> which seems highly experimental. The other replies seem to be for
> embedded Linux systems running on FPGAs and perhaps Cortex-A parts.
> 
> If I were using a microcontroller I could get uA or nA draw per MHz
> and I know my operating voltage and operating time. However, desktop
> processors are much more complex, and I am not sure if they have been
> entirely characterized. The most advanced tool I can find is PowerTOP
> and it does not seem very accurate.
> 
> Does anyone have any suggestions? Should I start reading source code
> or post on the forums? 

Both of these sound good

> Or perhaps someone has used PowerTOP and found
> it to be reasonably accurate?

No not this. PowerTOP was designed to find badly-behaving programs like
pidgin that woke up and polled it's queue every 1ms or so. It's not for
what you want at all, not even close.


> 
> R0b0t1.
> 


-- 
Alan McKinnon
alan.mckin...@gmail.com