Most of the people who have been doing resource feedback to storm have built an
external feedback system, and written a scheduler that knows how to access that
external data. Typically this information would be send as part of the worker
heartbeat. You might want to look at doing that, but you are going to have to
extend how it does the heartbeat, because most of the metrics are on a per task
basis, and CPU is very difficult to capture per task. Capacity is a very rough
approximation of this, but it only works if the bolt is truly single threaded.
You could use it and the actual CPU measurement for the worker to approximate
the CPU utilization for a single task, but it is not going to be simple.
Especially if you use a Shell bolt or Shell spout, as they are not even part of
the same process.
- Bobby
On Monday, May 25, 2015 3:25 AM, 陳怡仁 <[email protected]> wrote:
thanks for your tips again,
These days I just find out a interesting paper on implementing a storm
plug-gable scheduler that can run-time measure CPU utilization of each worker
node. For Storm 0.9.4, it can not do that, right? where i can try to add extra
measuring code into Storm to support measurement mechanism?
Best regards, -YR
