AM
To: "dev@storm.apache.org<mailto:dev@storm.apache.org>"
<dev@storm.apache.org<mailto:dev@storm.apache.org>>, Bobby Evans
<ev...@yahoo-inc.com<mailto:ev...@yahoo-inc.com>>
Cc: Roshan Naik <ros...@hortonworks.com<mailto:ros...@hortonworks.com>>
Also your bolt may be pending on a call to an external resource (e.g. DB)
and thus not consuming much CPU despite a relatively high usage.
On Wed, Aug 24, 2016 at 10:09 AM, Bobby Evans
wrote:
> But CGroups is restricting the actual CPU usage and scheduling is taking
But CGroups is restricting the actual CPU usage and scheduling is taking the
CPU usage into account so as to not overload a box. You can use the latency to
guess how much CPU is being used, but that only works for a single threaded
bolt/spout. Not all bolts/spouts are single threaded. Think
On 8/22/16, 6:55 AM, "Bobby Evans" wrote:
>Getting the CPU used for a worker is simple, but getting the CPU used for
>individual components is not so simple/almost impossible for
>multi-threaded bolts/spouts. The current scheduling assumes that the CPU
>for all
That is something that we have been thinking about for a while (elasticity in a
topology). There are a lot of obstacles to overcome, beyond just the
scheduler.
1) The metrics feedback loop is far from ideal in being able to automatically
detect a bottleneck. Capacity kind of works, but the
Auto scaling doesn't exist right now. And yes adding it will definitely add
a lot of value
On Aug 21, 2016 6:38 PM, "Aakash Khochare"
wrote:
Greetings,
I am Aakash Khochare. I am currently a Masters student in Indian Institute
of Science, Bangalore. I know that
Greetings,
I am Aakash Khochare. I am currently a Masters student in Indian Institute of
Science, Bangalore. I know that the number of worker processes to be used by a
topology and the number of executors can be changed using storm rebalance.
However, is there any facility that does this work