Just a guess, but if you said:
topologyBuilder.setBolt("green-bolt", new GreenBolt(), 2).setNumTasks(1)
(2 executor threads, but only 1 task)
Then you'd get two executors with 1 task each?
On 02/02/2015 08:15 PM, Kosala Dissanayake wrote:
Yes it specifies the number of executors.
*But by default, storm assigns one task per executor. *
Therefore, when you set the number of executors, by default, this will
be equal to the number of tasks.
If you wish to, you can override the one task per executor default and
manually set the number of tasks
topologyBuilder.setBolt("green-bolt", new GreenBolt(), 2).setNumTasks(4)
This will set 4 tasks to the green bolt. Since the parallelism hint is
2, there will be 2 executors. Therefore, each executor will get 4/2 =
2 tasks.
Your other question is whether we can do this
topologyBuilder.setBolt("green-bolt", new GreenBolt(),
2).setNumTasks(1) (2 executor threads, but only 1 task)
I do not know off the top of my head but I suspect that you can't do
this. Maybe you can it out.
You can read this thread as well to understand this better
http://stackoverflow.com/questions/20371073/how-to-tune-the-parallelism-hint-in-storm
On Tue, Feb 3, 2015 at 10:44 AM, clay teahouse <[email protected]
<mailto:[email protected]>> wrote:
According to the storm docs, as of Storm 0.8
the/parallelism_hint/ parameter now specifies the initial number
of executors (not tasks!) for that bolt.
I assume this means that number of tasks and the number of
parallelism hints have to be set separately and the number of
executor threads (i.e., the parallelism hint) does not set the
value of the number of tasks. Hence my question, what happens if
the number of tasks is less than the number of executors?
On Mon, Feb 2, 2015 at 5:37 PM, Kosala Dissanayake
<[email protected] <mailto:[email protected]>> wrote:
1. The default number of tasks for an executor thread is 1. So
if you just specify the parallelism for a bolt as 3, you will
have 3 executor threads. Since the default num of tasks for an
executor thread is 1, this means you will get 3 tasks.
You can increase the number of tasks for an executor thread
from the default 1 by using setNumTasks.
2. I don't think that's possible.
3. No. The number of workers is just the number of processes
to which executors can be allocated. If you have less workers,
more executors will be assigned to each worker and vice versa.
http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/
is
a pretty good introduction to these concepts
On Mon, Feb 2, 2015 at 9:54 AM, clay teahouse
<[email protected] <mailto:[email protected]>> wrote:
Hi,
I have a few simple questions.
1)In storm .9.x, what is the default value for the bolt
num tasks? According to the docs, the parallelism hint no
longer sets the number of tasks, but the number of
executor threads.
2)What happens if the number of tasks is less than the
number of threads? Should I assume this results in idle
threads?
3)Does the number of workers multiplies the number of
tasks and threads?
feedback appreciated,
Clay
