Sachin Goyal created STORM-1686:
-----------------------------------
Summary: Make local-grouping an independent option rather than
just LOCAL_OR_SHUFFLE
Key: STORM-1686
URL: https://issues.apache.org/jira/browse/STORM-1686
Project: Apache Storm
Issue Type: Improvement
Reporter: Sachin Goyal
Currently, Storm distributes the tuples to bolts irrespective of their locality.
So if 1 spout and 5 bolts are running in the same process, storm does not give
any preference to the local 5 bolts (except for LOCAL_OR_SHUFFLE grouping).
Due to this, there is a lot of inter-machine/inter-process communication for
sending out the tuples.
It would be really good if local-mode can be made as an independent option
(*preferLocalBolts*) rather than having just a single mode LOCAL_OR_SHUFFLE
that can distribute locally.
At least for fields-grouping, the local-mode would make a lot of sense.
Currently, the alternatives to the local-mode-with-fields-grouping is:
# Run several topologies, each with 1 worker so as to limit the spouts/bolts to
one machine only.
# Have another bolt between a spout and bolt that can be used for directing the
tuple traffic by directed streams.
But both of these seem to be a lot of work for something that should be made an
independent option - *preferLocalBolts*.
Also see the following SO content:
#
[way-to-apply-multiple-groupings-in-storm|http://stackoverflow.com/questions/36368224/is-there-a-way-to-apply-multiple-groupings-in-storm/36374837#36374837]
#
[fields-grouping-for-same-machine|http://stackoverflow.com/questions/35132136/storm-fields-grouping-for-same-machine?lq=1]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
