Robert Joseph Evans created STORM-1328:
------------------------------------------
Summary: Evaluate/Port JStorm tuple groupings
Key: STORM-1328
URL: https://issues.apache.org/jira/browse/STORM-1328
Project: Apache Storm
Issue Type: New Feature
Reporter: Robert Joseph Evans
Storm:
Load aware balancing in shuffle grouping
JStorm:
Has a "localfirst" grouping that causes tuples to be sent to the tasks in
the same worker by default. But if the load of all local tasks is high, the
tuples will be sent out to remote tasks.
Improve localOrShuffle grouping from Storm. In Storm's localOrShuffle
grouping the definition of "local" is local within the same worker process.
i.e., if there is a bolt that the component can send to in the current worker
process it will send the tuples there. If there is not one, it will do round
robin between all of the instances of that bolt no matter which hosts they are
on. JStorm has extended that so that other workers/JVMs on the same host are
considered "local" as well, taking into account the load of the network
connections on the local worker.
We should look at combining both of these to have shuffle look at both distance
and load to decide where to send a tuple, in addition the the new JStorm
localfirst grouping.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
