Zhuo Liu created STORM-1657:
-------------------------------
Summary: A new strategy in RAS that allows one worker to run
executors from one component
Key: STORM-1657
URL: https://issues.apache.org/jira/browse/STORM-1657
Project: Apache Storm
Issue Type: Story
Reporter: Zhuo Liu
There are still lots of complaints from users and industry saying that Storm is
hard to debug, mainly because Storm put executors from different components
(spout, bolt) to run in a single JVM process.
The original motivation for doing that is to allow inter-component
communication to happen within a single process rather than over network socket
as much as possible.
For small topology, this seems make sense, where we can place executors even
within one or several worker. But for larger topology with more executors in
each component, the saving in network transfer by using round-robin executor
distribution to different workers is marginal. And the trade-off is that we
will lose lots of debuggability, which makes problem-cracking difficult and
prevents some potential users to start using Storm.
With RAS (STORM-894), it should be convenient for us to add a new strategy to
allow one worker to only run executors (>= 1) from a single component, which
will increase a few workers but not waste any memory /CPU resources.
By using this strategy (per-topology basis), it will be easier for users to
find which component in a topology is having problems by checking their worker
log, gc log and heap dump, without being bothered by noise made by other
components' instances.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
