Read Sprabery created STORM-815:
-----------------------------------
Summary: Auto tuning topologies based on user submitted SLA
Key: STORM-815
URL: https://issues.apache.org/jira/browse/STORM-815
Project: Apache Storm
Issue Type: New Feature
Reporter: Read Sprabery
Priority: Blocker
Hi all – not sure if this is the place for this or not, so please feel free to
move this ticket around.
I have been working on making Storm SLA aware and wanted to create a ticket
focused on auto-tuning of submitted topologies. Having talked with [~revans2] ,
I think this feature will go a long way in improving the usability of Storm.
This will be particularly useful as more abstractions are developed on top of
Storm and developers move further away from the internals of topologies with
systems like Summingbird (https://github.com/twitter/summingbird). Storm
already has the majority of the metrics required to made decisions regarding
the stability of a given stream in Zookeeper; these can be used to decide if a
given SLA is being met.
The system I'm working on functions externally to Storm, getting measurements
from the Thrift interface and then performing parallelism level tuning through
the “rebalance” command. If the community is interested, I can upload a
document in the upcoming weeks highlighting the challenges and benefits of an
auto-tuning system in Storm and that can serve as a first start for developing
an auto-tuning scheduler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
