[
https://issues.apache.org/jira/browse/STORM-2286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated STORM-2286:
----------------------------------
Labels: pull-request-available (was: )
> Storm Rebalance command should support arbitrary component parallelism
> ----------------------------------------------------------------------
>
> Key: STORM-2286
> URL: https://issues.apache.org/jira/browse/STORM-2286
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
> Affects Versions: 0.9.3, 0.9.6, 0.10.1, 1.0.1
> Reporter: Yuzhao Chen
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> For legacy reasons, config TOPOLOGY-TASKS is considered first when schedule a
> topology, for a component, if user don’t specify TOPOLOGY-TASKS, storm just
> override it to be equal to component parallelism hint, and schedule based on
> TOPOLOGY-TASKS later on.
> This works for the most cases, but not Rebalance command. Now, when do
> Rebalance, the StormBase :component->executors attribute will be overridden
> in Zookeeper which is used to partition component tasks into executors, as we
> said above, the TOPOLOGY-TASKS is considered here as the real tasks number
> for components, something goes weird here:
> If we override a bigger executor numbers for a component when do rebalance,
> it just don’t work because smaller TOPOLOGY-TASKS [ not changed since first
> submitted at all ]is partitioned into bigger number of executors which read
> from ZooKeeper overridden by Rebalance command, but for smaller task, it
> works fine.
> I see that storm support a command like this now: [storm rebalance
> topology-name [-w wait-time-secs] [-n new-num-workers] [-e
> component=parallelism]*] which indicate that user can override a component
> parallelism freely, i think it’s more sensible to support this and it's
> meaningless to have a restriction like before.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
