[
https://issues.apache.org/jira/browse/STORM-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Li updated STORM-2738:
----------------------------
Description:
*Problem*:
If topology.acker.executors is not set, the number of ackers will be equal to
topology.workers. But on RAS cluster, we don't set topology.workers because the
number of workers will be determined by the scheduler. So in this case, the
number of ackers will always be 1 (see attached screenshot)
*Analysis*:
The number of ackers has to be computed before scheduling happens, so it knows
how to schedule the topology. The number of workers is not set until the
topology is scheduled, so it is a bit of a chicken and egg problem.
*Solution*:
We could probably use the total amount of requested memory when the topology is
submitted divided by the memory per worker to get an estimate that is better
than 1.
was:
I am pushing back our internal code change.
*Problem*:
If topology.acker.executors is not set, the number of ackers will be equal to
topology.workers. But on RAS cluster, we don't set topology.workers because the
number of workers will be determined by the scheduler. So in this case, the
number of ackers will always be 1 (see attached screenshot)
*Analysis*:
The number of ackers has to be computed before scheduling happens, so it knows
how to schedule the topology. The number of workers is not set until the
topology is scheduled, so it is a bit of a chicken and egg problem.
*Solution*:
We could probably use the total amount of requested memory when the topology is
submitted divided by the memory per worker to get an estimate that is better
than 1.
> The number of ackers should default to the number of actual running workers
> on RAS cluster
> ------------------------------------------------------------------------------------------
>
> Key: STORM-2738
> URL: https://issues.apache.org/jira/browse/STORM-2738
> Project: Apache Storm
> Issue Type: Bug
> Reporter: Ethan Li
> Assignee: Ethan Li
> Priority: Minor
> Attachments: Screen Shot 2017-09-13 at 11.13.41 AM.png
>
>
> *Problem*:
> If topology.acker.executors is not set, the number of ackers will be equal
> to topology.workers. But on RAS cluster, we don't set topology.workers
> because the number of workers will be determined by the scheduler. So in this
> case, the number of ackers will always be 1 (see attached screenshot)
> *Analysis*:
> The number of ackers has to be computed before scheduling happens, so it
> knows how to schedule the topology. The number of workers is not set until
> the topology is scheduled, so it is a bit of a chicken and egg problem.
> *Solution*:
> We could probably use the total amount of requested memory when the topology
> is submitted divided by the memory per worker to get an estimate that is
> better than 1.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
