[
https://issues.apache.org/jira/browse/STORM-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425223#comment-16425223
]
Roshan Naik commented on STORM-2983:
------------------------------------
Mutability of topology configuration is explicitly allowed in the documentation
(previously referenced).
Usefulness of single worker use case should not be under-estimated. For perf
critical topos it is a reasonable option to consider deploying multiple single
worker instances of the topo as opposed to a single multiworker instance to
avoid interworker communication. I have recommended users on mailing list to
consider it (for things like video processing) and I have seen others recommend
considering 1worker mode as well.
Given the fact that RA creates the divergence in specification v/s reality
confusion is bound to exist no matter which we resolve this value of this
setting. Although IMO it is not a strong enough case to not reflect actual
worker count ... if it is contentious ... I am ok to resolve it either way ....
as long we provide an API for so that code can check. Currently I am aware of
a few additional use cases for knowing worker count.
* enable/disable interworker BP subsystem.(already in place)
* auto disabling Locality Awareness for 1 worker (i have measured this impact
to be a bit under 10% or something for 1 worker)
* user bolts inspecting this to make decisions.. like potentially allocate a
certain number of external resources based on worker count... or who knows what.
My concern is limited to the fact that we must have official support to be able
to detect the worker count. It is simply too heavy handed to simply say NO to
all current and potential use cases.
> Some topologies not working properly
> -------------------------------------
>
> Key: STORM-2983
> URL: https://issues.apache.org/jira/browse/STORM-2983
> Project: Apache Storm
> Issue Type: Bug
> Reporter: Ethan Li
> Assignee: Ethan Li
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For example,
> {code:java}
> bin/storm jar storm-loadgen-*.jar
> org.apache.storm.loadgen.ThroughputVsLatency --spouts 1 --splitters 2
> --counters 1 -c topology.debug=true
> {code}
> on ResourceAwareScheduler not working properly.
> With default cluster settings, there will be only one __acker-executor and it
> will be on a separate worker. And it looks like the __acker-executor was not
> able to receive messages from spouts and bolts. And spouts and bolts
> continued to retry sending messages to acker. It then led to another problem:
> STORM-2970
> I tried to run on storm right before
> [https://github.com/apache/storm/pull/2502] and right after and confirmed
> that this bug should be related to it
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
