Thomas L Redman created STORM-3716:
--------------------------------------
Summary: Nodes underutilized
Key: STORM-3716
URL: https://issues.apache.org/jira/browse/STORM-3716
Project: Apache Storm
Issue Type: Bug
Components: storm-core
Affects Versions: 2.1.0
Reporter: Thomas L Redman
Topologies employing anchored tuples do not distribute across multiple nodes,
regardless of the computation demands of the bolts. It works fine on a single
node, but when throwing multiple nodes into the mix, only one machine gets
pegged. When we disable anchoring, it will distribute across all nodes just
fine, pegging each machine appropriately.
This bug manifests from version 2.1 forward. I first encountered this issue
with my own production cluster on an app that does significant NLP computation
across hundreds of millions of documents. This topology is fairly complex, so I
developed a very simple exemplar that demonstrates the issue with only one
spout and bolt. I pushed this demonstration up to github to provide the
developers with a mechanism to easily isolate the bug, and maybe provide some
workaround. I used gradle to build this simple topology and software and
package the results. This code is well documented, so it should be fairly
simple to reproduce the issue. I first encountered this issue on 3 32 core
nodes, but when I started experimenting, I set up a test cluster with 8 cores,
and then I increased each node to 16 cores, and plenty of memory in every case.
The topology can be accessed from github at
https://github.com/cowchipkid/storm-issue.git
<https://github.com/cowchipkid/storm-issue.git>.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
