Hello Thomas, Apologies for delay in responding here. I tested the topology code provided in storm-issue repo. *only one machine gets peggeg*: Although it appears, his is not a bug. This is related to Locality Awareness. Please refer to https://github.com/apache/storm/blob/master/docs/LocalityAwareness.md It appears spout to bolt ratio is 200, so if there are enough bolt's on single node to handle events generated by the spout, it won't send events out to another node unless it runs out of capacity on single node. If you do not like this and want to distribute events evenly, you can try disabling this feature. You can turn off LoadAwareShuffleGrouping by setting topology.disable.loadaware.messaging to true. -Kishor
On 2020/10/28 15:21:54, "Thomas L. Redman" <tomred...@mchsi.com> wrote: > What’s the word on this? I sent this out some time ago, including a GitHub > project that clearly demonstrates the brokenness, yet I have not heard a > word. Is there anybody supporting Storm? > > > On Sep 30, 2020, at 9:03 AM, Thomas L. Redman <tomred...@mchsi.com> wrote: > > > > I believe I have encountered a significant bug. It seems topologies > > employing anchored tuples do not distribute across multiple nodes, > > regardless of the computation demands of the bolts. It works fine on a > > single node, but when throwing multiple nodes into the mix, only one > > machine gets pegged. When we disable anchoring, it will distribute across > > all nodes just fine, pegging each machine appropriately. > > > > This bug manifests from version 2.1 forward. I first encountered this issue > > with my own production cluster on an app that does significant NLP > > computation across hundreds of millions of documents. This topology is > > fairly complex, so I developed a very simple exemplar that demonstrates the > > issue with only one spout and bolt. I pushed this demonstration up to > > github to provide the developers with a mechanism to easily isolate the > > bug, and maybe provide some workaround. I used gradle to build this simple > > topology and software and package the results. This code is well > > documented, so it should be fairly simple to reproduce the issue. I first > > encountered this issue on 3 32 core nodes, but when I started > > experimenting, I set up a test cluster with 8 cores, and then I increased > > each node to 16 cores, and plenty of memory in every case. > > > > The topology can be accessed from github at > > https://github.com/cowchipkid/storm-issue.git > > <https://github.com/cowchipkid/storm-issue.git>. Please feel free to > > respond to me directory if you have any questions that are beyond the scope > > of this mail list. > >
