How to define grouping in a Topology with an 'a priori' unknown number of streams to subscribe?

[Susana González]

Hi,


I need help to go from a simple Storm topology like this:



TopologyBuilder builder = *new* TopologyBuilder();

builder.setSpout("mySpout", spout, spoutParallelism);

builder.setBolt("myBoltA", boltA, boltAParallelism).shuffleGrouping(
"mySpout");

builder.setBolt("myBoltB", boltB, boltBParallelism).shuffleGrouping(
"myBoltA");



to a new topology  where there are several processes myBoltA* whose emitted
tuples need to be processed by a same process boltB.


The problem is that the number of processes myBoltA* I have is read from a
configuration file when the topology is started, so I don't know them a
priori to define the grouping in the code just with:



builder.setBolt("myBoltB", boltB, boltBParallelism)

   .shuffleGrouping("myBoltA1")

   .shuffleGrouping("myBoltA2")

   .etc...

   .shuffleGrouping("myBoltAn");



I've searched if it's possible to do it using a CustomStreamGrouping or
using Trident... but I haven't found how to implement it yet.



Any idea?



Thanks in advance!

     Susana