Hello All,
Is there a way to make the "shape" of topology parallelism explicit?
For example, assume:
* W workers on W nodes
* 2 spouts with 1 executor each (total of 2 executors)
* 1 "mapper" bolt with M executors
* 1 "process" bolt with N executors.
Is it currently possibly to co-locate the spouts with the "mappers" in 2
workers, while pinning the N "process" executors to the remaining (W-2)
workers?
Visually this is what I want to do:
2 Workers W-2 Workers
+-----------+
+----------+ | process |
| spout | +-----------+
| + +--> +-----------+
| map | | process |
+----------+ +-----------+
+-----------+
+----------+ | process |
| spout | +-----------+
| + +--> ...
| map | +-----------+
+----------+ | process |
+-----------+
+-----------+
| process |
+-----------+
Thanks,
Arthur
