The reason for this design is that it allows for very basic elasticity in a topology. This only works for cpu and we are working on a better solution that involves resource aware scheduling. My guess is that once we have this in place we can remove the extra complexity.
- Bobby | From:"Chuanlei Ni" <[email protected]> Date:Mon, Jul 20, 2015 at 6:38 AM Subject:[DISCUSSION]One Task One Executor Hi, Now the executor can deal with a range of tasks. However, I cannot get the advantage of this design. If we simplify the design, we can get a more clear execution model and more clear scheduler (such as optimized for the most frequent shuffleGrouping). If you think this idea can be useful for Storm, I will create a JIRA for that and have a try to disable the setNumTasks interface. Maybe another simple scheduler can be tried. Looking forward to some suggestion. Thanks. |
