Hi, all In HAWQ, different stages in HAWQ will be treated as the same. no matter from the scheduler view or consider the number of processes. But in some other systems like Presto, There are two schedulers, one is sourcePartitionedScheduler used to dispatch scan stage, the other is FixedCountScheduler, used to dispatch intermediate stages. I think that one is more flexible. Flexible means that we can write a new scanScheduler, which dispatches at split level, for some nodes, which are faster than others, will scan more splits than others. This strategy may reduce the average IO time. Is there any suggestion?
-- Thanks Hubert Zhang
