Hello Hadoop Team, I need some help/support in evaluating Hadoop YARN from the perspective of:
- Adding/removing test nodes dynamically without restart Is this possible in YARN? or is it only a MapReduce application feature? - Failover from crashed test nodes Is this automatically done by the YARN framework? - Simple prioritization of jobs I know there is a prioritization possibility of the tasks in the Scheduler, right? - Insight into queue (simple reporting, including running jobs on nodes) Is there any reporting possibility in YARN regarding the status of the not executed, running, finished jobs? - Support heterogenous OSes on nodes (or use separate masters for homogenous grids) I think this is supported, right? Another point I would like to evaluate is the Distributed Shell example usage. Our use case is to start different scripts on a grid. Once a node has finished a script a new script has to be started on it. A report about the scripts execution has to be provided. in case a node has failed to execute a script it should be re-executed on a different node. Some scripts are Windows specific other are Unix specific and have to be executed on a node with a specific OS. The question is: Would it be feasible to adapt the example "Distributed Shell" application to have the above features? If yes how could I run some specific scripts only on a specific OS? Is this the ResourceManager responsability? What happens if there is no Windows node for example in the grid but in the queue there is a Windows script? How to re-execute failed scripts? Does it have to be implemented by custom code, or is it a built in feature of YARN? Thank you in advance for your support, Ioan Zeng
