On Thu, 28 Jan 2010 11:12:07 -0800 Matei Zaharia <[email protected]> wrote:
> Hi Erik, > > With four priority levels like this, you should just be able to use > Hadoop's priorities, because it has five of them (very high, high, > normal, low and very low). You can just use the default scheduler for > this (i.e. don't enable either the fair or the capacity scheduler). > Or am I missing something about your question? You are right, given my silly example :-). In reality, I require more than 5 different priorities. I think I might be able to use the CapacityScheduler, setting up one pool (highprio) that is guaranteed 100% of the cluster, and another one (lowprio) that is guaranteed 0% of the cluster. That will, if I understand correctly, give me the equivalent of 10 different priorities. Have not yet tried that, though. Regards, \EF > > *) I have four scheduling pools: highprio-daily, highprio-monthly, > > lowprio-daily, lowprio-monthly. > > > > *) Tasks for jobs that are put in highprio-daily always get priority > > before tasks in highprio-monthly. Highprio-monthly always get > > priority before lowprio-daily, and lowprio-daily always get > > priority before lowprio-monthly? > > > > If there are several jobs in the same pool, run them in order of > > submission. A job should finish as quickly as possible, so if the > > currently most highly prioritized job needs all task slots, it > > should get them. > > > > Thanks! > > \EF > > -- > > Erik Forsberg <[email protected]> > > Developer, Opera Software - http://www.opera.com/ > -- Erik Forsberg <[email protected]> Developer, Opera Software - http://www.opera.com/
