Hi, Just noticed Hadoop's new fair sharing job scheduler ( https://issues.apache.org/jira/browse/HADOOP-3746 ). It seems to be in 0.19, which I think Nutch is not on yet... but still:
- is this something that would benefit Nutch? The last time I used Nutch I remember having to be careful about mostly sequential job runs and having to pay close attention to number of max map/reduce tasks, etc. in order to maximize the cluster, and I wonder if the above would make that easier, less manual, or more efficient? Thanks, Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
