Dear Wiki user, You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.
The following page has been changed by DevarajDas: http://wiki.apache.org/lucene-hadoop/FAQ ------------------------------------------------------------------------------ * `dfs.block.size = 134217728` * `dfs.namenode.handler.count = 40` - * `mapred.speculative.execution = true` * `mapred.reduce.parallel.copies = 20` * `mapred.child.java.opts = -Xmx512m` * `fs.inmemory.size.mb = 200` @@ -45, +44 @@ * `io.sort.mb = 200` * `io.file.buffer.size = 131072` - Sort performances on 1400 nodes and 2000 nodes are pretty good too - sorting 14TB of data on a 1400-node cluster takes 2.2 hours; sorting 20TB on a 2000-node cluster takes 2.5 hours. + Sort performances on 1400 nodes and 2000 nodes are pretty good too - sorting 14TB of data on a 1400-node cluster takes 2.2 hours; sorting 20TB on a 2000-node cluster takes 2.5 hours. The updates to the above configuration being: + * `mapred.job.tracker.handler.count = 60` + * `mapred.reduce.parallel.copies = 50` + * `tasktracker.http.threads = 50` == 4. Do I have to write my application in Java? ==
