Dear Wiki user, You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.
The following page has been changed by NigelDaley: http://wiki.apache.org/lucene-hadoop/FAQ The comment on the change is: added 900 node config values ------------------------------------------------------------------------------ == 3. How well does Hadoop scale? == - Hadoop has been demonstrated on clusters of up to 2000 nodes. Sort performance is good (sorting 9TB of data on 900 nodes takes around 4 hours) and [attachment:sort900-20070607.png improving]. + Hadoop has been demonstrated on clusters of up to 2000 nodes. Sort performance on 900 nodes is good (sorting 9TB of data on 900 nodes takes around 4 hours) and [attachment:sort900-20070607.png improving] using these non-default configuration values: + + * `dfs.block.size = 134217728` + * `dfs.namenode.handler.count = 40` + * `mapred.speculative.execution = true` + * `mapred.reduce.parallel.copies = 20` + * `mapred.tasktracker.tasks.maximum = 2` + * `mapred.child.java.opts = -Xmx512m` + * `fs.inmemory.size.mb = 200` + * `io.sort.factor = 100` + * `io.sort.mb = 200` + * `io.file.buffer.size = 131072` == 4. Do I have to write my application in Java? ==
