Hi all, iam using nutch 0.8., when iam doing entire crawling process i took approximate 281 seconds having 3 SLAVES then i have added 2 more SLAVES then also it took 261 seconds to finish the crawling process.
How can i improve this performance problem...please help me... my configuration hadoop-site.xml <property> <name>mapred.map.tasks</name> <value>100</value> <description>The default number of map tasks per job. Typically set to a prime several times greater than number of available hosts. Ignored when mapred.job.tracker is "local". </description> </property> <property> <name>mapred.reduce.tasks</name> <value>8</value> <description>The default number of reduce tasks per job. Typically set to a prime close to the number of available hosts. Ignored when mapred.job.tracker is "local". </description> </property> <property> <name>mapred.reduce.parallel.copies</name> <value>5</value> <description>The default number of parallel transfers run by reduce during the copy(shuffle) phase. </description> </property> Thanks & Regards Mohan Lal -- View this message in context: http://www.nabble.com/Performance-problem-in-nutch-0.8.1-tf2376490.html#a6621127 Sent from the Nutch - User mailing list archive at Nabble.com. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
