I suggest to use nutch 0.8 on several computers with DFS. But I'm worried about nutch's requirements to HDD free space.
For example, suppose I have 1) server with job tracker and namenode 2) 5 servers with task trackers and 20 Gb HDDs 3) 5 servers with datenode and 20 Gb HDDs also (DFS, the replication will be equal 1) There are some questions: 1) Is this HDD space enough to run task trackers? 2) How to calculate the approximate free HDD space needed for servers with task trackers, servers with with job trackers and name node? 3) Will I be able to increase the data storage space while increasing the number of servers with date node? Or will it not be enough to increase the number of date nodes? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
