You will also need more than 1 terabyte to get to 100 million pages. A good rule of thumb is 2 gigs * replication factor for every 1 million pages.
Dennis Dan Morrill wrote: > Hi, > > I found that with a 3 meg DSL line I was averaging 8 pages per second with a > similar set up, to reach 100 million pages would take about 144 days. > > 100,000,000 / 8 pages per second / 60 seconds per minute / 60 minutes per > hour / 24 hours in a day. > > Just a FYI rule of thumb on a qwest DSL line with no metering. > > r/d > > -----Original Message----- > From: Bui Quang Hung [mailto:[EMAIL PROTECTED] > Sent: Wednesday, August 23, 2006 4:50 AM > To: [email protected] > Subject: How long to get 100 million page > > > > Hi, > I am planning to create an index of 100 million pages by using a back-end > machine which includes a single-processor box with 1 gigabyte of RAM, 1 > terabyte hard disk. Can you teach me that how long it will take? > Thank you in advance. > Regards, > B.Q. Hung > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
