Hi, Thanks for the reply. I have list? of those domains only. I am not sure how many pages they have. Is a DSL? connection sufficient to run nutch in my case. Did you run nutch for all of your pages at once or separately for a given subset of them. Btw, yesterday I tried to use merge shell script that we have on wiki. It gave a lot of errors. I run it on cygwin though.
Thanks. A. -----Original Message----- From: John Martyniak <[email protected]> To: [email protected] Sent: Tue, 3 Mar 2009 1:44 pm Subject: Re: what is needed to index for about 10000 domains I think that in order to answer that questions, it is necessary to know how many total pages are being indexed.? ? I currently have ~3.5 million pages indexed, and the segment directories are around 45GB, The response time is relatively fast.? ? In the test site it is running on a dual processor Dell 1850 with 3GB of RAM.? ? -John? ? On Mar 3, 2009, at 3:44 PM, [email protected] wrote:? ? > Hello,? >? > I use nutch-0.9 and need to index about 10000? domains.? I want to > know? > minimum requirements to hardware and memory.? >? > Thanks in advance.? > Alex.? ?
