Setting up a crawler for a country.

Hello, Is there anyone that can implement a country crawler? I estimate around 40m documents. Please send me info about your prev work and how much time it would take to setup and money :-) Regards Asgeir Halldorsson

MergeSegs remove dubs hangs

Hi, I have been crawling a little. And I was wondering if there anyone else is having problems with Mergeing segments that remove Segs hags if you have over 500 pages? The data is about 70 gigs (pages + db). Regards Asgeir Halldorsson