I am just curious how long it typically takes to crawl a site? I am crawling server side which I realize is big. I also changed the urlfilter file to accept urls with ? so I could grab all the stuff from the forums and am doing a depth of 10 but I started crawling around 9 last night and now 13 hours later, it is still going. The crawl directory is up to about 627 meg. I can't imagine how long it would take if I tried to crawl the web.
Thanks!
