Hi, I am running version 0.7.1 and am crawling different sites around the web with ? & allowed in the url's and at different depths
I can get the crawl working with fetcher.threads.fetch = 1 but for anything larger than this it hangs after a while of crawling and on no particular site. fetcher.threads.per.host = 1 and I have tried different variations of these settings with the same problem occuring in different lpaces If anyone is experiencing this same problem or knows the fix for this issue I would greatly appreciate any help!!! cheers steve
