Hi, my teammate is also suffering from this situation now and I encountered this situation last night. But I am able to crawl now almost without doing anything. The reason I may guess is that your crawler is blocked by the website because not being polite. At least I believe that's the reason why I got the same *Could not initialize class org.apache.http.impl.conn.* last night. I don't how to solve it, though..... Fortunate enough I think I am unbanned now, I guess? Hope it helps......
On Thu, Feb 19, 2015 at 9:44 PM, Nagarjun Pola <[email protected]> wrote: > I get the following error when tried with selenium. Firefox pops up couple > of times but fetches nothing. > > Can anyone help me on this issue? > > *-activeThreads=50, spinWaiting=49, fetchQueues.totalSize=1, > fetchQueues.getQueueCount=1* > > ** queue: http://gcmd.gsfc.nasa.gov <http://gcmd.gsfc.nasa.gov>* > > * maxThreads = 1* > > * inProgress = 1* > > * crawlDelay = 5000* > > * minCrawlDelay = 0* > > * nextFetchTime = 1424410799976* > > * now = 1424410803146* > > * 0. http://gcmd.gsfc.nasa.gov/ <http://gcmd.gsfc.nasa.gov/>* > > *fetch of > http://gcmd.gsfc.nasa.gov/KeywordSearch/Home.do?Portal=amd&MetadataType=0 > <http://gcmd.gsfc.nasa.gov/KeywordSearch/Home.do?Portal=amd&MetadataType=0> > failed with: java.lang.NoClassDefFoundError: Could not initialize class > org.apache.http.impl.conn.ManagedHttpClientConnectionFactory* > > *-activeThreads=50, spinWaiting=50, fetchQueues.totalSize=1, > fetchQueues.getQueueCount=1* > > ** queue: http://gcmd.gsfc.nasa.gov <http://gcmd.gsfc.nasa.gov>* > > * maxThreads = 1* > > * inProgress = 0* > > * crawlDelay = 5000* > > * minCrawlDelay = 0* > > * nextFetchTime = 1424410808305* > > * now = 1424410804147* >
