Thanks Andrzej, but the thing is not working at my end. any how I commented it and have done some fetching.
-----Original Message----- From: Andrzej Bialecki [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 14, 2005 11:49 PM To: [email protected] Subject: Re: What exactly is blockAddr(url) doing Chetan Sahasrabudhe wrote: > Hi, > > I am still trying to get my nutch fetcher run through proxy. > I have made a small change in, > > Http.java under configureClient() method > > //client.getState().setCredentials(null, null, ntCreds); > client.getState().setProxyCredentials(null, null, ntCreds); > > I have explicitly called setProxyCredentials and it is working fine. Not I am > no more getting 407's. > > Once I get robots.txt file from server and run it through, isAllowed. > > The code fails at blockAddr(url) call. Me still trying to figure out what > this call is doing. This code makes sure that only a limited number of concurrent threads (usually 1) accesses any given host. When the code detects that the limit was hit, the current thread is put into a spinlock, waiting until there are fewer threads than the limit allows. -- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __________________________________ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com ------------------------------------------------------- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_idt77&alloc_id492&op=click _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
