Thanks Andrzej, but the thing is not working at my end. any how I commented it and have done some fetching.
-----Original Message----- From: Andrzej Bialecki [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 14, 2005 11:49 PM To: [email protected] Subject: Re: What exactly is blockAddr(url) doing Chetan Sahasrabudhe wrote: > Hi, > > I am still trying to get my nutch fetcher run through proxy. > I have made a small change in, > > Http.java under configureClient() method > > //client.getState().setCredentials(null, null, ntCreds); > client.getState().setProxyCredentials(null, null, ntCreds); > > I have explicitly called setProxyCredentials and it is working fine. Not I am > no more getting 407's. > > Once I get robots.txt file from server and run it through, isAllowed. > > The code fails at blockAddr(url) call. Me still trying to figure out what > this call is doing. This code makes sure that only a limited number of concurrent threads (usually 1) accesses any given host. When the code detects that the limit was hit, the current thread is put into a spinlock, waiting until there are fewer threads than the limit allows. -- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __________________________________ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com
