On Tue, May 05, 2009 at 02:36:27PM -0700, Ken Krugler wrote: >> On Mon, May 04, 2009 at 04:30:15PM -0700, Ken Krugler wrote: >>> Hi all, >>> >>> In Http 3.1, the Nutch code base would configure timeouts using the >>> following snippet of code: >>> >>> MultiThreadedHttpConnectionManager connectionManager = >>> new MultiThreadedHttpConnectionManager(); >>> >>> HttpClient client = new HttpClient(connectionManager); >>> >> > HttpConnectionManagerParams params = connectionManager.getParams(); >> > params.setConnectionTimeout(timeout); >>> params.setSoTimeout(timeout); >>> >>> // executeMethod(HttpMethod) seems to ignore the connection timeout >>> on the connection manager. >>> // set it explicitly on the HttpClient. >>> >>> client.getParams().setConnectionManagerTimeout(timeout); >>> >>> What's the functional equivalent in 4.0? I'm assuming that: >>> >>> HttpParams params = new BasicHttpParams(); >>> ConnManagerParams.setTimeout(params, timeout); >>> >>> is equivalent to the 3.1 call to params.setConnectionTimeout(timeout). >>> But what about the setSoTimeout() call? >>> >> >> HTTP parameter: >> >> 'http.socket.timeout' sets the socket timeout >> 'http.connection.timeout' sets the connect timeout >> 'http.conn-manager.timeout' sets the connection request timeout >> >> Corresponding utility methods: >> >> HttpParams params = new BasicHttpParams(); >> HttpConnectionParams.setSoTimeout(params, 2000); >> HttpConnectionParams.setConnectionTimeout(params, 2000); >> ConnManagerParams.setTimeout(params, 2000); > > Good stuff, thanks. > > One question - why is the timeout for ConnManagerParams.setTimeout a > long, while the others are ints? Is this value expected to be really big > at times? >
No good reason. Heritage from 2.x days. Long as a data type for connection manager timeouts certainly does not make much sense. Oleg > [snip] > >> > But independent of the above, I'm interested in the best way to prevent >>> all cases of long timeouts, with 4.0. >>> >> >> I believe setting socket and connect timeouts to some reasonable value, say >> 30 >> seconds, should be sufficient. > > OK, thanks. > > I'll be doing a bigger crawl soon (> 1M pages) so I'll have more data on > problems encountered. > > -- Ken > -- > Ken Krugler > +1 530-210-6378 > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
