Hi Folks, I am running Nutch 1.7 on Hadoop 2.2 and in the Hadoop logs for FetcherThread, I see the following statements , which tells me that the HttpCleints are being created per URL, is this correct assumption? Also after a few fetches I also notice that the Hadoop job throws a OOM error , please advise.
2013-12-15 23:47:31,921 INFO [FetcherThread] org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false 2013-12-15 23:47:31,931 INFO [FetcherThread] org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false 2013-12-15 23:47:31,932 INFO [FetcherThread] org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false 2013-12-15 23:47:32,034 INFO [FetcherThread] org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false 2013-12-15 23:47:32,034 INFO [FetcherThread] org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false 2013-12-15 23:47:32,040 INFO [FetcherThread] org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false 2013-12-15 23:47:32,187 INFO [FetcherThread] org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false 2013-12-15 23:47:32,214 INFO [FetcherThread] org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false 2013-12-15 23:47:32,250 INFO [FetcherThread] org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false 2013-12-15 23:47:32,264 INFO [FetcherThread] org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
