This is fantastic. Are you able to submit a patch for this? If not then I will do so. Thanks again. Lewis
On Thu, Jul 10, 2014 at 5:22 AM, Konstantinos Mavrommatis < [email protected]> wrote: > Hi, > > Following the suggestion at > > > http://blogs.nuxeo.com/development/2013/02/using-httpclient-properly-avoid-closewait-tcp-connections/ > > > > I modified the code in > > src/main/java/org/apache/oodt/cas/filemgr/catalog/solr/SolrClient.java > > > > and added the line highlighted in red > (method.setRequestHeader(“Connection”,”close”). > > This seems to have resolved the problem at least until now ingestion of > few thousand files have not produced any stale connection. > > Please let me know your thoughts on this solution. > > Best, > > Konstantinos > > > > ====================================================== > > private String doHttp(HttpMethod method) throws Exception { > > > > StringBuilder response = new StringBuilder(); > > BufferedReader br = null; > > try { > > > > // send request > > HttpClient httpClient = new HttpClient(); > > // 10 July 2014. attempting to avoid problems with > CLOSE_WAIT connections Based on > > // > http://blogs.nuxeo.com/development/2013/02/using-httpclient-properly-avoid-closewait-tcp-connections/ > > method.setRequestHeader("Connection", "close"); > > int statusCode = httpClient.executeMethod(method); > > > > // read response > > if (statusCode != HttpStatus.SC_OK) { > > > > // still consume the response > > method.getResponseBodyAsString(); > > throw new CatalogException("HTTP method failed: " + > method.getStatusLine()); > > > > } else { > > > > // read the response body. > > br = new BufferedReader(new > InputStreamReader(method.getResponseBodyAsStream())); > > String readLine; > > while(((readLine = br.readLine()) != null)) { > > response.append(readLine); > > } > > > > ======================================================= > > > > Information about the system: > > OS is MacOS Mavericks, > > SOLR version : 4.6.1 , SOLR runs on a single server, no replication at all. > > Tomcat verision: 7.0.50 > > OODT version is 0.6 > > Java(TM) SE Runtime Environment (build 1.7.0_51-b13) > > > > > > > > > -----Original Message----- > > > From: Lewis John Mcgibbney [mailto:[email protected]] > > > Sent: Wednesday, July 09, 2014 4:57 AM > > > To: [email protected] > > > Subject: Re: File manager keeps connections to SOLR in CLOSE_WAIT state > > > for hours > > > > > > It's just occoured to me that everything in > > > org.apache.oodt.cas.filemgr.catalog.solr.SolrCatalog (well, > > > specifically > > > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient) is done through > > > HTTPClient so all of the above may not be relevant. > > > Can you please scope all the same? > > > Thanks > > > Lewis > > > > > > > > > On Tue, Jul 8, 2014 at 10:54 PM, Lewis John Mcgibbney < > > > [email protected]<mailto:[email protected]>> wrote: > > > > > > > Hi Konstantinos, > > > > OK, I was ablel to scope this one and I have a few questions for you. > > > > 1) Which version of Solr are you using? Is it 3.5, 3.6, 4.0-ALPHA? > > > If > > > > so please scope this issue [0], the solution would be to upgrade if > > > > you are not too long ahead with ingestion as fixes in Solr are worth > > > > having based on recent release cycles. > > > > 2) How many cores do you have on Solr server? Also what kind of setUp > > > > do you have? Replication at all? > > > > In recent versions of Solr 4.X SolrJ clients should now call > > > > shutdown() on their SolrServer object to let it know they don't want > > > > to re-use any existing connections anymore, and when Solr internally > > > > uses SolrJ to talk to other nodes in SolrCloud it should be doing > > > this > > > > (as of 4.0-ALPHA)so this is why I ask. > > > > Lewis > > > > > > > > [0] https://issues.apache.org/jira/browse/SOLR-3280 > > > > > > > > > > > > On Tue, Jul 8, 2014 at 7:14 AM, Konstantinos Mavrommatis < > > > > [email protected]<mailto:[email protected]>> wrote: > > > > > > > >> Hi, > > > >> I have setup OODT filemanager on port 9000, using SOLR as the > > > >> indexing service on port 8081. They are both setup on the same > > > >> computer, while crawler runs on a number of different compute nodes > > > >> spread across the local network and the cloud. > > > >> > > > >> When the crawler runs and ingests files I notice that there are > > > >> several connections that open to solr and remain in CLOSE_WAIT state > > > for hours. > > > >> any idea why this happens? Moving forward I am planning to use > > > >> several hundreds of crawler instances, each running on different > > > >> computer, that will create thousands of such connections and will > > > >> probably create problems to the system. > > > >> Thanks in advance for any help > > > >> Kostas > > > >> > > > >> $lsof -i :8081 > > > >> COMMAND PID USER FD TYPE DEVICE SIZE/OFF > > > NODE > > > >> NAME > > > >> java 92065 kmavrommatis 75u IPv6 0x392c3fa3b63b29cf 0t0 > > > TCP > > > >> localhost:49205->localhost:sunproxyadmin (CLOSE_WAIT) > > > >> java 92065 kmavrommatis 76u IPv6 0x392c3fa3b6dcbbcf 0t0 > > > TCP > > > >> localhost:49206->localhost:sunproxyadmin (CLOSE_WAIT) > > > >> java 92065 kmavrommatis 77u IPv6 0x392c3fa39fd12e0f 0t0 > > > TCP > > > >> localhost:49207->localhost:sunproxyadmin (CLOSE_WAIT) > > > >> java 92065 kmavrommatis 78u IPv6 0x392c3fa39fdcdbcf 0t0 > > > TCP > > > >> localhost:49208->localhost:sunproxyadmin (CLOSE_WAIT) > > > >> java 92065 kmavrommatis 79u IPv6 0x392c3fa3b62cde0f 0t0 > > > TCP > > > >> localhost:49209->localhost:sunproxyadmin (CLOSE_WAIT) > > > >> java 92065 kmavrommatis 80u IPv6 0x392c3fa39fa2714f 0t0 > > > TCP > > > >> localhost:49210->localhost:sunproxyadmin (CLOSE_WAIT) > > > >> java 92065 kmavrommatis 81u IPv6 0x392c3fa3b6c32acf 0t0 > > > TCP > > > >> localhost:49211->localhost:sunproxyadmin (CLOSE_WAIT) > > > >> java 92065 kmavrommatis 82u IPv6 0x392c3fa3b6aa714f 0t0 > > > TCP > > > >> localhost:49212->localhost:sunproxyadmin (CLOSE_WAIT) > > > >> > > > >> > > > >> process 92065 is: > > > >> /usr/bin/java -Djava.ext.dirs=../lib > > > >> -Djava.util.logging.config.file=../etc/logging.properties > > > >> -Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties > > > >> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager --portNum 9000 > > > >> > > > >> ********************************************************* > > > >> THIS ELECTRONIC MAIL MESSAGE AND ANY ATTACHMENT IS CONFIDENTIAL AND > > > >> MAY CONTAIN LEGALLY PRIVILEGED INFORMATION INTENDED ONLY FOR THE USE > > > >> OF THE INDIVIDUAL OR INDIVIDUALS NAMED ABOVE. > > > >> If the reader is not the intended recipient, or the employee or > > > agent > > > >> responsible to deliver it to the intended recipient, you are hereby > > > >> notified that any dissemination, distribution or copying of this > > > >> communication is strictly prohibited. If you have received this > > > >> communication in error, please reply to the sender to notify us of > > > >> the error and delete the original message. Thank You. > > > >> > > > > > > > > > > > > > > > > -- > > > > *Lewis* > > > > > > > > > > > > > > > > -- > > > *Lewis* > > ********************************************************* > THIS ELECTRONIC MAIL MESSAGE AND ANY ATTACHMENT IS > CONFIDENTIAL AND MAY CONTAIN LEGALLY PRIVILEGED > INFORMATION INTENDED ONLY FOR THE USE OF THE INDIVIDUAL > OR INDIVIDUALS NAMED ABOVE. > If the reader is not the intended recipient, or the > employee or agent responsible to deliver it to the > intended recipient, you are hereby notified that any > dissemination, distribution or copying of this > communication is strictly prohibited. If you have > received this communication in error, please reply to the > sender to notify us of the error and delete the original > message. Thank You. > -- *Lewis*
