Re: [SolrJ Client] Error calling add: connection is still allocated
On 9/21/2018 10:31 AM, Christopher Schultz wrote: For those interested, it looks like I was naïvely using BasicHttpClientConnectionManager, which is totally inappropriate in a multi-user threaded environment. I switched to PooledHttpClientConnectionManager and that seems to be working much better, now. :) Why not let SolrJ just create the HttpClient itself? Older versions of SolrJ would naively create the HttpClient with its defaults, only allowing two threads to run at once ... but newer versions of SolrJ create the HttpClient with better defaults, and now it can handle many threads without a custom HttpClient. I do not know which version made that change, but I think it was late in 6.x or early in 7.x. Thanks, Shawn
Re: [SolrJ Client] Error calling add: connection is still allocated
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 All, On 9/18/18 11:10, Christopher Schultz wrote: > All, > > Our single-instance Solr server is just getting its first taste of > production load, and I'm seeing this periodically: > > java.lang.IllegalStateException: Connection is still allocated > > The stack trace shows it's coming from HTTP Client as called from > within Solr. > > We are using SolrJ 7.2.1 and Solr (server) 7.4.0. > > Our code looks something like this: > > private HashMap CLIENT_REGISTRY = new > HashMap(); > > synchronized HttpSolrClient getSolrClient(String url) throws > ServiceException, SolrServerException, IOException, > GeneralSecurityException { HttpSolrClient solrClient = > CLIENT_REGISTRY.get(url); > > if(null == solrClient) { log.info("Creating new HttpSolrClient > connected to " + url); > > solrClient = new HttpSolrClient.Builder(url) > .withHttpClient(getHttpClient()) .build(); > > solrClient.ping(); > > CLIENT_REGISTRY.put(url, solrClient); } > > return solrClient; } > > > [here's the code that uses the above] > > SolrClient solr = getSolrRegistry().getSolrClient(url); > > SolrInputDocument doc = new SolrInputDocument(); > > // Add stuff to the document > > solr.add(doc); solr.commit(); > > That's it. > > Other than not really needing the "commit" at the end, is there > anything wrong with how we are using SolrJ client? Are instances > of SolrJClient not thread-safe? My assumption was that they were > threadsafe and that HTTP Client would manage the connection pool > under the covers. > > Here is the full stack trace: > > com.chadis.api.business.RegistrationProcessor- Error processing > registration request java.lang.IllegalStateException: Connection is > still allocated at > org.apache.http.util.Asserts.check(Asserts.java:34) at > org.apache.http.impl.conn.BasicHttpClientConnectionManager.getConnecti on > > (BasicHttpClientConnectionManager.java:251) > at > org.apache.http.impl.conn.BasicHttpClientConnectionManager$1.get(Basic Ht > > tpClientConnectionManager.java:202) > at > org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.j av > > a:191) > at > org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java: 18 > > 5) > at > org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) > > at > org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java: 11 > > 1) > at > org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpC li > > ent.java:185) > at > org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpC li > > ent.java:83) > at > org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpC li > > ent.java:56) > at > org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSol rC > > lient.java:542) > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClien t. > > java:255) > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClien t. > > java:244) > at > org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194) > > at > org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173) > at > org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138) > at > org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152) at > [my code, calling SolrClient.add()] > > Any ideas? > > Thanks, -chris > For those interested, it looks like I was naïvely using BasicHttpClientConnectionManager, which is totally inappropriate in a multi-user threaded environment. I switched to PooledHttpClientConnectionManager and that seems to be working much better, now. :) - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlulHN0ACgkQHPApP6U8 pFiWnRAAxIprILEjjF6rhwVZfmIFjLo8G5QNqMLZZ/4lIPEIDE8ojnCT8oiFpOru LakIn60DkQvgIioihvNtILjO7YJUPr0Pmoq+1feCQSSRtIFwBRoyvYahzUfx0v55 rWMRcJwWo/Vr1YsyQpH33O80F07himXxqmpiQeaQd+t+d9WYOpmBn8ENuG8QEd9g fc6yELLpJSpC6DFslCjRtAhMVNt3thdpbmYBwuKtoxHV8tuenXoxm/QQRLzJia/J AWsNB9boYNPF1T8rGt+eft4wej71t8ac00jzj+ylkQjPpPdexp+NSEGDRCfYoz6I bEVIVEy39f1SoyAlBnrS1QJqas9FwzMPd2tNv3y5fFCbYnKnHh50YaLgv1JAUali UQVDtlKGwPOrbbB2SBJiX3dK263RCQSSP9eJIDvyrGzRyRAgE9fzsVmvpokicMzx ZFiCZuIPPvmmGDvXBQ+lmtBvbav6ajsU3XyGEu+aawo6Lo7MgbdcLCPj839GR5Yd tDxMM2O8Wpkr4FRo7hbMlKJb5KoWJNtHjs5QQNFYUFmYwSXnU9OwH7B3fCpPVC2t OfBT5EKb8L1TWPog3zxFzrY5MQgJ2wSfBBphh2zeiFUSSLzb6T6F+ryv3rAzRO1U 6u6pfdf8AZ22gonPXs/mM4HbsL8dpP1Oyb6poHlaprxggKP7XqQ= =sHv2 -END PGP SIGNATURE-
[SolrJ Client] Error calling add: connection is still allocated
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 All, Our single-instance Solr server is just getting its first taste of production load, and I'm seeing this periodically: java.lang.IllegalStateException: Connection is still allocated The stack trace shows it's coming from HTTP Client as called from within Solr. We are using SolrJ 7.2.1 and Solr (server) 7.4.0. Our code looks something like this: private HashMap CLIENT_REGISTRY = new HashMap(); synchronized HttpSolrClient getSolrClient(String url) throws ServiceException, SolrServerException, IOException, GeneralSecurityException { HttpSolrClient solrClient = CLIENT_REGISTRY.get(url); if(null == solrClient) { log.info("Creating new HttpSolrClient connected to " + url); solrClient = new HttpSolrClient.Builder(url) .withHttpClient(getHttpClient()) .build(); solrClient.ping(); CLIENT_REGISTRY.put(url, solrClient); } return solrClient; } [here's the code that uses the above] SolrClient solr = getSolrRegistry().getSolrClient(url); SolrInputDocument doc = new SolrInputDocument(); // Add stuff to the document solr.add(doc); solr.commit(); That's it. Other than not really needing the "commit" at the end, is there anything wrong with how we are using SolrJ client? Are instances of SolrJClient not thread-safe? My assumption was that they were threadsafe and that HTTP Client would manage the connection pool under the covers. Here is the full stack trace: com.chadis.api.business.RegistrationProcessor- Error processing registration request java.lang.IllegalStateException: Connection is still allocated at org.apache.http.util.Asserts.check(Asserts.java:34) at org.apache.http.impl.conn.BasicHttpClientConnectionManager.getConnection (BasicHttpClientConnectionManager.java:251) at org.apache.http.impl.conn.BasicHttpClientConnectionManager$1.get(BasicHt tpClientConnectionManager.java:202) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.jav a:191) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:18 5) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:11 1) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpCli ent.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpCli ent.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpCli ent.java:56) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrC lient.java:542) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient. java:255) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient. java:244) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194) at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173) at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138) at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152) at [my code, calling SolrClient.add()] Any ideas? Thanks, - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAluhFVQACgkQHPApP6U8 pFhGeRAAgrg2GAmwhS9J/RBQC19SnebhevncmgMAF6nHhKegnXr8uv2fGvvySg53 BHCW0N3dtt9ZhI1VB7C9aBO65o/esW5rHi3/sIiY5QRfNIl39ajL8y98RWHJQEeA mhjoqNdqW/GopA3YaiCmf1YJZ0FsZV7iK04KboD5DRwhsqoa8XVDa44RYfdU4iDP cleMkQYY2KDSID0gJ2pf/Qj1acwR/hI2Q9+6kxc11/bXKCrWYAmLawV+DH6ZHqLF HT/7bNNJ+zV0df0WEKzUDQ9wVzTKXkzvYP7ueINIiomyZN7Pv+pF58BaAiICdlUr aqQMulLcKRC7qmN/5XqBZG00hkbH82n80o5foveTlQlC9yltSTbXjwFqd+FfOH8Y kBU+mHWkrZr/Ic29LkgLLzX1tG+QoXAgoEAASHOockaTX5oj2vsyFYQ5nVddOMNj /w1AgdpNztP5DLr1HQ6JhA+3nLZX43GaDxs/nENIOI2Xe36kXfS/so9Cv7DaAjQ8 OkGdOLUksQaukFZ/3MUwbgan5tQYYp4zSmky4RGS7Nd0ePTgvk4pH1uD4NFJnHWK fsSydLT43tiOWltQkzzby6QcpSg9WrV+0zsnEPQSQHH+ubDbFt03aXS1/tjYAZTF r8ttwGFfMQLa58hfWwBKMWtyM8m6n9gVMivhp5oENa3uFdo76kQ= =+WJu -END PGP SIGNATURE-