> Are the JVM parameters good enough? Non search-related http requests run smoothly as long as not all AJPs are blocked waiting for Solr-resonses 😉
-----Ursprüngliche Nachricht----- Von: Deepak Goel <[email protected]> Gesendet: Montag, 30. Mai 2022 13:16 An: [email protected] Betreff: Re: SolrJ connection handling, best practice(s)? Could you please share your Tomcat web server configuration? Are the JVM parameters good enough? Deepak "The greatness of a nation can be judged by the way its animals are treated - Mahatma Gandhi" +91 73500 12833 [email protected] Facebook: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.facebook.com%2Fdeicool&data=05%7C01%7CClemens.Wyss%40helbling.ch%7Ccaf45aa7c3294dbc7b3708da422dd127%7Cbcbab93b0f0d4747b4a4bc57e6753bb4%7C0%7C0%7C637895061793187418%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DMh0j1aYYos3wMw9Dcgfh46yHIU5HYjklrmX4TSFPt0%3D&reserved=0 LinkedIn: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fdeicool&data=05%7C01%7CClemens.Wyss%40helbling.ch%7Ccaf45aa7c3294dbc7b3708da422dd127%7Cbcbab93b0f0d4747b4a4bc57e6753bb4%7C0%7C0%7C637895061793187418%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=xoALkzrXnivcwVxZdBx3pTXCc%2BJShGEkeSSRvlj%2BU3M%3D&reserved=0 "Plant a Tree, Go Green" Make In India : https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.makeinindia.com%2Fhome&data=05%7C01%7CClemens.Wyss%40helbling.ch%7Ccaf45aa7c3294dbc7b3708da422dd127%7Cbcbab93b0f0d4747b4a4bc57e6753bb4%7C0%7C0%7C637895061793187418%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=xLgg5Q8i85NNCe7M2sC7Bd9y8C1MhuGubxt2jAaOg0U%3D&reserved=0 On Mon, May 30, 2022 at 4:41 PM Clemens WYSS (Helbling Technik) < [email protected]> wrote: > Ciao Vincenzo, > > Why do you think that this problem is caused by the wrong usage of > >the clients? > * during the "peak" the solr server itself was not really "under pressure" > * stressing the solr server with k6 showed that the Solr-Server(setup) > can handle the amount of expected concurrent requests > > So to me it's either within tomcat, i.e. the HttpClient "connection > handling" (e.g. MaxConnections, MaxConnectionsPerHost, ...), or the > "tomcat host server setup" (limit of outgoing connections, ephemeral > ports, ulimit?)? > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhc.a > pache.org%2Fhttpclient-legacy%2Fperformance.html&data=05%7C01%7CCl > emens.Wyss%40helbling.ch%7Ccaf45aa7c3294dbc7b3708da422dd127%7Cbcbab93b > 0f0d4747b4a4bc57e6753bb4%7C0%7C0%7C637895061793187418%7CUnknown%7CTWFp > bGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn > 0%3D%7C3000%7C%7C%7C&sdata=LAbOw8pr0D7Ut2kVm%2F%2FZMjXIrcQMS3toLZ5 > 4tTHNKcI%3D&reserved=0 Does HttpSolrClient make use of > MultiThreadedHttpConnectionManager? > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhc.a > pache.org%2Fhttpclient-legacy%2Fthreading.html&data=05%7C01%7CClem > ens.Wyss%40helbling.ch%7Ccaf45aa7c3294dbc7b3708da422dd127%7Cbcbab93b0f > 0d4747b4a4bc57e6753bb4%7C0%7C0%7C637895061793187418%7CUnknown%7CTWFpbG > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > 3D%7C3000%7C%7C%7C&sdata=iTNWeMzrY2jHzUS6ZXCvXBl9zanMTJ4jLpRXuDBEz > LY%3D&reserved=0 Other things HttpSolrClient may need to consider > > Thx > - Clemens > -----Ursprüngliche Nachricht----- > Von: Vincenzo D'Amore <[email protected]> > Gesendet: Montag, 30. Mai 2022 09:12 > An: [email protected] > Betreff: Re: SolrJ connection handling, best practice(s)? > > Hi Clemens, Solrj clients are thread-safe singletons, from my > experience the strategy you are using works well even under high load, > when calls to Solr take seconds to answer usually it happens because > of the number of the queries, the kind of queries (grouping or too > many facets) or even for garbage collection problems. > Why do you think that this problem is caused by the wrong usage of the > clients? > > On Mon, May 30, 2022 at 8:30 AM Clemens WYSS (Helbling Technik) < > [email protected]> wrote: > > > We have a non clustered single (8.x) Solr server setup and use SolrJ > > to connect our tomcat to the Solr server. Under high load (e.g. last > > black > > friday) we face a bottleneck due to our webapp-search-requests. > > Calls to solr take seconds, dangling http-connecitons, ... > > Firing the same ( or even way more ) search requests directly onto > > our Solr server showed, as expected, no problem. Therefore my > > implication is that SolrJ/HttpSolrClient/HttpClient is not setup as > > expected. > > Our search API is setup alike: > > > > private final Map<String, SolrClient> coreConnections; // one > > SolrClient per core ... > > > > // create a new SolrClient for a core if not existant > > HttpSolrClient.Builder builder = new HttpSolrClient.Builder( coreUrl > > ); solrClient = builder.build(); coreConnections.put( key, > > solrClient ); > > > > As you can see > > * we keep one SolrClient-connection per core in a map and make use > > of these connections > > * making use of HttpSolrClient ( Http2SolrClient is denoted > "experimental" > > and has cause some problem ) > > * no special tweaking of underneath HttpClient > > > > Any reason for this minimal setup not being able to stem many many > > concurrent requests? What is the proposed SolrJ based solr client > > setup (for a single Solr server)? > > > > Thx > > - Clemens > > > > > -- > Vincenzo D'Amore >
