> Does Solr have a project to track basic performance metrics from version to version, similar to Rally (the macro-benchmarking framework for Elasticsearch) <https://www.alibabacloud.com/blog/esrally-official-stress-testing-tool-for-elasticsearch_597102> ?
There's an ongoing effort to have periodic benchmarks. This type of testing would be ideal for repeating periodically, hence I asked. JIRA: https://issues.apache.org/jira/browse/SOLR-16525 Current link: http://mostly.cool/graph.html (click branch_9x on left). On Tue, Feb 28, 2023 at 6:29 AM Fikavec F <fika...@yandex.ru> wrote: > Thank you for helping me figure out the performance of "Response Writers". > If you wish, you can close the topic or move it to issues under the title > performance of non-binary types of "Response Writers" (if in your opinion > the difference of 4-8 times between wt=javabin and wt=json is not normal). > Maybe it is worth adding to the documentation a mention of the order of the > difference in the speeds of "Response Writers" in /select and /export > handlers? I would really like wt=json to work as fast as possible, as it is > popular and compatible, moreover, Solr itself works much faster than > wt=json. > > I have read and tested the specified wt=smile format (it is slower than > java bin, but also significantly faster than wt=json). Does the JDBC driver > (when working from python) offered in Solr 9.1.1 work internally using > javabin for client-server interaction (that is, data is received on it at > javabin speeds or as in wt=json)? > > As I wrote earlier - jetty is not a bottleneck in this issue and in nginx > activated { io threads; sendfile on; } as in the article on configuring > nginx <https://www.nginx.com/blog/thread-pools-boost-performance-9x/>. I > haven't thoroughly tested jetty specifically, just quickly checked that > jetty is able to transfer data (files) faster than when they are fetched > from Solr. I saw that jetty can work to transmit data at very high speeds > of the order of 6-9 gigabytes, and this was enough for me, I did not think > to compare and demand from it the speed of work as nginx optimized for file > transfer. > > Michael Gibney - without using sorting in my tests on Solr 9.1.1, fetching > looks a bit faster, but not so much that it is significant: > > - 409 Mb/s sort=id asc wt=javabin > - 411 Mb/s without sort wt=javabin > - 1.40 Gb/s sort=id asc wt=javabin > - 1.48 Gb/s without sort wt=javabin > - 2.38 Gb/s sort=id asc wt=smile > - 2.59 Gb/s without sort wt=smile > > All measurement results: > > <<<<----- SOLR 9.1.1 tests 28.02.2023 field with stored=true > docValues=false ------>>>>> > !!![sort=id asc] > /select HANDLER wt=json sort=id asc > wget --report-speed=bits --server-response -O /dev/null > --header='Accept-Encoding: ' --post-data > 'indent=false&wt=json&q=*%3A*&rows=1000000&sort=id%20asc' > http://192.168.220.135:8983/solr/test_collection/select > 10.03G in 3m 30s > 2023-02-27 21:55:53 (409 Mb/s) - ‘/dev/null’ saved [10772687921] > > /select HANDLER wt=javabin sort=id asc > wget --report-speed=bits --server-response -O /dev/null > --header='Accept-Encoding: ' --post-data > 'indent=false&wt=javabin&q=*%3A*&rows=1000000&sort=id%20asc' > http://192.168.220.135:8983/solr/test_collection/select > 10.01G in 61s > 2023-02-27 22:01:32 (1.40 Gb/s) - ‘/dev/null’ saved [10749142324] > > /select HANDLER wt=smile sort=id asc > wget --report-speed=bits --server-response -O /dev/null > --header='Accept-Encoding: ' --post-data > 'indent=false&wt=smile&q=*%3A*&rows=1000000&sort=id%20asc' > http://192.168.220.135:8983/solr/test_collection/select > 10.01G in 36s > 2023-02-27 22:31:40 (2.38 Gb/s) - ‘/dev/null’ saved [10748488299] > > !!![without sort] > /select HANDLER wt=json without sort > wget --report-speed=bits --server-response -O /dev/null > --header='Accept-Encoding: ' --post-data > 'indent=false&wt=json&q=*%3A*&rows=1000000' > http://192.168.220.135:8983/solr/test_collection/select > 10.03G in 3m 30s > 2023-02-27 22:06:31 (411 Mb/s) - ‘/dev/null’ saved [10772687903] > > /select HANDLER wt=javabin without sort > wget --report-speed=bits --server-response -O /dev/null > --header='Accept-Encoding: ' --post-data > 'indent=false&wt=javabin&q=*%3A*&rows=1000000' > http://192.168.220.135:8983/solr/test_collection/select > 10.01G in 58s > 2023-02-27 22:13:36 (1.48 Gb/s) - ‘/dev/null’ saved [10749142310] > > /select HANDLER wt=smile without sort > wget --report-speed=bits --server-response -O /dev/null > --header='Accept-Encoding: ' --post-data > 'indent=false&wt=smile&q=*%3A*&rows=1000000' > http://192.168.220.135:8983/solr/test_collection/select > 10.01G in 33s > 2023-02-27 22:29:38 (2.59 Gb/s) - ‘/dev/null’ saved [10748488286] > > > Ishan Chattopadhyaya - yes, regarding my tests, it looks like javabean in > Solr 9.1.1 significantly degraded in speed relative to version 8.11.2. > Since I was taking measurements in a 10 Gigabit network infrastructure > (emulated on Vms and real) with special optimization of the OS for it > (enable Jumbo Frames mtu 9000; tcp_window_scaling; 128MB tcp_wmem and > tcp_rmem) and disabled gzip, then the difference in performance could > become noticeable, which under normal conditions was not noticeable at all > due to the impossibility of achieving such network speeds with standard > settings. Therefore, my aspirations and measurements at speeds over gigabit > are not yet a mass need and phenomenon and may not be worth close > attention. The most interesting thing is that in today's measurement for > Michael Gibney in Solr 9.1.1 (presented above) javabin was slower than > wt=smile, but in version 8.11.2 wt=smile was fast, but noticeably slower > than javabin. If necessary, I will provide you with everything you may need > to reproduce. Performance is a very complicated matter, it is affected by a > lot of things, it is possible that the slowdown occurred due to the > peculiarities of my test data or test queries. Does Solr have a project to > track basic performance metrics from version to version, similar to Rally > (the macro-benchmarking framework for Elasticsearch) > <https://www.alibabacloud.com/blog/esrally-official-stress-testing-tool-for-elasticsearch_597102> > ? > > Best Regards, >
