I benchmark Elasticsearch and my application together by replaying searches from my old search system against the one I'm deploying. I've benchmarked indexing in the past just by jamming data in really fast. My experience has been that if I find Elasticsearch is slow it is because I need to tweek some setting or change the instructions I send to it.
Example 1: I sped up rebuilding my index from scratch by 2.5x by turning off replicas while rebuilding the index and setting the refresh interval to -1 during the process. Adding the replicas later is much faster and causes much less load on the servers. Example 2: The plain highlighter is very slow at fuzzy queries so I don't use it for them. Example 3: The phrase suggester was slow for large queries if I set the maximum number of errors to correct to a big number (5). When I set it to a small number (2) it was much faster. Stuff like that. Nik On Fri, Apr 11, 2014 at 8:54 AM, Jianjun Hu <[email protected]> wrote: > Hi Rafał, > > Can't the JMeter be used to test the indexing performance? > > Can I adding replicas and re-sharding on my virtual machine? The OS of my > PC is Windows 7. I use VirtualBox Debian for testing ElasticSearch. > > Which tool is suitable for monitoring the cluster, OS and the garbage? > > Regards, > > Jianjun > > > On Thursday, April 10, 2014 10:57:42 AM UTC-4, Rafał Kuć wrote: > >> Hello! >> >> Depends on what you want to test. If you want to test indexing - start >> from scratch, see how many documents you can index with the clean >> Elasticsearch cluster, when you start to see slowdowns. Try seeing what the >> bottleneck is if any. In general, such tests depend on how much data you >> have and how much indexing you are planning to do. >> >> If you want to test querying performance - start with some setup that you >> assume should be OK. Index your data and start testing, for example by >> sending production like queries with JMeter. That will give you information >> on how the cluster behaves in such setup. Then you can change the >> deployment and tune. For example adding replicas, re-sharding and having >> more/less shards, depending on the data and its volume. >> >> During all tests, you should be monitoring your cluster. You can use >> anything available, although for performance testing anything will be good, >> even the command line tools that will help you understand what is happening >> on the operating system level, how garbage collector is performing. >> >> >> >> >> >> *-- Regards, Rafał Kuć Performance Monitoring * Log Analytics * Search >> Analytics Solr & Elasticsearch Support * *http://sematext.com/ >> >> >> >> Could you please give me some more detailed suggestions on how to test >> the elasticsearch? thanks. >> >> On Thursday, April 10, 2014 10:13:49 AM UTC-4, Mark Walkom wrote: >> The best way to know is to test it yourself. >> It's very dependant on your hardware, your settings and the data that you >> are indexing. >> >> Regards, >> Mark Walkom >> >> Infrastructure Engineer >> Campaign Monitor >> email: [email protected] >> web: www.campaignmonitor.com >> >> >> On 10 April 2014 23:49, Leslie Hawthorn <[email protected]> >> wrote: >> Hi Jianjun, >> >> Thank you for the additional details. I think we will need some more >> information but I will let someone who is better skilled at benchmarking >> Elasticsearch ask you about it. >> >> Welcome to the mailing list! We are happy to have you here and to help. >> >> Cheers, >> LH >> >> On Thu, Apr 10, 2014 at 3:42 PM, Jianjun Hu <[email protected]> wrote: >> Hi Leslie, >> >> Thanks for your reply. >> >> We have some data in MySQL database and some PDF documents. We want to >> index them and let our user to use. So we decide to use the ElasticSearch. >> Before using the ElsaticSearch, we want to know the performance of the >> ElasticSearch, so we want to do benchmark of the ElasticSearch. However, we >> don't know how to do. I'm not sure if I express my question clearly. >> >> Best, >> Jianjun >> >> >> On Thursday, April 10, 2014 9:11:37 AM UTC-4, Leslie Hawthorn wrote: >> Hi Jianjun, >> >> >> >> On Thu, Apr 10, 2014 at 3:02 PM, Jianjun Hu <[email protected]> wrote: >> Hi all, >> >> How to do benchmark of the ElasticSearch? Thanks! >> >> >> The answer to this question is "it depends." It would be very helpful to >> describe a bit more about what you are working on so people can give a >> better answer to you. :) >> >> Best, >> LH >> >> -- >> Leslie Hawthorn >> Community Manager >> http://elasticsearch.com >> >> Other Places to Find Me: >> Freenode: lh >> Twitter: @lhawthorn >> Skype: mebelh >> Voice: +31 20 794 7300 >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit https://groups.google.com/d/ >> msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e% >> 40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> >> >> >> -- >> Leslie Hawthorn >> Community Manager >> http://elasticsearch.com >> >> Other Places to Find Me: >> Freenode: lh >> Twitter: @lhawthorn >> Skype: mebelh >> Voice: +31 20 794 7300 >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit https://groups.google.com/d/ >> msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB% >> 2Bnmh%2B2cveKoq_BVQ%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> . >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/d8f2dd67- >> c113-4965-8f4c-7002c2053a1d%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/d8f2dd67-c113-4965-8f4c-7002c2053a1d%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/91d41ad8-b322-46a3-b027-7efe2de7d60e%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/91d41ad8-b322-46a3-b027-7efe2de7d60e%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3FX_OjK%2Bwn%3DQ9fn%2BvpBR4bAg_QGhGf%2BVjfzyEmfJ3EaA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
