When you add documents to Elasticsearch, this creates new files on disk that form what is called a segment. Having several segments is fine, but when you start having too many of them, search is going to be slower this is why Elasticsearch has a background process that takes care of merging these segments, so that the total number of segments remains low enough (usually in the order of ~50 per shard). However, running a background merge can take lots of resources on the server, especially I/O, and this might defeat the purpose of making search remain fast since search operations don't have much I/O capacity left. In order to prevent it from happening, merges are throttled[1], meaning that they can't write more than X bytes of data per second. If they try to, Elasticsearch will pause them for a while before they can keep on merging again.
The throttle_time reported by the stats API gives you the total number of time that merges have been paused in order to prevent them from stealing all the server I/O. [1] http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-store.html#store-throttling On Tue, Mar 4, 2014 at 8:39 AM, Isaac Hazan <[email protected]>wrote: > I created an index in a 4 nodes Elasticsearch cluster. I added about 3.5 M > documents using the java Elasticsearch API. > When asking for the stats i get a very high number in > throttle_time_in_millis as follows: > > { > "_shards": { > "total": 10, > "successful": 10, > "failed": 0 > }, > "_all": { > "primaries": { > "docs": { > "count": 3855540, > "deleted": 0 > }, > "store": { > "size_in_bytes": 1203074796, > "throttle_time_in_millis": 980255 > }, > "indexing": { > "index_total": 3855540, > "index_time_in_millis": 426300, > "index_current": 0, > "delete_total": 0, > "delete_time_in_millis": 0, > "delete_current": 0 > }, > > 1. What is the meaning of throttle_time_in_millis? > 2. What could be the reason for this to increase? > > Thx in advance > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/73633235-ae82-4cd2-a74a-3de6cff5cd47%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/73633235-ae82-4cd2-a74a-3de6cff5cd47%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/groups/opt_out. > -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j7RW1YhyZYBKxXQ183yjSTX-aMXipBR8DZatCmkFRQp4g%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
