We're using the Java API. I assume that it is using a binary representation of some kind that is more compact than JSON. I just mentioned JSON to illustrate the size of the response. I'll certainly try to monitor disk/network activity.
Nils-H On Friday, May 30, 2014 7:16:38 AM UTC+2, Otis Gospodnetic wrote: > > Massive JSON responses could indeed be a problem. I think you can easily > see if CPU, Disk, or Network are the bottleneck using really any monitoring > tool. Even dstat --cpu --mem --disk --net will give you an idea. :) > > Otis > -- > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > > > On Tuesday, May 27, 2014 6:44:45 AM UTC-4, [email protected] wrote: >> >> Thank you for your reply. >> >> Here are some observations from a couple of days testing: >> >> - Setting up routing manually reduced the aggregation time about 40%! >> - ... however, manual routing caused data to distribute unevenly. I >> assume we could have taken steps to improve the distribution, but we didn't >> investigate any further >> - Upgrading from 1.1.0 to 1.2.0 didn't seem to improve speed nor memory >> usage, although we didn't do any accurate measurements of RAM usage >> - Changing the compression value for percentiles did indeed have an effect >> - Increasing number of nodes from 3 to 5 didn't seem to improve >> performance >> >> Since adding additional nodes didn't seem to improve the performance, >> there seem to be a bottleneck somewhere. The result of the aggregation is >> very large (as a JSON-result, it results in about a million lines of text), >> so maybe data transfer or constructing the result can be a bottleneck? >> > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c626684e-a3f3-4188-9bce-368853138315%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
