Adrien, I get the feeling that you're a pretty heavy contributor to the aggregation module. In your experience, would a shard per cpu core strategy be an effective performance solution in a pure aggregation use case? If this could proportionally reduce the aggregation time, would a node local reduce (in which all shard aggregations on a given node are reduced prior to being sent to the client node) be a good follow on strategy for further enhancement?
On Wednesday, January 14, 2015 at 10:56:03 AM UTC-5, Adrien Grand wrote: > > > > On Wed, Jan 14, 2015 at 4:16 PM, Elliott Bradshaw <[email protected] > <javascript:>> wrote: > >> Just out of curiosity, are aggregations on multiple shards on a single >> node executed serially or in parallel? In my experience, it appears that >> they're executed serially (my CPU usage did not change when going from 1 >> shard to 2 shards per node, but I didn't test this extensively). I'm >> interested in maximizing the parallelism of an aggregation without creating >> a massive number of nodes. >> >> > Requests are processed serially per shard, but several shards can be > processed at the same time. So if you have an index that consists of N > primaries, this would run on N processors of your cluster in parallel. > > > -- > Adrien Grand > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/03f2db56-8c37-4f10-9383-7e8591aed865%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
