Hey all, We're bumping up against a production problem I could use a hand with. We're experiencing steadily decreasing index speeds. We have 12 c3.4xl data nodes, and 1 c3.8xl master node (with 2 backups that are smaller). We're indexing 45 million documents into a single index. Single shard only, no replicas. As our number of documents grow, our indexing speed slows to a crawl. We've applied all the standard mlockall, ulimit, and ssd merge throttling tuning settings, so I feel our cluster is pretty good.
When I inspected the data, I've noticed our user is adding a new field on every document. When I view the pending tasks on our master, the task queue is always at least 300+ attempting to perform dynamic mapping. I've also checked segment merging, we never have more than 1 merge going on, and even then it lasts for a second or two, not long at all. This brings me to my question. When dynamic mapping is performed, is this on the master only? Obviously this would introduce a bottleneck, and explain our sudden performance drop. I'm at a loss to explain this issue. Any advice would be appreciated. Thanks, Todd -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0611317c-d3c1-4894-8fac-8ac4b36cbf15%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
