This question may be better answered on the Elasticsearch forum but I thought I would give the GL list a try first. I recently added two additional nodes to a working cluster and would like some help/ideas on tuning for optimized performance and growth. My environment has 4 data nodes each spec'd out with 4 vCPU's, 12GB of Ram (ES HEAP is at 6GB), 250GB of storage (207GB on /var) running CentOS v6.7. Graylog is at v1.1.6, ES at v1.6.2 and openjdk 1.8. I am also using the stock settings for 20 indices with 20 Million records each. I have set 4 shards with one replica. The master node runs ES, GL, and GL web using the same specs, except instead of 250GB of storage, it only has 120GB. All nodes are thick provisioned VMDK's on a VMware cluster. Right now with our current sending rate, I see indices rotate about every 4-12 hours and generally shards have a size between 1.5GB's to 2GB's. The total used storage on the data nodes is ~73GB used with ~124GB available.
Okay, so finally to my question. I would like to increase either the number of indices or increase the number of records per index. Is one method preferred over the other? If the records count increases from 20 Million to 30 Million, would that increase/decrease index/search performance or should the index limit be set to 30 indices. Basically, which method would allow for increased historical data retention with the least overhead if that makes sense. Regards, Brandon -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/e70dce5b-450a-4822-b75a-642e35b14e6f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
