Question #202816 on Graphite changed: https://answers.launchpad.net/graphite/+question/202816
bhardy posted a new comment: Petri, I am doing some of the same testing now for local disks. Everyone does not have the liberty to move to SSDs. :-) Sorry if some of the info below is Dr. Obvious but I was hoping to be thorough. Hope it helps. Like you, my systems have plenty of CPUs and Memory but the crux is disk I/O. I believe a good starting point is knowing how many metrics you expect to receive each minute. With this in hand you can test and tune the three values below in carbon.conf. You can also run multiple carbon-cache daemons on one server possibly fed by a separate server solely running carbon-relay. MAX_CACHE_SIZE = 300000 MAX_UPDATES_PER_SECOND = 300 MAX_CREATES_PER_MINUTE = 500 Thus far in my testing I have found that you may want to set MAX_UPDATES_PER_SECOND to a third of the metrics you expect to receive each minute. By not maximizing the updates setting, you force the system to use cache. It then writes multiple updates to the disk in an optimal fashion (*seen in updates.log). You can check your disk using the command below. I believe in a former thread it was stated that ideally this should be between 20%-50% utilized. iostat -dxk 30 Example of tuning the system. (YMMV) metrics-per-min / updates-per-min = optimal writes 60000 / 18000 = 3 optimal wites updates-per-min is gathered from the MAX_UPDATES_PER_SECOND setting (300/sec * 60 = 12000) Additionally, I have the liberty of plenty of memory so I set MAX_CACHE_SIZE to 5times the volume I receive each min. (60000 * 5) *After testing, you may also want to set LOG_UPDATES to false as this is expensive as it writes a line for each metric. -- You received this question notification because you are a member of graphite-dev, which is an answer contact for Graphite. _______________________________________________ Mailing list: https://launchpad.net/~graphite-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~graphite-dev More help : https://help.launchpad.net/ListHelp
