Question #214631 on Graphite changed: https://answers.launchpad.net/graphite/+question/214631
Yee-Ting Li posted a new comment: 5,000: trail and error i'm afraid. i guess the systematic way would be to run a bonnie++ test; however, with the read and write YMMV - but with linux kernel caching etc, it gets messy. i would like to say that i'm trying to keep the IOWAIT to around 20%, but that was just a happy coincidence ;) wrt WHISPER_AUTOFLUSH = True, i haven't tried it but it does sound interesting. please tell me if you have any good experiences with it (i'm rather nervous about prodding a stable system) FLOW_CONTROL: so i would use the carbon-relay to provide a single point of entry into carbon, however, i noticed that it gets rather cpu bound with a large number of metrics. so i basically wrote a simple process that spawns off n processes (let's call them feeders); each process connects to a single carbon-cache (so 16 carbon-caches = 16 feeders). within my code that feeds into carbon, i precompute which carbon-cache would be hit using consistent hashing and then send it just to the relevant feeder. i find that i am able to push >1million metrics/minute using this method (whether carbon can actually consume it is another question). therefore, if the underlying carbon-instance hits it's cache limit, the feeder's socket to that instance is closed, i notice in the feeder and start buffering. so the amount of time it takes to appear depends on what you mean. i believe the issue is that the webapp requires that the whisper file exists on the disk for it to show up in the side bar. therefore, with metrics that have never been seen before, it can take a long time (MAX_CREATES). however, if you already know the name of the metric, the webapp /should/ calculate the relevant carbon-instance to query it's cache so it should be there as soon as an instance receives it. the best way to check this is to look at log/webapp/cache.log. no probs with sharing my experiences: i find that tuning carbon for high performance is really quite a pain. -- You received this question notification because you are a member of graphite-dev, which is an answer contact for Graphite. _______________________________________________ Mailing list: https://launchpad.net/~graphite-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~graphite-dev More help : https://help.launchpad.net/ListHelp
