Question #83274 on Graphite changed: https://answers.launchpad.net/graphite/+question/83274
Steve Brunton posted a new comment: A little update in case anyone is interested. After looking through the render function a little I realized it's a sequential request through whisper as you ask for more than one datapoint per request. I ended up using the python 2.6 multiprocessing module to abuse me some cpu time. A calculator process fires up per range and finds all the items per that range it needs to calculate data for. Each caculator then breaks up those items into groups of five and creates up to five processes to make requests against graphite render to get the data back and continues creating them until it has retrieved all the data sets that it needs. Then they each do all their calculations and store the data off. On the production hardware 4,581 data items is taking about 10 miuntes from start to finish. Each calculator range also has a flock() so they don't stomp on each other if they run for more than fifteen minutes. The final result is the popularity calculation that is used to generate the results on the new NewsPulse product for CNN.com which launches this weekend. -- You received this question notification because you are a member of graphite-dev, which is an answer contact for Graphite. _______________________________________________ Mailing list: https://launchpad.net/~graphite-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~graphite-dev More help : https://help.launchpad.net/ListHelp
