Hi everyone, We also found out, that replacing the system allocator for collect with jemalloc or tcmalloc significantly improves the enduring memory footprint.
We suffer from a problem called 'heap fragmentation' here to wich the glibc's allocation algorith is vulnerable. It will reserve chunks of memory from the system; and rent it to parts of the process. Once such a block is full, a new blog will be reserved. However, only the very latest block will be used to rent out memory to the application. So, if you have a single byte valid allocated for a longer time in such a memory block, it will remain 'forever yours'. > Mon Dec 14 2015 14:09:35 EST from "Vaughn Wernersbach" ><[email protected]> Subject: Re: [collectd] Preventing >collectd high memory usage > > > > >Hi André Luiz da Silva! Thanks for the suggestion. I’ll keep >switching to UDP in mind, though I’m inclined to stick with TCP if at all >possible (largely to minimize the required changes to my environment). > > > >All - Anyone have info on using CollectInternalStats? Still need to know >what value to set WriteQueueLimitHigh to and having collectd tell me >definitely seems like the best bet. > > > > > > > >*Vaughn Wernersbach* > >*BIDS Trading, L.P.* > > > >111 Broadway, Suite 1603 | New York, NY 10006 > >T: 212.618.2071 | F: 212.618.2031 | C: 732.397.8344 | E: >[email protected] > > > >*From:* André Luiz da Silva [mailto:[email protected]] > *Sent:* Monday, December 14, 2015 11:18 AM > *To:* Vaughn Wernersbach <[email protected]> > *Cc:* [email protected] > *Subject:* Re: [collectd] Preventing collectd high memory usage > > > >Hi, Wenersbach! Did you try to send your metrics over UDP instead of TCP? >It's just a suggestion if you have a reliable network. > > > >Regards. > > > >Em 14/12/2015 1:54 PM, "Vaughn Wernersbach" ><[email protected]> escreveu: >> >> >>Hi, >> >> >> >>In order to limit high memory usage (one of my servers hit 94% for >>collectd) I’m interested in enabling the WriteQueueLimitHigh option, but I >>don’t know what value to use. The docs suggest enabling >>CollectInternalStats, and that the “write_queue” plugin instance reports >>the info from CollectInternalStats that I’m looking for. However, I >>don’t see any “write_queue” plugin nor do I see anything relevant being >>written by my existing write plugins (syslog and graphite) after enabling >>CollectInternalStats. >> >> >> >>· How do I obtain data written collected when setting >>CollectInternalStats to true? >> >>· Alternately, are there any guidelines for setting the >>values of WriteQueueLimitHigh? >> >> >> >> >> >> >> >>*Vaughn Wernersbach* >> >>*BIDS Trading, L.P.* >> >> >> >>111 Broadway, Suite 1603 | New York, NY 10006 >> >>T: 212.618.2071 | F: 212.618.2031 | C: 732.397.8344 | E: >>vaughn.wernersbach@bidstradingcom >> >> >> >> >> >> >> >> >> _______________________________________________ >> collectd mailing list >> [email protected] >> http://mailman.verplant.org/listinfo/collectd > > > > > > > > > > (, 0 bytes) [View| Download] > > > > > > >
_______________________________________________ collectd mailing list [email protected] http://mailman.verplant.org/listinfo/collectd
_______________________________________________ collectd mailing list [email protected] http://mailman.verplant.org/listinfo/collectd
