Hello,
When rrdcreate starts to create a lot of new rrds, it slows collectd.
This is not new : issue #75.
A side effect is that while the rrd files are created, the
dispatch_thread() function (in network.c) is freezed (at parse_packet()
call at the end of dispatch_thread()).
Consequence : the receive_thread() (and more exactly network_receive())
thread continues to receive packets from the network and store them into
memory.
I noticed the TODO line (possible performance enhancement : do not free
the entries in the dispatch thread but put them in another list, so we
don't have to allocate more and more...
But I don't think this will do much if we receive more and more packets
and the dispatch_thread do not eat them.
Fixing issue #75 solves the problem ? I think not, even if it really
improve the situation. I think not because if too many packets are sent
and there are not enough threads to "eat" them, the problem remains.
--> I think we should limit the max number of packets to accept (or
limit memory : divide by sizeof(structure...)).
If we don't, the OOMkiller does it for us before we get a "network
plugin : malloc failed".
What to do if the max number of packets to accept is reached ? Well,
drop packets. Drop last packets, or accept them but drop the older ones
? I don't know (need to think to that, or allow the admin to configure
it in the config file).
Note that if you have a dedicated collectd for rrd files creation,
dropping packets is not so important : they will come back in a few
minutes.
Does anyone have a better solution ?
Is someone already working on this ?
May I try something (and submit a patch) ?
Regards,
Yves
--
- Homepage - http://ymettier.free.fr
-
- GPG key - http://ymettier.free.fr/gpg.txt
-
- C en action - http://ymettier.free.fr/livres/C_en_action_ed2.html
-
- Guide Survie C - http://www.pearson.fr/livre/?GCOI=27440100673730
-
_______________________________________________
collectd mailing list
[email protected]
http://mailman.verplant.org/listinfo/collectd
