On Wed 08 Nov 2006 09:00, Chris Wilson wrote: --snip-- > >> I'm still concerned about the performance of the MySQL plugin with > >> threading, so I'm considering providing an option to disable the extra > >> threads, and run updates synchronously. > > > > Interesting. What about having also a switch to have "numbers-only" > > tables, that is IP addresses, timestamps, class_id, mac addresses and > > protocol are all stored as integers? > > I don't see how that would help. It's basically just changing the constant > multiplier cost. The problem I'm having is that when the database or the > box is busy, pmacct starts spawning more and more threads that end up > sleeping on the database. This eats resources and can lead to catastrophic > failure (it has done it to me at least once). I would rather delay writing > to the database by having it done synchronously, to limit the damage that > it can do to the rest of the box.
This is something I have discussed with Paolo on a number of occasions also. I see no practical reason why pmacct cannot store all flows with src and dest port and ip for a low speed link (<10Mbit) yet 256K of traffic manages to overwhelm an opteron DB server. This is purely because of the WAY that pmacct access the database, not the amount of data (At least from the testing I have done), but it would require a change to the way DB access is handled to fix. Basically any query over threadlimit (which should default to 2 or 3) should be queued instead of dropped...This obviously requires someone write (or borrow) a decent queuing system for db access.. Cheers -- Peter Nixon http://www.peternixon.net/ PGP Key: http://www.peternixon.net/public.asc _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
