Hi Peter, Paolo and all, On Sat, 11 Nov 2006, Peter Nixon wrote:
> This is something I have discussed with Paolo on a number of occasions > also. I see no practical reason why pmacct cannot store all flows with > src and dest port and ip for a low speed link (<10Mbit) yet 256K of > traffic manages to overwhelm an opteron DB server. Ouch, I didn't know that the performance was that bad, thanks for the warning. Do you mean 256 kbits or kbytes? For what it's worth I have pmacct monitoring an 8mbit/384kbit link which is often full, on a Celeron 366, and it "normally" runs OK, but it has brought down that box on one occasion due to the number of database threads spawned, and the resulting system load. > This is purely because of the WAY that pmacct access the database, not > the amount of data (At least from the testing I have done), but it would > require a change to the way DB access is handled to fix. Basically any > query over threadlimit (which should default to 2 or 3) should be queued > instead of dropped...This obviously requires someone write (or borrow) a > decent queuing system for db access.. I don't think it's as hard as all that. The OS will happily queue UDP and pcap packets for us while we write to the database. So I'd just drop the separate thread entirely, and write to the database in the main thread. Under normal conditions this will not cause any problems at all, and it will degrade gracefully under load, unlike the current situation. Paolo, please could you help us to do something about this? It appears to be a real problem with pmacct that affects several users. Cheers, Chris. -- (aidworld) chris wilson | chief engineer (http://www.aidworld.org) _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
