On Fri, 16 Jan 2009, RB wrote: > On Thu, Jan 15, 2009 at 16:01, Daniel Anson <[email protected]> wrote: >> I would hope that there is an easy solution as my next idea is to write >> some type of daemonized process that can insert messages from a pool of >> MySQL connections. I can achieve this in C but would rather hopefully >> find a solution inside of the configuration. > > Short of implementing the queue/worker configuration (no idea how), it > seems the only current option would be to implement something of the > sort, either by an update to the ommysql module (optimal, as it gets > your code supported by someone else for its lifetim) or by some > external program.
multiple workers will help if mySQL can handle more transactions at a time if the hit in parallel. the fact that you are doing 4000/sec indicates that you are not doing a fsync for each insert, so it is unlikly to help (if you are fsync limited the data rates are probably gong to be closer to 100-200/sec depending on your drives) > I'd think an optimal external solution would be some sort of > relp2mysql bridge, but suspect that would end up reimplementing a good > chunk of rsyslog. actually, the optimal solution is to modify rsyslog to be able to handle multiple messages at once in the output queues. that is a major effort (2-4 man weeks) that will require a sponser. Once this is implemented I would expect the throughput to go up by 2-3 orders of magnatude for database inserts. David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
