> > > > About up to 3000 Messages/s To be more precise: That will be the expected peak. So the average will be much lower.
And we have decent hardware to do this. > > at that data rate you will have a very hard time doing things the way > you > are trying. > > at the database level, doing 3000 inserts/sec requires _very_ high-end > hardware. > rsyslog supports inserting multiple messages in one command, > and > with that you could probably handle 3000 messages/sec on very low-end > hardware (because rsyslog could insert messages in batches of 100 or > more > if needed). unfortunantly, when you enable this, you don't maintain > the > message order. > > at the rsyslog level, doing disk based queues with everything tuned to > come as close to minimizing the chance of data loss is very hard to do > at > that traffic level. I ran tests last year on a 8 core box with 64G of > ram > and a fusion-io SSD pci card, and depending on the filesystem I used, > I > was able to get from 2K to 8K messages/sec where I wasn't trying to do > anything more than write the message to a log file. > Thanks for the numbers. > you are going to have to think very hard about how critical it really > is > for you to maintain the same message order and what type of > reliability > you really need for your messages. > > it's counter-intuitive, but it may be that you end up with better > overall > reliability with a much faster configuration that has worse > 'worst-case' > data loss, but is fast enough that under normal conditions everything > is > processed really fast than you would under a configuration that is > much > slower all the time, but has a better worst-case data loss. Yes, I will keep that in mind, thanks. > > David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
