On Mon, 17 Jun 2013, Igor Sverkos wrote:
Hi,
Mahesh wrote:
Howver, I find that mysqld reaches > 80% CPU Usage and rsyslog (20% CPU).
The system is a 4 core x86_84 centos system.
1) What are cores? Well you said x86 so you don't run it on an ARM
device, but it still can be a Celeron/ATOM processor. MIPS and MHz are
still important to know.
1a) that is your disk subsystem, a single SATA "green" drive directly attached
to the system will give you vastly different performance than a RAID card with
battery backed cache, even with the same drive attached to the card.
2) You need to know how mySQL is working. For example, mySQL will
split multiple SESSIONs across the available cores. But each SESSION
is bound to one Core. So you can have 64 Cores, but when doing a large
INSERT in ONE SESSION you will only utilize 1 of 64 cores with that
operation.
3) Your SCHEMA. We don't know anything about it. INDEXes are
important. When you have many INDEXes (and triggers) across your
table, you have to update multiple things next to the row you will
insert itself.
4) And you can do batch INSERTs or you can INSERT each line in a
single statement. The different: Doing a batch INSERT will allow you
to write the new INDEX only once at the end. But when you INSERT each
message in a SINGLE statement mySQL will have to update the INDEX
after EVERY INSERT statement.
As of rsyslog 7.3.5, rsyslog inserts multiple messages per insert (per his
message, he has it configured to insert 1000 messages per transaction). So he is
in good shape on this one.
5) There can be a different between MyISAM and InnoDB.
So mySQL (and any other SQL database engine) is probably not the best
backend for rsyslog. Maybe that's the reason why log analysis toools
(or big data analysis tools in general) won't index into such a
backend, instead they user their own.
agreed.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
