We use graylog 1.3.3, elasticsearch 1.7.4, mongodb 3.1.6, java 1.8,
OpenIndiana (Illumos kernel from December 2015).
The last week we installed the stack for graylog in a 4 node cluster.
Each node has a zone with graylog and elasticsearch and a zone with mongodb.
Everything works so far.
We have 33 streams but the rate of incoming messages is low (about 700
messages/s). We expect higher rates in the next weeks.
In mongodb we created a replicaset and defined it in the mongodb-uri.
The uris are:
mongodb_uri =
mongodb://username:password@mongodb-s1:27017,mongodb-s2:27017,mongodb-s3:27017,mongodb-s4:27017/graylog?replicaSet=rs1&readPreference=primary
and
mongodb_uri =
mongodb://username:password@mongodb-s1:27017,mongodb-s2:27017,mongodb-s3:27017,mongodb-s4:27017/graylog?replicaSet=rs1&readPreference=secondary
When I used the option "readPreference=secondary" the processing of
messages is still working (elasticsearch writes new documents) but the
login in the graylog-web-interface does not work anymore.
After the typing in of username/password and clicking on "Sign in" I am
back on the login screen.
If I change "readPreference=secondary" to "readPreference=primary" (which
is the default) the login works as it should.
I tried to use this option because I observed a very cpu usage of the
mongodb process.
With "readPreference=primary" all writes and all reads go to the master
The output of mongostat is like this and I have no idea if 9349 read
queries are normal for the processing of 700 messages:
insert query update delete getmore command % dirty % used
flushes vsize res qr|qw ar|aw netIn netOut conn set repl time
localhost:27017 *11 *0 *43 *0 36 44|0 0.4
1.3 0 217.0M 193.0M 0|0 1|0 31k 55k 12 kwh SEC 14:03:45
mongodb-s1:27017 *0 9349 15 8 322 212|0 0.2
4.8 0 543.0M 379.0M 0|0 3|0 806k 7m 141 kwh PRI 14:03:46
mongodb-s2:27017 *11 *0 *41 *0 0 8|0 0.3
1.4 0 211.0M 191.0M 0|0 1|0 964b 17k 9 kwh SEC 14:03:45
mongodb-s3:27017 *11 *0 *40 *0 0 7|0 0.2
1.3 0 211.0M 191.0M 0|0 1|0 702b 17k 9 kwh SEC 14:03:45
mongodb-s4:27017 *11 *0 *43 *0 37 42|0 0.4
1.3 0 217.0M 193.0M 0|0 1|0 30k 55k 12 kwh SEC 14:03:45
The mongodb process in mongodb-s1 used 25% of cpu which corresponds to 6 of
24 logical cores. That is really too much.
When I changed the option to "readPreference=secondary" the reads are
equally handled by the secondaries and the master handles only the writes.
CPU usage on all 4 instances was then between 0 and 1%. And this was my
expectation.
Do you have a clue why the mongodb master had this high rate of cpu usage?
Is there a chance to use "readPreference=secondary" and still having a
login?
Regards
Andreas
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/48782b60-29f2-4cbe-a8ef-bf70dce40287%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
