Hi Lennart, I have managed to run graylog2 with localhost configuration. Setting this "rest_transport_uri = http://127.0.0.1:12900/"; in graylog2.conf file fixes the problem with login to web interface, but logs are still missing :(
Also i ran all programs (graylog2-server, graylog2-webserver and elasticsearch) with different java's. Openjdk: *# java -versionjava version "1.7.0_25"OpenJDK Runtime Environment (rhel-2.3.10.4.el6_4-x86_64)OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)* and Oracle's native java: *# /opt/java/bin/java -versionjava version "1.7.0_55"Java(TM) SE Runtime Environment (build 1.7.0_55-b13)Java HotSpot(TM) 64-Bit Server VM (build 24.55-b03, mixed mode)* And the results are also the same - not all logs are written to elasticsearch :( Another weird thing i noticed. My graylog2.conf has *output_batch_size = 500 *(rest of configuration is default) Now i'm sending 500 log lines to raw_tcp input (*cat 500.log |nc localhost 4450*). Elasticsearch head plugin shows 391 lines in graylog index. A lot of events are missing. Now i'm deleting this index and wait 5 seconds, until graylog2-server recreates empty index and graylog2-deflector alias. Ok, empty index is created (note, graylog2 server was NOT restarted). Next run for "*cat 500.log |nc localhost 4450"* - now elasticsearch has 531 lines of logs. Have you seen similar behavior with graylog2? Any ideas about that issue? regards, On Tuesday, May 6, 2014 12:18:33 AM UTC+3, lennart wrote: > > Hey Dmitri, > > thanks for your detailled analysis! The TCP input logging as UDP input > was a mistake on our side but just on the logger configuration. (fixed > in f6a978eb151c88f6241aabf4c7bd93c76be70bac) Thanks for spotting this! > > The extreme amount of logging for users and auth is fine in TRACE mode. > > The missing messages are still strange. Did you try listening on > 127.0.0.1 already? This avoids big parts of the network stack. > > Thanks, > Lennart > > On Mon, May 5, 2014 at 9:14 PM, Dmitri Stoljarov > <[email protected] <javascript:>> wrote: > > Hi, > > > > Just downloaded and installed graylog2 v 0.20.2-rc1. > > Configuration of graylog2 is default, except mongoDB and some ES > settings > > (my configuration file: http://dimka.ee/foo/graylog2.conf.txt) > > In Graylog2 i do not have any streams, drools or field extractors. Only > > thing is set - is RawTCP Gelf input on port 4452. > > > > Elasticsearch's version is 0.90.10. Here's my Elasticsearch > configuration > > file: http://dimka.ee/foo/elasticsearch.yml.txt > > > > Graylog server was started with "--debug" command line key. After gl2 > server > > and webserver were up and running i enabled all four dropdowns in > logging > > page to "TRACE" level. > > > > Next step was to send some logs (just 5 lines) to graylog2 server with > > following command "cat 5.log | nc 172.28.28.27 4452" (here's my 5.log > file: > > http://dimka.ee/foo/5.log.txt). > > > > Result was: from 5 messages only 3 messages were in Elasticsearch. > Here's > > full debug-trace file of graylog server > > (http://dimka.ee/foo/graylog2-server.log.3.messages.gz) > > > > After that, i stopped all graylog2 server and webserver instances, > cleared > > elasticsearch database and repeated again. > > Result was: from 5 messages only 4 messages were in Elasticsearch. > Here's > > graylog2 server log file from my second try > > (http://dimka.ee/foo/graylog2-server.log.4.messages.gz) > > > > Server, where i run these test is totally idle. All software (ES, mongo, > > GL2) is on the same server. But i don't think that it's an issue for > just 5 > > lines of logs. > > > > Here's some my observations of gathered logs: > > > > 1. I created RawTCP input, but logs are showing UDP: > > 2014-05-05 19:18:32,219 INFO : org.graylog2.inputs.raw.udp.RawUDPInput - > > Started raw TCP input on /172.28.28.27:4452 > > > > 2. Both times all 5 messages are getting to graylog2 server with unique > IDs. > > All messages are sent to [StreamMatcher], [Static field appender], > > [Blacklister], [Rewriter] processors. OutputBufferProcessor processes > also > > all messages. But "Message ids in batch of …." happens only 3 or 4 times > in > > my case, not 5. > > > > 3. security.realm.MongoDbAuthorizationRealm - almost 700 events per > second > > is written to log file. Isn't it too much? > > > > 4. org.graylog2.users.User - also almost 700 events per second fired. > > > > > > Any ideas or additional steps i can do to help solve this issue? > > > > best regards, > > > > > > > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to [email protected] <javascript:>. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
