Hi, I'm running GL 0.20.1 (web & server) on one dedicated server (16x2.93GHz, 32GB RAM) and Elasticsearch (v0.90.10) on second dedicated server (16x2.93GHz, 32GB RAM, SSD disks). ES is not used by any other applications, except this GL2 server. Both servers are in the same network. No firewalls, IDS/IPSs or content filters between them.
Graylog2-server configuration is default, except following lines: processbuffer_processors = 10 outputbuffer_processors = 10 ring_size = 2048 On GL2 server i've setup Raw/Plaintext TCP local input. For testing purposes i have plain text file with exactly 10 log lines. Now i'm pumping logs (10 lines) to graylog: *cat test-10.log |nc localhost 6667* 1st run: ES (head plugin) shows 6 events GL2: 6 messages 2nd run (~20-50 seconds after previous run): ES: 7 events GL2: 7 events 3rd run (~20-50 seconds after previous run): ES: 13 events GL2: 13 events 4th run (~20-50 seconds after previous run): ES: 17 events GL2: 17 events 5th run (~20-50 seconds after previous run): ES: 28 events GL2: 28 events 6th run (~20-50 seconds after previous run): ES: 29 events GL2: 29 events 7th run (~2 minutes after previous run): ES: 33 events GL2: 33 events I have also run tcpdump (tcpdump -i lo -s0 -axX -w /tmp/gl2.pcap port 6667) - pcap files always contains data of 10 events. GL2-server in debug mode shows 10 messages "*org.graylog2.inputs.raw.RawProcessor - Adding received raw message*" (with correct log data at the end of message) 10 messages "*org.graylog2.filters.StreamMatcherFilter - Routed message*" 10 messages "*org.graylog2.buffers.processors.ProcessBufferProcessor - Finished processing message. Writing to output buffer*" And only 5 messages "*org.graylog2.buffers.processors.OutputBufferProcessor - Writing message batch to [ElasticSearch Output]. Size <1>*". Exactly same number of messages were written to ES. So, my question is - what's happening to some logs? Why aren't they stored? regards, -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
