Hi, I don't have any drools or extractors configured.
Here's debug output (http://dimka.ee/foo/gl2-0.20.1_debug_output.txt). Hope it helps somehow. I sent 5 events to graylog2 Gelf UDP input, but only 3 events were written to ES. regards, On Thursday, March 13, 2014 12:09:30 AM UTC+2, lennart wrote: > > Seems like some filter is discarding messages. Do you have drools > rules in place or any extractors running? > > On Wed, Mar 12, 2014 at 1:36 PM, Dmitri Stoljarov > <[email protected] <javascript:>> wrote: > > Hi, > > > > I'm running GL 0.20.1 (web & server) on one dedicated server > (16x2.93GHz, > > 32GB RAM) and Elasticsearch (v0.90.10) on second dedicated server > > (16x2.93GHz, 32GB RAM, SSD disks). ES is not used by any other > applications, > > except this GL2 server. > > Both servers are in the same network. No firewalls, IDS/IPSs or content > > filters between them. > > > > Graylog2-server configuration is default, except following lines: > > processbuffer_processors = 10 > > outputbuffer_processors = 10 > > ring_size = 2048 > > > > On GL2 server i've setup Raw/Plaintext TCP local input. > > For testing purposes i have plain text file with exactly 10 log lines. > > > > Now i'm pumping logs (10 lines) to graylog: cat test-10.log |nc > localhost > > 6667 > > > > 1st run: > > ES (head plugin) shows 6 events > > GL2: 6 messages > > > > 2nd run (~20-50 seconds after previous run): > > ES: 7 events > > GL2: 7 events > > > > 3rd run (~20-50 seconds after previous run): > > ES: 13 events > > GL2: 13 events > > > > 4th run (~20-50 seconds after previous run): > > ES: 17 events > > GL2: 17 events > > > > 5th run (~20-50 seconds after previous run): > > ES: 28 events > > GL2: 28 events > > > > 6th run (~20-50 seconds after previous run): > > ES: 29 events > > GL2: 29 events > > > > 7th run (~2 minutes after previous run): > > ES: 33 events > > GL2: 33 events > > > > I have also run tcpdump (tcpdump -i lo -s0 -axX -w /tmp/gl2.pcap port > 6667) > > - pcap files always contains data of 10 events. > > > > GL2-server in debug mode shows 10 messages > > "org.graylog2.inputs.raw.RawProcessor - Adding received raw message" > (with > > correct log data at the end of message) > > 10 messages "org.graylog2.filters.StreamMatcherFilter - Routed message" > > 10 messages "org.graylog2.buffers.processors.ProcessBufferProcessor - > > Finished processing message. Writing to output buffer" > > And only 5 messages > "org.graylog2.buffers.processors.OutputBufferProcessor - > > Writing message batch to [ElasticSearch Output]. Size <1>". Exactly same > > number of messages were written to ES. > > > > > > So, my question is - what's happening to some logs? Why aren't they > stored? > > > > regards, > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to [email protected] <javascript:>. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
