Hi Jochen, I have added some more logs in my script, if you can run this you should be able identify the alerts missing,
>>> https://github.com/rayeesnp/graylog-performance/tree/master >>> <https://github.com/rayeesnp/graylog-performance/tree/master> > python log_gen.py -p 1 -t 3 above script run for 3 minute with one process, in each cycle it should generate 69 - 70 alerts, but atleast 5 alert missing in each cycle Regards, Rayees > On Jan 25, 2017, at 11:18 AM, Rayees Namathponnan <[email protected]> > wrote: > > Hi Jochen, > > My script generate 70 messages in one minute, i.e. 70 messages pushed to > graylog in 1 minute, > > I think the issue is what ever the messages processed before the alert > trigger its getting added in alert, example 50 message received and process > in graylogm then this will be part of next alert and ignoring the remaining > 20. > > also i can see all the 70 messages in graylog, i.e. if i do a search in > graylog after sometime, I can see all messages. > > > Regards, > Rayees > > > > > >> On Jan 25, 2017, at 11:02 AM, Rayees Namathponnan <[email protected] >> <mailto:[email protected]>> wrote: >> >> graylog version Graylog 2.1.2+50e449a >> >> >>> On Jan 25, 2017, at 7:22 AM, Jochen Schalanda <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hi Rayees, >>> >>> which version of Graylog are you using? >>> >>> Cheers, >>> Jochen >>> >>> On Monday, 23 January 2017 17:03:09 UTC+1, Rayees Namathponnan wrote: >>> Hi All, >>> >>> I have written a script to generate 501 message / second with 1 unique >>> string in 501 th to generate alert, if you run the script it generate >>> messages 70 unique string and expecting an alert with message count is 70, >>> but graylog always report only 60-65 messages, looks like some alerts as >>> missing in graylog, more details are below >>> >>> >>> Script >>> -------- >>> >>> You can get script from >>> https://github.com/rayeesnp/graylog-performance/tree/master >>> <https://github.com/rayeesnp/graylog-performance/tree/master> >>> >>> There are two scripts one generate logs “log_gen.py” and fl_app.py is >>> python flask app it can receive alert from graylog alert HTTP call back and >>> report number of alert received from graylog >>> >>> if you run this script, it will generate 500 message like message A >>> [random ip address before GET] and 1 message like B [ >>> hostname_process_string_uniquenumber ] >>> >>> message a >>> --------------- >>> 2017-01-19 19:00:01.612519 - sjelk34_0 - [218.193.16.244] "GET /wheelsets >>> HTTP/1.0" 200 3148 "http://bleater.com <http://bleater.com/>" "Mozilla/5.0 >>> (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 >>> (KHTML, like Gecko) Chrome/36.0.1944.0 Safari/537.36" >>> >>> message B >>> --------------- >>> 2017-01-19 19:00:01.612573 - sjelk34_0 - [sjelk34_0_uni_68] "GET /wheelsets >>> HTTP/1.0" 200 4879 "http://bleater.com <http://bleater.com/>" "Mozilla/5.0 >>> (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) >>> Chrome/36.0.1944.0 Safari/537.36” >>> >>> >>> >>> Graylog Configurations >>> ------------------------------- >>> File beat to collect log, graylog collector log harvest log from /data/logs >>> >>> Configured extractor to extract the the string “hostname_process", my cases >>> added regular expression “(sjvm34_0+)”, field contains the string >>> “sjvm34_0_uni”, store as filed “message_tag_0” >>> >>> Created stream rule with field tag “message_tag_0” contains “sjvm34_0” >>> >>> In manage alert configure “Alert is triggered when there is more than one >>> message in the last 3 minutes. Grace period: 3 minutes.” >>> >>> Then execute the script, by default script will execute 1 minute and >>> generate 70 unique script with sequence number, i am expecting alert >>> message with 70 message in alert, but alert generating only for 60-65 >>> messages. >>> >>> >>> Regards, >>> Rayees >>> >>> >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Graylog Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected] >>> <mailto:[email protected]>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/graylog2/a2faaee1-235a-448b-a8e6-79f74a15d916%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/graylog2/a2faaee1-235a-448b-a8e6-79f74a15d916%40googlegroups.com?utm_medium=email&utm_source=footer>. >>> For more options, visit https://groups.google.com/d/optout >>> <https://groups.google.com/d/optout>. >> > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/898DD8C7-8E60-4EF1-B770-613A85133E8D%40gmail.com. For more options, visit https://groups.google.com/d/optout.
