Try to add a tag_on_failure on each grok filter to identify which grok filter is failing. I had the same issue and by explicitly setting a tag on each grok I could determine the one causing issues.
Op vrijdag 14 november 2014 16:23:41 UTC+1 schreef Billy F: > > I have a message that is driving me nuts and I don't know how to fix it. > For some reason I'm getting a _grokparsefailure for every one the entries I > have for one of our blade enclosures. Everything that I can see shows that > it's actually working the right way and doing everything I ask it to. > > I did test this out with the grokdebug <https://grokdebug.herokuapp.com/> > and it works exactly the way I expect it to. I also did the config check > and it came back fine as well. > > There is nothing in the logstash logs that show any reason why it's > failing. In fact I stripped my test machine of all of the other config > information and only have this one thing hitting. > > The raw messages are below. Please note that there is no year in the > message, don't ask me why. > > <12> Nov 13 10:06:00 8.10.255.10 OA: Administrator logged into the Onboard > Administrator from 10.121.255.205 > <12> Nov 13 17:16:10 8.10.255.10 OA: Administrator logged out of the > Onboard Administrator > > > My config looks like this. > > input { > redis { > host => "8.8.8.8" > data_type => "list" > type => "redis-input" > key => "logstash" > } > syslog { > type => "HPOAEvents" > port => 1551 > } > } > > filter { > if [type] == "HPOAEvents" { > dns { > reverse => [ "host" ] > action => "replace" > } > mutate { > add_tag => [ "HPOnboardAdminEventlog" ] > } > } > } > > filter { > if [type] == "HPOAEvents" { > grok { > match => [ > "message", "<%{POSINT:syslog_pri}> > (?<timestamp>%{MONTH} %{MONTHDAY} %{TIME}) %{IPORHOST:syslog_hostname} > %{GREEDYDATA:syslog_message}" > ] > } > mutate { > replace => [ "@source_host", "%{host}" ] > replace => [ "@source_host_ip", > "%{syslog_hostname}" ] #ip comes across into this field so just moving it > to the right place. > rename => [ "syslog_message", "@message" ] > rename => [ "timestamp", "@time" ] > } > } > } > > output { > stdout { } > elasticsearch { > cluster => "logstash" > protocol => "http" > } > } > > > Here is the output. > { > "_index": "logstash-2014.11.14", > "_type": "HPOAEvents", > "_id": "uVyt2i8CQfiSkGyowdfqlw", > "_score": null, > "_source": { > "message": "<12> Nov 13 10:06:01 8.10.255.10 OA: Administrator logged > out of the Onboard Administrator", > "@version": "1", > "@timestamp": "2014-11-14T15:06:00.274Z", > "type": "HPOAEvents", > "host": "oa1enc1rk5.mydomain.com", > "tags": [ > "_grokparsefailure", > "HPOnboardAdminEventlog" > ], > "priority": 13, > "severity": 5, > "facility": 1, > "facility_label": "user-level", > "severity_label": "Notice", > "syslog_pri": "12", > "syslog_hostname": "8.8.8.8", > "@message": "OA: Administrator logged out of the Onboard > Administrator", > "@time": "Nov 14 10:06:01", > "@source_host": "oa1enc1rk5.mydomain.com", > "@source_host_ip": "8.8.8.8" > }, > "sort": [ > 1415977560274, > 1415977560274 > > Is there somewhere else I should be looking for the parse failure? > > Thanks in advance, > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/bd15cb8d-f1a1-470e-b662-f56bab950938%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
