Have you tested that grok against that message?
On September 6, 2017 at 06:36:24, Girish N ([email protected]) wrote: Thanks for your response Otto Fowler, I tried with the below config. Still the same exception. Kindly let me know if anything else has to be changed. { "parserClassName": "org.apache.metron.parsers.GrokParser", "sensorTopic": "log", "parserConfig": { "grokPath": "/patterns/log", "patternLabel": "SYS_DELIMITED" }} java.lang.IllegalStateException: Unhandled bulk errors in response: {java.lang.IllegalArgumentException: Topic cannot be null=[source: parserBolt:4, stream: error, id: {}, [{"exception":"java.lang.IllegalStateException: Grok parser Error: Grok statement produced a null message. Original message was: Sep 6 12:41:02 exza-ThinkPad-X240 systemd[1]: Started CUPS Scheduler. and the parsed message was: {} . Check the pattern at: \/patterns\/log on Sep 6 12:41:02 exza-ThinkPad-X240 systemd[1]: Started CUPS Scheduler.","failed_sensor_type":"log","stack":"java.lang.IllegalStateException: Grok parser Error: Grok statement produced a null message. Original message was: Sep 6 12:41:02 exza-ThinkPad-X240 systemd[1]: Started CUPS Scheduler. and the parsed message was: {} Regards Girish N On Wed, Sep 6, 2017 at 3:45 PM, Otto Fowler <[email protected]> wrote: > Change the grokPath to match below and try. > > { > "parserClassName": "org.apache.metron.parsers.GrokParser", > "sensorTopic": "log", > "parserConfig": { > "grokPath": "/patterns/log", > "patternLabel": "SYS_DELIMITED" > }} > > > This path is not an absolute path. > > On September 6, 2017 at 05:16:19, Girish N (giri.narasimha.murthy@gmail. > com) wrote: > > Hi, > > I am trying to parse the syslog I am getting below exceptions. Kindly help > to resolve the issue. Thanks > > > Sample Syslog- > Sep 6 14:13:42 exza-ThinkPad-X240 systemd[1]: Started Suspend. > > > 1. Created a Grok pattern in path /usr/metron/0.4.0/patterns/log > SYS_DELIMITED > <%{NUMBER:queue_id}>+%{SYSLOGTIMESTAMP:timestamp2}(?:%{SYSLOGFACILITY} )? > %{IPORHOST} %{SYSLOGPROG}: %{GREEDYDATA:msg} > > 2. Created a Parser config log.json > { > "parserClassName": "org.apache.metron.parsers.GrokParser", > "sensorTopic": "log", > "parserConfig": { > "grokPath": "usr/metron/0.4.0/patterns/log", > "patternLabel": "SYS_DELIMITED" > }} > > 3. Created a indexing config log.json > { > "hdfs" : { > "index": "log", > "batchSize": 5, > "enabled" : true > }, > "elasticsearch" : { > "index": "log", > "batchSize": 5, > "enabled" : true > }, > "solr" : { > "index": "log", > "batchSize": 5, > "enabled" : true > } > } > > 4. Pushed and Dumped using > usr/metron/0.4.0/bin/zk_load_configs.sh -z localhost:2181 -m PUSH -i > /usr/metron/0.4.0/config/zookeeper > usr/metron/0.4.0/bin/zk_load_configs.sh -z localhost:2181 -m DUMP > > 5. Then started the parser topology , getting the below exceptions in log > worker-artifacts ( > usr/share/apache-storm/logs/workers-artifacts/log-1-1504683893/6700) > > java.lang.IllegalStateException: Unhandled bulk errors in response: > {java.lang.IllegalArgumentException: Topic cannot be null=[source: > parserBolt:4, stream: error, id: {}, > [{"exception":"java.lang.IllegalStateException: Grok parser Error: Grok > statement produced a null message. Original message was: Sep 6 12:41:02 > exza-ThinkPad-X240 systemd[1]: Started CUPS Scheduler. and the parsed > message was: {} . Check the pattern at: \/patterns\/log on Sep 6 12:41:02 > exza-ThinkPad-X240 systemd[1]: Started CUPS > Scheduler.","failed_sensor_type":"log","stack":"java. > lang.IllegalStateException: > Grok parser Error: Grok statement produced a null message. Original message > was: Sep 6 12:41:02 exza-ThinkPad-X240 systemd[1]: Started CUPS Scheduler. > and the parsed message was: {} > > 6. For the enrichment topology, getting the below exception. > 2017-09-06 07:46:16.249 o.a.k.c.p.ProducerConfig [WARN] The configuration > request.required.acks = 1 was supplied but isn't a known config. > 2017-09-06 07:46:16.249 o.a.k.c.u.AppInfoParser [INFO] Kafka version : > 0.10.0.1 > 2017-09-06 07:46:16.249 o.a.k.c.u.AppInfoParser [INFO] Kafka commitId : > a7a17cdec9eaa6c5 > 2017-09-06 07:46:16.250 o.a.s.d.executor [INFO] Prepared bolt > enrichmentErrorOutputBolt:(1) > 2017-09-06 07:46:16.704 o.a.m.c.d.f.r.BaseFunctionResolver [WARN] Using > System classloader > 2017-09-06 07:46:16.811 o.a.s.d.executor [INFO] Prepared bolt > threatIntelSplitBolt:(14) > 2017-09-06 07:46:16.813 o.a.s.d.executor [INFO] Prepared bolt > enrichmentSplitBolt:(3) > 2017-09-06 07:46:16.815 o.a.s.d.executor [INFO] Prepared bolt > hostEnrichmentBolt:(5) > 2017-09-06 07:46:16.818 o.a.s.d.executor [INFO] Prepared bolt > enrichmentJoinBolt:(2) > 2017-09-06 07:46:16.832 o.a.c.f.r.c.TreeCache [ERROR] > org.apache.metron.jackson.core.JsonParseException: Unrecognized token > 'enrichments': was expecting ('true', 'false' or 'null') > at [Source: java.io.ByteArrayInputStream@6624ac0e; line: 1, column: 23] > at > org.apache.metron.jackson.core.JsonParser._constructError(JsonParser. > java:1586) > ~[stormjar.jar:?] > at > org.apache.metron.jackson.core.base.ParserMinimalBase._ > reportError(ParserMinimalBase.java:521) > ~[stormjar.jar:?] > at > org.apache.metron.jackson.core.json.UTF8StreamJsonParser._ > reportInvalidToken(UTF8StreamJsonParser.java:3466) > ~[stormjar.jar:?] > at > org.apache.metron.jackson.core.json.UTF8StreamJsonParser._ > handleUnexpectedValue(UTF8StreamJsonParser.java:2625) > ~[stormjar.jar:?] > at > org.apache.metron.jackson.core.json.UTF8StreamJsonParser._ > nextTokenNotInObject(UTF8StreamJsonParser.java:854) > ~[stormjar.jar:?] > at > org.apache.metron.jackson.core.json.UTF8StreamJsonParser.nextToken( > UTF8StreamJsonParser.java:748) > ~[stormjar.jar:?] > at > org.apache.metron.jackson.databind.ObjectMapper._ > initForReading(ObjectMapper.java:3847) > ~[stormjar.jar:?] > at > org.apache.metron.jackson.databind.ObjectMapper._ > readMapAndClose(ObjectMapper.java:3792) > ~[stormjar.jar:?] > at > org.apache.metron.jackson.databind.ObjectMapper. > readValue(ObjectMapper.java:2867) > ~[stormjar.jar:?] > at org.apache.metron.common.utils.JSONUtils.load(JSONUtils.java:55) > ~[stormjar.jar:?] > > 7. For the Indexing topology , getting below exception > com.fasterxml.jackson.core.metron.elasticsearch.JsonParseException: > Unrecognized token 'indexing': was expecting ('true', 'false' or 'null') > at [Source: java.io.ByteArrayInputStream@6c1f5065; line: 1, column: 17] > at > com.fasterxml.jackson.core.metron.elasticsearch. > JsonParser._constructError(JsonParser.java:1581) > ~[stormjar.jar:?] > at > com.fasterxml.jackson.core.metron.elasticsearch.base.ParserMinimalBase._ > reportError(ParserMinimalBase.java:533) > ~[stormjar.jar:?] > at > com.fasterxml.jackson.core.metron.elasticsearch.json. > UTF8StreamJsonParser._reportInvalidToken(UTF8StreamJsonParser.java:3451) > ~[stormjar.jar:?] > at > com.fasterxml.jackson.core.metron.elasticsearch.json. > UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java: > 2610) > ~[stormjar.jar:?] > at > com.fasterxml.jackson.core.metron.elasticsearch.json. > UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:841) > ~[stormjar.jar:?] > at > com.fasterxml.jackson.core.metron.elasticsearch.json.UTF8StreamJsonParser. > nextToken(UTF8StreamJsonParser.java:737) > ~[stormjar.jar:?] > at > com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper. > java:3847) > ~[stormjar.jar:?] > at > com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper. > java:3792) > ~[stormjar.jar:?] > at > com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java: > 2874) > ~[stormjar.jar:?] > at org.apache.metron.common.utils.JSONUtils.load(JSONUtils.java:41) > ~[stormjar.jar:?] > at > org.apache.metron.common.configuration.IndexingConfigurations. > updateSensorIndexingConfig(IndexingConfigurations.java:52) > ~[stormjar.jar:?] > > > Kindly help to resolve the issue. > > Regards > Girish N > >
