Hi, I've setup metron-docker and successfully have snort and bro logs streaming into their respective kafka topics (I tweaked the docker-compose configs because I didn't want to use docker-machine plus I have live bro and snort sensors running). The enrichment toploogy starts fine, and I can see enriched data if I consume the kafka topic.
The issue I have is that the indexing topology doesn't seem to generate anything into it's kafka topoc, there are no errors in the logs aside from the below. What is it that creates the elasticsearch index and thus allow kibana to search against that ES index? No indexes ever get created, per http://elasticsearch:9200/_cat/indices?v health status index pri rep docs.count docs.deleted store.size pri.store.size yellow open .kibana 1 1 1 0 3.1kb 3.1kb Excerpt of errors from /usr/share/apache-storm/logs/workers-artifacts/indexing-4-1500464220/6703/worker.log 2017-07-19 11:37:30.219 o.a.z.ClientCnxn [INFO] Socket connection established to elasticsearch/192.168.111.3:2181, initiating session 2017-07-19 11:37:30.217 o.a.c.f.r.c.TreeCache [ERROR] com.fasterxml.jackson.core.metron.elasticsearch.JsonParseException: Unrecognized token 'indexing': was expecting ('true', 'false' or 'null') at [Source: java.io.ByteArrayInputStream@3c456c02; line: 1, column: 17] at com.fasterxml.jackson.core.metron.elasticsearch.JsonParser._constructError(JsonParser.java:1581) ~[stormjar.jar:?] at com.fasterxml.jackson.core.metron.elasticsearch.base.ParserMinimalBase._reportError(ParserMinimalBase.java:533) ~[stormjar.jar:?] at com.fasterxml.jackson.core.metron.elasticsearch.json.UTF8StreamJsonParser._reportInvalidToken(UTF8StreamJsonParser.java:3451) ~[stormjar.jar:?] at com.fasterxml.jackson.core.metron.elasticsearch.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2610) ~[stormjar.jar:?] at com.fasterxml.jackson.core.metron.elasticsearch.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:841) ~[stormjar.jar:?] at com.fasterxml.jackson.core.metron.elasticsearch.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:737) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3847) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3792) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2874) ~[stormjar.jar:?] at org.apache.metron.common.utils.JSONUtils.load(JSONUtils.java:41) ~[stormjar.jar:?] at org.apache.metron.common.configuration.IndexingConfigurations.updateSensorIndexingConfig(IndexingConfigurations.java:52) ~[stormjar.jar:?] at org.apache.metron.common.configuration.IndexingConfigurations.updateSensorIndexingConfig(IndexingConfigurations.java:48) ~[stormjar.jar:?] at org.apache.metron.common.bolt.ConfiguredIndexingBolt.updateConfig(ConfiguredIndexingBolt.java:54) ~[stormjar.jar:?] at org.apache.metron.common.bolt.ConfiguredBolt$1.childEvent(ConfiguredBolt.java:94) ~[stormjar.jar:?] at org.apache.curator.framework.recipes.cache.TreeCache$2.apply(TreeCache.java:685) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.TreeCache$2.apply(TreeCache.java:679) [stormjar.jar:?] at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92) [stormjar.jar:?] at org.apache.metron.guava.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) [stormjar.jar:?] at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:84) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.TreeCache.callListeners(TreeCache.java:678) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.TreeCache.access$1400(TreeCache.java:69) [stormjar.jar:?] at org.apache.curator.framework.recipes.cache.TreeCache$4.run(TreeCache.java:790) [stormjar.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_101]
