So something ugly did just appear in the kafka "indexing" topic but no ES
indexes generated.
docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper
localhost:2181 --topic indexing
{"exception":"org.apache.metron.guava.enrichment.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException: Theat Intel Unable to retrieve
value","failed_sensor_type":"error","stack":"org.apache.metron.guava.enrichment.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException: Theat Intel Unable to retrieve value\n\tat
org.apache.metron.guava.enrichment.cache.LocalCache$Segment.get(LocalCache.java:2256)\n\tat
org.apache.metron.guava.enrichment.cache.LocalCache.get(LocalCache.java:3980)\n\tat
org.apache.metron.guava.enrichment.cache.LocalCache.getOrLoad(LocalCache.java:3984)\n\tat
org.apache.metron.guava.enrichment.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4868)\n\tat
org.apache.metron.guava.enrichment.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4874)\n\tat
org.apache.metron.enrichment.bolt.GenericEnrichmentBolt.execute(GenericEnrichmentBolt.java:222)\n\tat
org.apache.storm.daemon.executor$fn__7953$tuple_action_fn__7955.invoke(executor.clj:728)\n\tat
org.apache.storm.daemon.executor$mk_task_receiver$fn__7874.invoke(executor.clj:461)\n\tat
org.apache.storm.disruptor$clojure_handler$reify__7390.onEvent(disruptor.clj:40)\n\tat
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:439)\n\tat
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:418)\n\tat
org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)\n\tat
org.apache.storm.daemon.executor$fn__7953$fn__7966$fn__8019.invoke(executor.clj:847)\n\tat
org.apache.storm.util$async_loop$fn__625.invoke(util.clj:484)\n\tat
clojure.lang.AFn.run(AFn.java:22)\n\tat
java.lang.Thread.run(Thread.java:745)\nCaused by:
java.lang.RuntimeException: Theat Intel Unable to retrieve value\n\tat
org.apache.metron.enrichment.adapters.threatintel.ThreatIntelAdapter.enrich(ThreatIntelAdapter.java:100)\n\tat
org.apache.metron.enrichment.adapters.threatintel.ThreatIntelAdapter.enrich(ThreatIntelAdapter.java:40)\n\tat
org.apache.metron.enrichment.bolt.GenericEnrichmentBolt$1.load(GenericEnrichmentBolt.java:150)\n\tat
org.apache.metron.enrichment.bolt.GenericEnrichmentBolt$1.load(GenericEnrichmentBolt.java:147)\n\tat
org.apache.metron.guava.enrichment.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3579)\n\tat
org.apache.metron.guava.enrichment.cache.LocalCache$Segment.loadSync(LocalCache.java:2372)\n\tat
org.apache.metron.guava.enrichment.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2335)\n\tat
org.apache.metron.guava.enrichment.cache.LocalCache$Segment.get(LocalCache.java:2250)\n\t...
15 more\nCaused by:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=35, exceptions:\nWed Jul 19 11:56:31 UTC 2017,
RpcRetryingCaller{globalStartTime=1500465391527, pause=100, retries=35},
java.lang.RuntimeException: java.lang.NullPointerException\nWed Jul 19
11:56:41 UTC 2017, RpcRetryingCaller{globalStartTime=1500465391527,
pause=100, retries=35}, java.lang.RuntimeException:
java.lang.NullPointerException\nWed Jul 19 11:56:52 UTC 2017,
RpcRetryingCaller{globalStartTime=1500465391527, pause=100, retries=35},
java.lang.RuntimeException: java.lang.NullPointerException\nWed Jul 19
11:57:02 UTC 2017, RpcRetryingCaller{globalStartTime=1500465391527,
pause=100, retries=35}, java.lang.RuntimeException:
java.lang.NullPointerException\nWed Jul 19 11:57:13 UTC 2017,
RpcRetryingCaller{globalStartTime=1500465391527, pause=100, retries=35},
java.lang.RuntimeException: java.lang.NullPointerException\nWed Jul 19
11:57:23 UTC 2017, RpcRetryingCaller{globalStartTime=1500465391527,
pause=100, retries=35}, java.lang.RuntimeException:
java.lang.NullPointerException\nWed Jul 19 11:57:33 UTC 2017,
RpcRetryingCaller{globalStartTime=1500465391527, pause=100, retries=35},
java.lang.RuntimeException: java.lang.NullPointerException\nWed Jul 19
11:57:43 UTC 2017, RpcRetryingCaller{
--
Regards,
Kashif Chowdhree
On 19 July 2017 at 13:04, Kashif Chowdhree <[email protected]> wrote:
> Hi,
>
> I've setup metron-docker and successfully have snort and bro logs
> streaming into their respective kafka topics (I tweaked the docker-compose
> configs because I didn't want to use docker-machine plus I have live bro
> and snort sensors running). The enrichment toploogy starts fine, and I can
> see enriched data if I consume the kafka topic.
>
> The issue I have is that the indexing topology doesn't seem to generate
> anything into it's kafka topoc, there are no errors in the logs aside from
> the below. What is it that creates the elasticsearch index and thus allow
> kibana to search against that ES index? No indexes ever get created, per
> http://elasticsearch:9200/_cat/indices?v
>
> health status index pri rep docs.count docs.deleted store.size
> pri.store.size
> yellow open .kibana 1 1 1 0 3.1kb
> 3.1kb
>
>
> Excerpt of errors from /usr/share/apache-storm/logs/
> workers-artifacts/indexing-4-1500464220/6703/worker.log
>
> 2017-07-19 11:37:30.219 o.a.z.ClientCnxn [INFO] Socket connection
> established to elasticsearch/192.168.111.3:2181, initiating session
> 2017-07-19 11:37:30.217 o.a.c.f.r.c.TreeCache [ERROR]
> com.fasterxml.jackson.core.metron.elasticsearch.JsonParseException:
> Unrecognized token 'indexing': was expecting ('true', 'false' or 'null')
> at [Source: java.io.ByteArrayInputStream@3c456c02; line: 1, column: 17]
> at com.fasterxml.jackson.core.metron.elasticsearch.
> JsonParser._constructError(JsonParser.java:1581) ~[stormjar.jar:?]
> at com.fasterxml.jackson.core.metron.elasticsearch.base.
> ParserMinimalBase._reportError(ParserMinimalBase.java:533)
> ~[stormjar.jar:?]
> at com.fasterxml.jackson.core.metron.elasticsearch.json.
> UTF8StreamJsonParser._reportInvalidToken(UTF8StreamJsonParser.java:3451)
> ~[stormjar.jar:?]
> at com.fasterxml.jackson.core.metron.elasticsearch.json.
> UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2610)
> ~[stormjar.jar:?]
> at com.fasterxml.jackson.core.metron.elasticsearch.json.
> UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:841)
> ~[stormjar.jar:?]
> at com.fasterxml.jackson.core.metron.elasticsearch.json.
> UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:737)
> ~[stormjar.jar:?]
> at com.fasterxml.jackson.databind.ObjectMapper._
> initForReading(ObjectMapper.java:3847) ~[stormjar.jar:?]
> at com.fasterxml.jackson.databind.ObjectMapper._
> readMapAndClose(ObjectMapper.java:3792) ~[stormjar.jar:?]
> at com.fasterxml.jackson.databind.ObjectMapper.
> readValue(ObjectMapper.java:2874) ~[stormjar.jar:?]
> at org.apache.metron.common.utils.JSONUtils.load(JSONUtils.java:41)
> ~[stormjar.jar:?]
> at org.apache.metron.common.configuration.IndexingConfigurations.
> updateSensorIndexingConfig(IndexingConfigurations.java:52)
> ~[stormjar.jar:?]
> at org.apache.metron.common.configuration.IndexingConfigurations.
> updateSensorIndexingConfig(IndexingConfigurations.java:48)
> ~[stormjar.jar:?]
> at org.apache.metron.common.bolt.ConfiguredIndexingBolt.updateConfig(
> ConfiguredIndexingBolt.java:54) ~[stormjar.jar:?]
> at
> org.apache.metron.common.bolt.ConfiguredBolt$1.childEvent(ConfiguredBolt.java:94)
> ~[stormjar.jar:?]
> at
> org.apache.curator.framework.recipes.cache.TreeCache$2.apply(TreeCache.java:685)
> [stormjar.jar:?]
> at
> org.apache.curator.framework.recipes.cache.TreeCache$2.apply(TreeCache.java:679)
> [stormjar.jar:?]
> at
> org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)
> [stormjar.jar:?]
> at org.apache.metron.guava.util.concurrent.MoreExecutors$
> SameThreadExecutorService.execute(MoreExecutors.java:297) [stormjar.jar:?]
> at org.apache.curator.framework.listen.ListenerContainer.
> forEach(ListenerContainer.java:84) [stormjar.jar:?]
> at org.apache.curator.framework.recipes.cache.TreeCache.
> callListeners(TreeCache.java:678) [stormjar.jar:?]
> at
> org.apache.curator.framework.recipes.cache.TreeCache.access$1400(TreeCache.java:69)
> [stormjar.jar:?]
> at
> org.apache.curator.framework.recipes.cache.TreeCache$4.run(TreeCache.java:790)
> [stormjar.jar:?]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [?:1.8.0_101]
>
