Re: Metron Not Reading From Kafka?

2018-08-17 Thread David McGinnis
Thank you all for the quick recommendations. We'll follow up on these recommendations and report back. In the meantime, I can say that we don't have an error index even created in Elastic, so there's nothing in there of value. On Fri, Aug 17, 2018 at 10:13 AM, Simon Elliston Ball < si...@simonelli

Re: Metron Not Reading From Kafka?

2018-08-17 Thread Simon Elliston Ball
It might be worth looking in the error and invalid topics, if you have any validation, or your parser is not producing proper timestamps (that's what I usually forget to check!) you may be getting messages routed to the error index. Are you indexing topologies picking any of this up? Is there anyth

Re: Metron Not Reading From Kafka?

2018-08-17 Thread Casey Stella
Can you turn on root level logging to DEBUG from the storm UI (see https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_storm-component-guide/content/enabling-storm-log-levels.html) in the parser topology for 1 minute and look at the logs for anything amiss? Also, can you do a console consu

Metron Not Reading From Kafka?

2018-08-17 Thread David McGinnis
All, We have a Metron 0.4.3 installation on an HDP cluster which has a sensor set up to read from a Kafka topic and write the data out to Elasticsearch. Data is being inserted into the Kafka topic, and we can read that through Kafka console consumer, but the system is not reporting any data coming