All, So I think we've found the root cause of this issue,, so I figured I'd report back. It appears that storm itself was not properly configured, and thus it was trying to pull in ganglia to log for all topologies. Once we removed the -javaagent flag from all of the Javaopts configurations for Storm, we got past this issue. See the link below for more details on this particular problem. We used the answer from aditya in this thread to solve the issue.
Thanks everyone for your help and input on this! https://community.hortonworks.com/questions/144755/storm-supervisor-and-nimbus-dropping-immediately-a.html -- David ________________________________ From: Simon Elliston Ball <[email protected]> Sent: Wednesday, September 12, 2018 7:06:54 AM To: [email protected] Cc: Ben Stoker Subject: Re: Metron Not Reading From Kafka? That's a pretty out dated version, and sounds like you may have an old version of HDP if you have the jmx error. >From the info here, there are a lot of things it could potentially be. I would >suggest working through the stages piece by piece, and checking that your >parsers are producing valid messages (correct timestamp?) and check what is >going through the error and invalid topics. Also worth turning up the logging in storm for the parser topologies for a bit (via storm ui) on org.apache.metron. Simon On Fri, 17 Aug 2018 at 16:07, David McGinnis <[email protected]<mailto:[email protected]>> wrote: All, We have a Metron 0.4.3 installation on an HDP cluster which has a sensor set up to read from a Kafka topic and write the data out to Elasticsearch. Data is being inserted into the Kafka topic, and we can read that through Kafka console consumer, but the system is not reporting any data coming through. The Storm spout says no data has been processed, and the index hasn't even been created in Elastic, despite running for nearly a month now. We've searched the worker logs for Storm, and the only error that comes up is a (we think) unrelated error about not being able to find the jmxmetrics JAR file. Metron reports that the topic is found, and does not tell us that the topic is not emitting, so we suspect it sees the data in there. Do you all have any ideas on where we can look to determine the cause of this issue, or things to try? Thanks! -- David McGinnis Staff Hadoop Consultant | Avalon Consulting, LLC <http://www.avalonconsult.com/>M: (513) 439-0082 LinkedIn<http://www.linkedin.com/company/avalon-consulting-llc> | Google+<http://www.google.com/+AvalonConsultingLLC> | Twitter<https://twitter.com/avalonconsult> ------------------------------------------------------------------------------------------------------------- This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited. -- -- simon elliston ball @sireb
