We did not enable it directly. We have two theories on how this flag was set:
1. Their cluster is pretty old (oldest version we've found is HDP 2.2), and back then Ganglia was used for monitoring instead of Ambari Metrics. This may have been a leftover from those days. Storm wasn't used until Metron was installed, so it could've hidden dormant for a while without being detected. They think Storm was installed with Metron, which would indicate this isn't the case, but we aren't confident in that assertion. 2. Metron added the flag as part of the Storm installation (or it is by default in the Storm installation that was used). I'm not sure why this would be the case if Ganglia isn't brought in automatically with Storm or Metron, but assuming that they are correct that Storm was installed with Metron, this is the only answer that makes sense. On Wed, Sep 19, 2018 at 9:52 AM James Sirota <[email protected]> wrote: > Was this flag set by default in the Metron install? Or is that something > that you enabled? > > > 14.09.2018, 13:52, "David McGinnis" <[email protected]>: > > All, > > So I think we've found the root cause of this issue,, so I figured I'd > report back. It appears that storm itself was not properly configured, and > thus it was trying to pull in ganglia to log for all topologies. Once we > removed the -javaagent flag from all of the Javaopts configurations for > Storm, we got past this issue. See the link below for more details on this > particular problem. We used the answer from aditya in this thread to solve > the issue. > > Thanks everyone for your help and input on this! > > > https://community.hortonworks.com/questions/144755/storm-supervisor-and-nimbus-dropping-immediately-a.html > > -- David > > ------------------------------ > *From:* Simon Elliston Ball <[email protected]> > *Sent:* Wednesday, September 12, 2018 7:06:54 AM > *To:* [email protected] > *Cc:* Ben Stoker > *Subject:* Re: Metron Not Reading From Kafka? > > That's a pretty out dated version, and sounds like you may have an old > version of HDP if you have the jmx error. > > From the info here, there are a lot of things it could potentially be. I > would suggest working through the stages piece by piece, and checking that > your parsers are producing valid messages (correct timestamp?) and check > what is going through the error and invalid topics. > > Also worth turning up the logging in storm for the parser topologies for a > bit (via storm ui) on org.apache.metron. > > Simon > > On Fri, 17 Aug 2018 at 16:07, David McGinnis <[email protected]> > wrote: > > All, > > We have a Metron 0.4.3 installation on an HDP cluster which has a sensor > set up to read from a Kafka topic and write the data out to Elasticsearch. > Data is being inserted into the Kafka topic, and we can read that through > Kafka console consumer, but the system is not reporting any data coming > through. The Storm spout says no data has been processed, and the index > hasn't even been created in Elastic, despite running for nearly a month > now. > > We've searched the worker logs for Storm, and the only error that comes up > is a (we think) unrelated error about not being able to find the jmxmetrics > JAR file. Metron reports that the topic is found, and does not tell us that > the topic is not emitting, so we suspect it sees the data in there. > > Do you all have any ideas on where we can look to determine the cause of > this issue, or things to try? > > Thanks! > > -- > David McGinnis > Staff Hadoop Consultant | Avalon Consulting, LLC > <http://www.avalonconsult.com/>M: (513) 439-0082 > LinkedIn <http://www.linkedin.com/company/avalon-consulting-llc> | Google+ > <http://www.google.com/+AvalonConsultingLLC> | Twitter > <https://twitter.com/avalonconsult> > > ------------------------------------------------------------------------------------------------------------- > This message (including any attachments) contains confidential information > intended for a specific individual and purpose, and is protected by law. > If > you are not the intended recipient, you should delete this message. Any > disclosure, copying, or distribution of this message, or the taking of any > action based on it, is strictly prohibited. > > > > -- > -- > simon elliston ball > @sireb > > > > ------------------- > Thank you, > > James Sirota > PMC- Apache Metron > jsirota AT apache DOT org > > -- David McGinnis Staff Hadoop Consultant | Avalon Consulting, LLC <http://www.avalonconsult.com/>M: (513) 439-0082 LinkedIn <http://www.linkedin.com/company/avalon-consulting-llc> | Google+ <http://www.google.com/+AvalonConsultingLLC> | Twitter <https://twitter.com/avalonconsult> ------------------------------------------------------------------------------------------------------------- This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited.
