That's great to hear David. Thanks for reporting back! On Fri, Sep 14, 2018, 2:52 PM David McGinnis <[email protected]> wrote:
> All, > > So I think we've found the root cause of this issue,, so I figured I'd > report back. It appears that storm itself was not properly configured, and > thus it was trying to pull in ganglia to log for all topologies. Once we > removed the -javaagent flag from all of the Javaopts configurations for > Storm, we got past this issue. See the link below for more details on this > particular problem. We used the answer from aditya in this thread to solve > the issue. > > Thanks everyone for your help and input on this! > > > https://community.hortonworks.com/questions/144755/storm-supervisor-and-nimbus-dropping-immediately-a.html > > -- David > > ------------------------------ > *From:* Simon Elliston Ball <[email protected]> > *Sent:* Wednesday, September 12, 2018 7:06:54 AM > *To:* [email protected] > *Cc:* Ben Stoker > *Subject:* Re: Metron Not Reading From Kafka? > > That's a pretty out dated version, and sounds like you may have an old > version of HDP if you have the jmx error. > > From the info here, there are a lot of things it could potentially be. I > would suggest working through the stages piece by piece, and checking that > your parsers are producing valid messages (correct timestamp?) and check > what is going through the error and invalid topics. > > Also worth turning up the logging in storm for the parser topologies for a > bit (via storm ui) on org.apache.metron. > > Simon > > On Fri, 17 Aug 2018 at 16:07, David McGinnis <[email protected]> > wrote: > >> All, >> >> We have a Metron 0.4.3 installation on an HDP cluster which has a sensor >> set up to read from a Kafka topic and write the data out to Elasticsearch. >> Data is being inserted into the Kafka topic, and we can read that through >> Kafka console consumer, but the system is not reporting any data coming >> through. The Storm spout says no data has been processed, and the index >> hasn't even been created in Elastic, despite running for nearly a month >> now. >> >> We've searched the worker logs for Storm, and the only error that comes >> up is a (we think) unrelated error about not being able to find the >> jmxmetrics JAR file. Metron reports that the topic is found, and does not >> tell us that the topic is not emitting, so we suspect it sees the data in >> there. >> >> Do you all have any ideas on where we can look to determine the cause of >> this issue, or things to try? >> >> Thanks! >> >> -- >> David McGinnis >> Staff Hadoop Consultant | Avalon Consulting, LLC >> <http://www.avalonconsult.com/>M: (513) 439-0082 >> LinkedIn <http://www.linkedin.com/company/avalon-consulting-llc> | >> Google+ <http://www.google.com/+AvalonConsultingLLC> | Twitter >> <https://twitter.com/avalonconsult> >> >> ------------------------------------------------------------------------------------------------------------- >> This message (including any attachments) contains confidential >> information >> intended for a specific individual and purpose, and is protected by law. >> If >> you are not the intended recipient, you should delete this message. Any >> disclosure, copying, or distribution of this message, or the taking of >> any >> action based on it, is strictly prohibited. >> > > > -- > -- > simon elliston ball > @sireb >
