Otto, Yes, document is based on this article: https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html and uses Ambari mpack to install it directly from Ambari. Less configuration on the Metron side, no ansible routine and more details (more user friendly).
Since multiple users requested it I will formalize it and publish to google docs in few hours. - Dima On 11/30/2016 03:13 PM, Otto Fowler wrote: Is this different from the how to install on an ambari managed cluster document we have in the wiki? That is what I use to do bare metal installs. On November 30, 2016 at 07:12:47, Dima Kovalyov (dima.koval...@sstech.us<mailto:dima.koval...@sstech.us>) wrote: Hello, I have a step by step document on how to install Metron from sources on bare-metal Centos 6. If someone could point me to appropriate place for publishing I would gladly do it. - Dima On 11/30/2016 11:05 AM, shoggi wrote: Hi Just to wrap this up. I've tried to deploy it on HDP 2.5 but I got the exact same problem. What I did was removing all components for 2.4.3.0 and then created a new cluster based on 2.5. I left it at that as the most likely issue could be with my base install. May that be wrong version of something or even a missing dev package. As my work also coincided with the announcement by Casey I went ahead and installed 0.3.0 as a dev installation instead. That worked great as the dev-full-platform guide is very complete. As a general feedback, in order to get more people interested and not having noobs like me come up with redundant questions - At this stage also sorry for not providing a meaningful subject, I simply was too excited :) There is much to win with a small extension of the installation guide. I don't think it needs much more prose around but a full package version dump with every new release would help greatly to get things going. For the things which need to be compiled in, the versioning information should be enough. I am saying this, as everyone starts with a different base system. Some have the chance to start from scratch, some have a system running and build something on top and then there are the experienced Metron users, who have many base requirements.. they tend to forget. And I believe at the end it always fails just because of some very tiny small difference. If there is a space somewhere available, this could even be a spot, where people can say, "hey, I got this running on XYZ and here is the package version dump". A simple table with 'Metron version', 'OS release', 'Package list file', 'Version Details of every other needed to-be-compiled package', is already a big help. This really could supplement the existing guide found here: https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+an+Ambari-Managed+Cluster. But then, I could be alone and others did not experience the same and simply got it to run, I don't know. Anyway, I would be happy to help if help is needed. Cheers Shoggi On Sun, Nov 27, 2016 at 3:00 PM, shoggi <sho...@gmail.com<mailto:sho...@gmail.com>> wrote: Hi Simon Thank you for your very quick response. I installed it on HDP 2.4.3.0. I had a 0.2.0 instance running and was very conscious about versioning the various components. Somehow I had the understanding of not touching 2.5 as of yet. I give that a try. Thanks heaps ! Shoggi On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <si...@simonellistonball.com<mailto:si...@simonellistonball.com>> wrote: Hi Shoggi, Can I ask which version of HDP you used as a base. The article is a little out of date in that is calls for 2.4, which the 0.3.0 release is based on 2.5. There was a major version difference in storm between the two, which seems likely to have caused the problem you're seeing. Simon Sent from my iPad On 27 Nov 2016, at 13:48, shoggi <sho...@gmail.com<mailto:sho...@gmail.com>> wrote: Hi all I followed the installation guide to deploy Metron via Ambari https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html. I ended up with all but the Metron Components (Enrichment, Parsing, Parsers) to start up. The closest to pinpoint the issue is, when I try to start any new parser topology such as with bro: /usr/metron/0.3.0/bin/start_parser_topology.sh -k x.localdomain:6667 -z x.localdomain:2181 -s bro The error I get is: Exception in thread "main" java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: org/apache/storm/Config Or in more details: Running: /usr/jdk64/jdk1.8.0_77/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.4.3.0-227/storm/bin -Dstorm.jar=/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar org.apache.metron.parsers.topology.ParserTopologyCLI -k x.localdomain:6667 -z x.localdomain:2181 -s bro Exception in thread "main" java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: org/apache/storm/Config at org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<init>(ParserTopologyCLI.java:199) at org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<clinit>(ParserTopologyCLI.java:47) at org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:266) Caused by: java.lang.NoClassDefFoundError: org/apache/storm/Config ... 3 more Caused by: java.lang.ClassNotFoundException: org.apache.storm.Config at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 3 more Must be something wrong with my config, possibly I have overlooked something but can't put my finger on it. Any help is appreciated. Many thanks Shoggi
