Hello, Who can provide me with an access to add new page in this Installation space below? https://cwiki.apache.org/confluence/display/METRON/Installation I want to add up-to-date article on how-to install Metron using Ambari mpack. Please advise.
My username is "dima" registered with electronic mail dimdroll hosted by gmail. Right now I have it posted here: https://goo.gl/HWGwpj But instead of maintaining it there it will be convenient for me to have it in confluence instead of gdocs as I am using confluence in our internal project as well, so I can basically copy formatted doc between each other. Thank you. - Dima On 11/30/2016 04:57 PM, [email protected]<mailto:[email protected]> wrote: Probably the best long term place for this would be here<https://cwiki.apache.org/confluence/display/METRON/Installation>, potentially with a new mpack article (to differentiate from the ambari install which is already there, but dated). I can help with migrating it to the wiki - I will run through your documentation later today and request edit permissions if I find anything worth updating. Would you be OK if I put your doc in the wiki at some point? Jon On Wed, Nov 30, 2016 at 9:51 AM Dima Kovalyov <[email protected]<mailto:[email protected]>> wrote: As multiple users asked I have published document for installing Metron on Centos 6 using Ambari 2.4 with HDP 2.5: https://goo.gl/HWGwpj It may look ugly as this is direct export from internal confluence with cosmetic changes and our internal details removed. Let me know if there is a better place to publish it. Feel free to request edit permissions, I would like the document to be up-to-date, simplified and corrected if there are any errors. The goal for this document is to provide installation guidance in a manner that Linux administrator with basic knowledge could perform installation without opening any other links or searching for commands, step by step. Thank you. p.s. feel free to let me know if you will encounter any errors as I've seen them bunch. - Dima On 11/30/2016 04:45 PM, [email protected]<mailto:[email protected]> wrote: I also have written up a method to retrieve and setup the required packages in my script<https://github.com/JonZeolla/Development/blob/master/Bash/setupMetron.sh> which could probably be cleaned up and incorporated. Jon On Wed, Nov 30, 2016 at 9:24 AM Nick Allen <[email protected]<mailto:[email protected]>> wrote: In regards to your comment about a "full package version dump" there is at least a start at `metron-deployment/scripts/platform-info.sh`. If we can enhance that, let's do it. On Wed, Nov 30, 2016 at 4:04 AM, shoggi <[email protected]<mailto:[email protected]>> wrote: Hi Just to wrap this up. I've tried to deploy it on HDP 2.5 but I got the exact same problem. What I did was removing all components for 2.4.3.0 and then created a new cluster based on 2.5. I left it at that as the most likely issue could be with my base install. May that be wrong version of something or even a missing dev package. As my work also coincided with the announcement by Casey I went ahead and installed 0.3.0 as a dev installation instead. That worked great as the dev-full-platform guide is very complete. As a general feedback, in order to get more people interested and not having noobs like me come up with redundant questions - At this stage also sorry for not providing a meaningful subject, I simply was too excited :) There is much to win with a small extension of the installation guide. I don't think it needs much more prose around but a full package version dump with every new release would help greatly to get things going. For the things which need to be compiled in, the versioning information should be enough. I am saying this, as everyone starts with a different base system. Some have the chance to start from scratch, some have a system running and build something on top and then there are the experienced Metron users, who have many base requirements.. they tend to forget. And I believe at the end it always fails just because of some very tiny small difference. If there is a space somewhere available, this could even be a spot, where people can say, "hey, I got this running on XYZ and here is the package version dump". A simple table with 'Metron version', 'OS release', 'Package list file', 'Version Details of every other needed to-be-compiled package', is already a big help. This really could supplement the existing guide found here: https://cwiki.apache.org/confluence/display/METRON/Metron+Installation+on+an+Ambari-Managed+Cluster. But then, I could be alone and others did not experience the same and simply got it to run, I don't know. Anyway, I would be happy to help if help is needed. Cheers Shoggi On Sun, Nov 27, 2016 at 3:00 PM, shoggi <[email protected]<mailto:[email protected]>> wrote: Hi Simon Thank you for your very quick response. I installed it on HDP 2.4.3.0. I had a 0.2.0 instance running and was very conscious about versioning the various components. Somehow I had the understanding of not touching 2.5 as of yet. I give that a try. Thanks heaps ! Shoggi On Sun, Nov 27, 2016 at 2:52 PM, Simon Elliston Ball <[email protected]<mailto:[email protected]>> wrote: Hi Shoggi, Can I ask which version of HDP you used as a base. The article is a little out of date in that is calls for 2.4, which the 0.3.0 release is based on 2.5. There was a major version difference in storm between the two, which seems likely to have caused the problem you're seeing. Simon Sent from my iPad On 27 Nov 2016, at 13:48, shoggi <[email protected]<mailto:[email protected]>> wrote: Hi all I followed the installation guide to deploy Metron via Ambari https://community.hortonworks.com/articles/60805/deploying-a-fresh-metron-cluster-using-ambari-serv.html. I ended up with all but the Metron Components (Enrichment, Parsing, Parsers) to start up. The closest to pinpoint the issue is, when I try to start any new parser topology such as with bro: /usr/metron/0.3.0/bin/start_parser_topology.sh -k x.localdomain:6667 -z x.localdomain:2181 -s bro The error I get is: Exception in thread "main" java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: org/apache/storm/Config Or in more details: Running: /usr/jdk64/jdk1.8.0_77/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.4.3.0-227/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.4.3.0-227/storm/lib/asm-4.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/cheshire-5.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-stacktrace-0.2.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/clj-time-0.8.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/clojure-1.6.0.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/clout-1.0.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/compojure-1.1.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/disruptor-2.10.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/hadoop-auth-2.7.1.2.4.3.0-227.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/jackson-core-2.3.1.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/jackson-dataformat-smile-2.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.4.3.0-227/storm/lib/jline-0.9.94.jar:/usr/hdp/2.4.3.0-227/storm/lib/kryo-2.21.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/log4j-api-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-core-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.4.3.0-227/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/minlog-1.2.jar:/usr/hdp/2.4.3.<http://2.4.3.>0-227/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.4.3.0-227/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-json-0.3.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.4.3.0-227/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.4.3.0-227/storm/lib/slf4j-api-1.7.7.jar:/usr/hdp/2.4.3.0-227/storm/lib/storm-core-0.10.0.2.4.3.0-227.jar:/usr/hdp/2.4.3.0-227/storm/lib/tigris-0.1.1.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.4.3.0-227/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.4.3.0-227/storm/lib/zookeeper.jar:/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.4.3.0-227/storm/bin -Dstorm.jar=/usr/metron/0.3.0/lib/metron-parsers-0.3.0-uber.jar org.apache.metron.parsers.topology.ParserTopologyCLI -k x.localdomain:6667 -z x.localdomain:2181 -s bro Exception in thread "main" java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: org/apache/storm/Config at org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<init>(ParserTopologyCLI.java:199) at org.apache.metron.parsers.topology.ParserTopologyCLI$ParserOptions.<clinit>(ParserTopologyCLI.java:47) at org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:266) Caused by: java.lang.NoClassDefFoundError: org/apache/storm/Config ... 3 more Caused by: java.lang.ClassNotFoundException: org.apache.storm.Config at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 3 more Must be something wrong with my config, possibly I have overlooked something but can't put my finger on it. Any help is appreciated. Many thanks Shoggi -- Nick Allen <[email protected]<mailto:[email protected]>> -- Jon Sent from my mobile device -- Jon Sent from my mobile device
