Sorry guys, Turn out I was leaving out JDK dependancy from site.yaml. So this went through ok now. Thanks for all the help!
--David On Sat, Mar 19, 2016 at 2:12 PM, David Starina <[email protected]> wrote: > And this is not just about Ignite, I can't install anything else either ... > > > --David > > On Sat, Mar 19, 2016 at 2:11 PM, David Starina <[email protected]> > wrote: > >> Guys, what am I (still) doing wrong? I keep getting this error: >> Error: Execution of '/bin/yum -d 0 -e 0 -y list jdk' returned 1: Error: >> No matching Packages to list >> >> This probably means I am not installing Java in a way Bigtop is expecting >> it. Do you have some idea what I should be doing differently? Of course, >> installing Oracle JDK manually won't work, because it won't be listed in >> yum. On the other hand, when I install OpenJDK via yum, "list jdk" doesn't >> list any package either. >> >> What am I missing? Should I be adding some other repo to yum beforehand? >> >> --David >> >> On Thu, Mar 17, 2016 at 7:52 PM, David Starina <[email protected]> >> wrote: >> >>> Some more problems ... Tried this on CentOS 6.6 now ... >>> >>> 1) this line in site.yaml: "bigtop::jdk_package_name: >>> "openjdk-7-jre-headless"" - not OK, you get the error: >>> >>> >>> >>> *Error: Execution of '/usr/bin/yum -d 0 -e 0 -y list >>> openjdk-7-jre-headless' returned 1: Error: No matching Packages to >>> listError: /Stage[main]/Main/Package[openjdk-7-jre-headless]/ensure: change >>> from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y list >>> openjdk-7-jre-headless' returned 1: Error: No matching Packages to list* >>> 2) so, I installed OpenJDK manually ... Got the following error: >>> >>> >>> >>> >>> >>> >>> *Error: Could not start Service[ignite-hadoop-service]: Execution of >>> '/sbin/service ignite-hadoop-service start' returned 1: >>> ignite-hadoop-service: unrecognized serviceError: >>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/Service[ignite-hadoop-service]/ensure: >>> change from stopped to running failed: Could not start >>> Service[ignite-hadoop-service]: Execution of '/sbin/service >>> ignite-hadoop-service start' returned 1: ignite-hadoop-service: >>> unrecognized serviceDebug: Executing '/bin/rpm -q jdk --nosignature >>> --nodigest --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} >>> %{ARCH}\n'Debug: Executing '/usr/bin/yum -d 0 -e 0 -y list jdk'Error: >>> Execution of '/usr/bin/yum -d 0 -e 0 -y list jdk' returned 1: Error: No >>> matching Packages to listError: /Stage[main]/Main/Package[jdk]/ensure: >>> change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 >>> -y list jdk' returned 1: Error: No matching Packages to listNotice: >>> /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Dependency >>> Package[jdk] has failures: true* >>> ..... >>> >>> >>> So, '*/usr/bin/yum -d 0 -e 0 -y list jdk*' doesn't return anything ... >>> this is how Java is listed when I run 'yum list installed': >>> java-1.8.0-openjdk.x86_64 >>> java-1.8.0-openjdk-headless.x86_64 >>> >>> >>> So, still can't install Ignite for now ... >>> >>> --David >>> >>> On Thu, Mar 17, 2016 at 12:46 PM, David Starina <[email protected] >>> > wrote: >>> >>>> Konstantin, >>>> >>>> So I should be safer using CentOS 6? >>>> >>>> Also - was I right to include "hadoop" in site.yaml components? It's >>>> not listed in the basic site.yaml, but I am guessing it is not installed >>>> automatically? >>>> >>>> -David >>>> >>>> On Thu, Mar 17, 2016 at 7:17 AM, David Starina <[email protected] >>>> > wrote: >>>> >>>>> That's right, I'm using CentOS 7. >>>>> >>>>> --David >>>>> >>>>> >>>>> On Thursday, 17 March 2016, Konstantin Boudnik <[email protected]> wrote: >>>>> >>>>>> Hmm... that's plain nuts... Looks like latest ignite is pulling in >>>>>> some crap >>>>>> like osgi... OMG, WTF? >>>>>> >>>>>> Thanks for discovering this David! Somehow it never got stepped upon >>>>>> in my >>>>>> tests. Are you on centos7? >>>>>> >>>>>> Cos >>>>>> >>>>>> On Wed, Mar 16, 2016 at 11:39PM, David Starina wrote: >>>>>> > Hi, >>>>>> > >>>>>> > Roman, Evans, thank you for the tips - that helped a lot. Still >>>>>> have some >>>>>> > problems though, an error when running puppet apply ... >>>>>> > >>>>>> > This is a sample of errors I got: >>>>>> > >>>>>> > >>>>>> > *Debug: Executing '/bin/yum -d 0 -e 0 -y install >>>>>> > ignite-hadoop-service'Error: Could not update: Execution of >>>>>> '/bin/yum -d 0 >>>>>> > -e 0 -y install ignite-hadoop-service' returned 1: Error: Package: >>>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>>>> Requires: >>>>>> > osgi(org.apache.servicemix.bundles.spring-beans)Error: Package: >>>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>>>> Requires: >>>>>> > osgi(org.apache.servicemix.bundles.spring-context-support)Error: >>>>>> Package: >>>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>>>> Requires: >>>>>> > osgi(org.apache.servicemix.bundles.spring-jdbc)Error: Package: >>>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>>>> Requires: >>>>>> > osgi(org.apache.servicemix.bundles.spring-core)* >>>>>> > ..... >>>>>> > >>>>>> > Also, after that: >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > *Notice: >>>>>> > >>>>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]: >>>>>> > Dependency Package[ignite-hadoop] has failures: trueWarning: >>>>>> > >>>>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]: >>>>>> > Skipping because of failed dependenciesNotice: >>>>>> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: >>>>>> Dependency >>>>>> > Package[openjdk-7-jre-headless] has failures: trueWarning: >>>>>> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: >>>>>> Skipping >>>>>> > because of failed dependenciesNotice: >>>>>> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]: >>>>>> > Dependency Package[openjdk-7-jre-headless] has failures: >>>>>> trueWarning: >>>>>> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]: >>>>>> Skipping >>>>>> > because of failed dependenciesNotice: >>>>>> > >>>>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/ignite-hadoop/conf/default-config.xml]: >>>>>> > Dependency Package[ignite-hadoop] has failures: true* >>>>>> > ..... >>>>>> > >>>>>> > >>>>>> > This is the "site.yaml" I used ("hadoop" wasn't listed in >>>>>> components, but I >>>>>> > think it should be there?) >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > *bigtop::hadoop_head_node: "NameNode"hadoop::hadoop_storage_dirs: - >>>>>> > /data/1hadoop_cluster_node::cluster_components: - hadoop - >>>>>> ignite_hadoop >>>>>> > - hue - mahout - yarn - zookeeperbigtop::jdk_package_name: >>>>>> > "openjdk-7-jre-headless"bigtop::bigtop_repo_uri: >>>>>> > " >>>>>> http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64 >>>>>> > < >>>>>> http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64 >>>>>> >"* >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > Any idea what I am doing wrong? >>>>>> > >>>>>> > --David >>>>>> > >>>>>> > On Tue, Mar 15, 2016 at 6:57 PM, Evans Ye <[email protected]> >>>>>> wrote: >>>>>> > >>>>>> > > David >>>>>> > > >>>>>> > > Most of the time you need to run same command on all nodes. >>>>>> "pdsh" would >>>>>> > > be a good tool to use. >>>>>> > > >>>>>> > > 1. You need to have puppet agent installed on every node. yum -y >>>>>> install >>>>>> > > puppet will do the trick, or you need to download repo, or use >>>>>> > > bigtop/bigtop_toolchain/bin/puppetize.sh directly. There's no >>>>>> deploy effort >>>>>> > > other than just install the package. >>>>>> > > >>>>>> > > 2. Masterless means each puppet agent takes commands and runs >>>>>> > > independently. We DONT use puppet master, which is a centralized >>>>>> master >>>>>> > > daemon to dispatch works to all the registered puppet agent. >>>>>> > > >>>>>> > > 3. Next, you need to copy bigtop puppet recipes and bigtop hiera >>>>>> > > configuration to all nodes, or at least make them available for >>>>>> all nodes >>>>>> > > via nfs. >>>>>> > > >>>>>> > > 4. Finally, execute the puppet apply command on all nodes and the >>>>>> puppet >>>>>> > > recipes will automatically figure out it's role. There no need to >>>>>> deploy >>>>>> > > head node first and then slaves. >>>>>> > > >>>>>> > > Ask if you still have questions. >>>>>> > > >>>>>> > > >>>>>> > > Thanks >>>>>> > > >>>>>> > > Evans >>>>>> > > 2016年3月15日 下午4:10於 "David Starina" <[email protected]>寫道： >>>>>> > > >>>>>> > >> Hi guys, >>>>>> > >> >>>>>> > >> me again. I haven't set up Puppet before, can you just point me >>>>>> in the >>>>>> > >> right direction? I know this is not a Puppet mailing list, but I >>>>>> bet you >>>>>> > >> have some tips for me :-) >>>>>> > >> >>>>>> > >> So what do I need to do? Install Puppet on every node? What does >>>>>> it mean >>>>>> > >> that the mode of Puppet is "*masterless"* - do I just have to do >>>>>> a basic >>>>>> > >> installation of Puppet and not care much about setting it up, >>>>>> and then >>>>>> > >> deploy bigtop on every node separately? And when setting the >>>>>> Hadoop head >>>>>> > >> node - does that mean that I need to deploy Bigtop on this node >>>>>> first, and >>>>>> > >> Bigtop will know this is the master node by the url or IP I >>>>>> provide to it? >>>>>> > >> And then deploy Bigtop via Puppet on all the other nodes and >>>>>> they will >>>>>> > >> connect to the head node (which is the NameNode, I suppose?). >>>>>> > >> >>>>>> > >> I can do this via trial and error, but if you guys can help me >>>>>> reduce the >>>>>> > >> "error" part, that will be great :) Sorry for such beginner >>>>>> questions, I >>>>>> > >> haven't used Puppet before ... The reason I'm asking is, I got >>>>>> access to a >>>>>> > >> "physical" (actually virtual, but it's the same) environment for >>>>>> a limited >>>>>> > >> amount of time (two weeks), and I need to start testing on it >>>>>> asap - and >>>>>> > >> since my day job takes a lot of my time, if I wanted to do this >>>>>> without >>>>>> > >> help it could take me two weeks just to set up the environment - >>>>>> but I need >>>>>> > >> to install it, which is not the central part of my project, and >>>>>> then run >>>>>> > >> some ML algorithms on a text corpus, which will be the main part >>>>>> of the >>>>>> > >> project ... >>>>>> > >> >>>>>> > >> --David >>>>>> > >> >>>>>> > >> On Thu, Mar 10, 2016 at 11:28 AM, David Starina < >>>>>> [email protected]> >>>>>> > >> wrote: >>>>>> > >> >>>>>> > >>> Thanks for the info! >>>>>> > >>> >>>>>> > >>> --David >>>>>> > >>> >>>>>> > >>> On Tue, Mar 8, 2016 at 7:15 PM, Jay Vyas < >>>>>> [email protected]> >>>>>> > >>> wrote: >>>>>> > >>> >>>>>> > >>>> I think there is enough info on youtube that you can cobble >>>>>> together a >>>>>> > >>>> syllabus if you need structured learning. >>>>>> > >>>> >>>>>> > >>>> See cos/Romans Bigtop talks which are on YouTube and then my >>>>>> Bigtop >>>>>> > >>>> internals hackathon video. That gives you both the high and >>>>>> low level >>>>>> > >>>> details to do whatever you want. >>>>>> > >>>> >>>>>> > >>>> > On Mar 8, 2016, at 1:11 PM, Konstantin Boudnik < >>>>>> [email protected]> >>>>>> > >>>> wrote: >>>>>> > >>>> > >>>>>> > >>>> > We also had this discussion to start producing "The Big Book >>>>>> of Apache >>>>>> > >>>> > Bigtop" like HBase does, but never got into the >>>>>> materialization part >>>>>> > >>>> of that >>>>>> > >>>> > project. >>>>>> > >>>> > >>>>>> > >>>> > Cos >>>>>> > >>>> > >>>>>> > >>>> >> On Tue, Mar 08, 2016 at 11:44PM, Evans Ye wrote: >>>>>> > >>>> >> Hi David, >>>>>> > >>>> >> >>>>>> > >>>> >> If you just want to take Bigtop binaries to deploy a >>>>>> cluster, you >>>>>> > >>>> can refer >>>>>> > >>>> >> to our puppet management recipes: >>>>>> > >>>> >> >>>>>> https://github.com/apache/bigtop/tree/master/bigtop-deploy/puppet >>>>>> > >>>> >> >>>>>> > >>>> >> BTW bigtop provisioner is the best source to reference for >>>>>> how bigtop >>>>>> > >>>> >> provision a cluster: >>>>>> > >>>> >> >>>>>> > >>>> >>>>>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Provisioner+User+Guide >>>>>> > >>>> >> >>>>>> > >>>> >> Please ask us if you find anything confusing. >>>>>> > >>>> >> >>>>>> > >>>> >> >>>>>> > >>>> >> Evans >>>>>> > >>>> >> >>>>>> > >>>> >> >>>>>> > >>>> >> >>>>>> > >>>> >> 2016-03-08 0:48 GMT+08:00 David Starina < >>>>>> [email protected]>: >>>>>> > >>>> >> >>>>>> > >>>> >>> Hi, >>>>>> > >>>> >>> >>>>>> > >>>> >>> Is there some version of Bigtop User Guide available? I >>>>>> know it >>>>>> > >>>> used to be >>>>>> > >>>> >>> on the be on http://bigtop.apache.org/ under the "User >>>>>> Guide" >>>>>> > >>>> link, but >>>>>> > >>>> >>> there's nothing there anymore ... >>>>>> > >>>> >>> >>>>>> > >>>> >>> I need some guide on how to install Bigtop on a physical >>>>>> cluster >>>>>> > >>>> ... It >>>>>> > >>>> >>> was easy to create a virtual cluster on my machine using >>>>>> Vagrant, >>>>>> > >>>> but I >>>>>> > >>>> >>> cannot find a manual to install on a physical cluster ... >>>>>> > >>>> >>> >>>>>> > >>>> >>> Cheers, >>>>>> > >>>> >>> David >>>>>> > >>>> >>> >>>>>> > >>>> >>>>>> > >>> >>>>>> > >>> >>>>>> > >> >>>>>> >>>>> >>>> >>> >> >
