And this is not just about Ignite, I can't install anything else either ...
--David On Sat, Mar 19, 2016 at 2:11 PM, David Starina <[email protected]> wrote: > Guys, what am I (still) doing wrong? I keep getting this error: > Error: Execution of '/bin/yum -d 0 -e 0 -y list jdk' returned 1: Error: No > matching Packages to list > > This probably means I am not installing Java in a way Bigtop is expecting > it. Do you have some idea what I should be doing differently? Of course, > installing Oracle JDK manually won't work, because it won't be listed in > yum. On the other hand, when I install OpenJDK via yum, "list jdk" doesn't > list any package either. > > What am I missing? Should I be adding some other repo to yum beforehand? > > --David > > On Thu, Mar 17, 2016 at 7:52 PM, David Starina <[email protected]> > wrote: > >> Some more problems ... Tried this on CentOS 6.6 now ... >> >> 1) this line in site.yaml: "bigtop::jdk_package_name: >> "openjdk-7-jre-headless"" - not OK, you get the error: >> >> >> >> *Error: Execution of '/usr/bin/yum -d 0 -e 0 -y list >> openjdk-7-jre-headless' returned 1: Error: No matching Packages to >> listError: /Stage[main]/Main/Package[openjdk-7-jre-headless]/ensure: change >> from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y list >> openjdk-7-jre-headless' returned 1: Error: No matching Packages to list* >> 2) so, I installed OpenJDK manually ... Got the following error: >> >> >> >> >> >> >> *Error: Could not start Service[ignite-hadoop-service]: Execution of >> '/sbin/service ignite-hadoop-service start' returned 1: >> ignite-hadoop-service: unrecognized serviceError: >> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/Service[ignite-hadoop-service]/ensure: >> change from stopped to running failed: Could not start >> Service[ignite-hadoop-service]: Execution of '/sbin/service >> ignite-hadoop-service start' returned 1: ignite-hadoop-service: >> unrecognized serviceDebug: Executing '/bin/rpm -q jdk --nosignature >> --nodigest --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} >> %{ARCH}\n'Debug: Executing '/usr/bin/yum -d 0 -e 0 -y list jdk'Error: >> Execution of '/usr/bin/yum -d 0 -e 0 -y list jdk' returned 1: Error: No >> matching Packages to listError: /Stage[main]/Main/Package[jdk]/ensure: >> change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 >> -y list jdk' returned 1: Error: No matching Packages to listNotice: >> /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Dependency >> Package[jdk] has failures: true* >> ..... >> >> >> So, '*/usr/bin/yum -d 0 -e 0 -y list jdk*' doesn't return anything ... >> this is how Java is listed when I run 'yum list installed': >> java-1.8.0-openjdk.x86_64 >> java-1.8.0-openjdk-headless.x86_64 >> >> >> So, still can't install Ignite for now ... >> >> --David >> >> On Thu, Mar 17, 2016 at 12:46 PM, David Starina <[email protected]> >> wrote: >> >>> Konstantin, >>> >>> So I should be safer using CentOS 6? >>> >>> Also - was I right to include "hadoop" in site.yaml components? It's not >>> listed in the basic site.yaml, but I am guessing it is not installed >>> automatically? >>> >>> -David >>> >>> On Thu, Mar 17, 2016 at 7:17 AM, David Starina <[email protected]> >>> wrote: >>> >>>> That's right, I'm using CentOS 7. >>>> >>>> --David >>>> >>>> >>>> On Thursday, 17 March 2016, Konstantin Boudnik <[email protected]> wrote: >>>> >>>>> Hmm... that's plain nuts... Looks like latest ignite is pulling in >>>>> some crap >>>>> like osgi... OMG, WTF? >>>>> >>>>> Thanks for discovering this David! Somehow it never got stepped upon >>>>> in my >>>>> tests. Are you on centos7? >>>>> >>>>> Cos >>>>> >>>>> On Wed, Mar 16, 2016 at 11:39PM, David Starina wrote: >>>>> > Hi, >>>>> > >>>>> > Roman, Evans, thank you for the tips - that helped a lot. Still have >>>>> some >>>>> > problems though, an error when running puppet apply ... >>>>> > >>>>> > This is a sample of errors I got: >>>>> > >>>>> > >>>>> > *Debug: Executing '/bin/yum -d 0 -e 0 -y install >>>>> > ignite-hadoop-service'Error: Could not update: Execution of >>>>> '/bin/yum -d 0 >>>>> > -e 0 -y install ignite-hadoop-service' returned 1: Error: Package: >>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>>> Requires: >>>>> > osgi(org.apache.servicemix.bundles.spring-beans)Error: Package: >>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>>> Requires: >>>>> > osgi(org.apache.servicemix.bundles.spring-context-support)Error: >>>>> Package: >>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>>> Requires: >>>>> > osgi(org.apache.servicemix.bundles.spring-jdbc)Error: Package: >>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>>> Requires: >>>>> > osgi(org.apache.servicemix.bundles.spring-core)* >>>>> > ..... >>>>> > >>>>> > Also, after that: >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > *Notice: >>>>> > >>>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]: >>>>> > Dependency Package[ignite-hadoop] has failures: trueWarning: >>>>> > >>>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]: >>>>> > Skipping because of failed dependenciesNotice: >>>>> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: >>>>> Dependency >>>>> > Package[openjdk-7-jre-headless] has failures: trueWarning: >>>>> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Skipping >>>>> > because of failed dependenciesNotice: >>>>> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]: >>>>> > Dependency Package[openjdk-7-jre-headless] has failures: trueWarning: >>>>> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]: >>>>> Skipping >>>>> > because of failed dependenciesNotice: >>>>> > >>>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/ignite-hadoop/conf/default-config.xml]: >>>>> > Dependency Package[ignite-hadoop] has failures: true* >>>>> > ..... >>>>> > >>>>> > >>>>> > This is the "site.yaml" I used ("hadoop" wasn't listed in >>>>> components, but I >>>>> > think it should be there?) >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > *bigtop::hadoop_head_node: "NameNode"hadoop::hadoop_storage_dirs: - >>>>> > /data/1hadoop_cluster_node::cluster_components: - hadoop - >>>>> ignite_hadoop >>>>> > - hue - mahout - yarn - zookeeperbigtop::jdk_package_name: >>>>> > "openjdk-7-jre-headless"bigtop::bigtop_repo_uri: >>>>> > "http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64 >>>>> > <http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64 >>>>> >"* >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > Any idea what I am doing wrong? >>>>> > >>>>> > --David >>>>> > >>>>> > On Tue, Mar 15, 2016 at 6:57 PM, Evans Ye <[email protected]> >>>>> wrote: >>>>> > >>>>> > > David >>>>> > > >>>>> > > Most of the time you need to run same command on all nodes. "pdsh" >>>>> would >>>>> > > be a good tool to use. >>>>> > > >>>>> > > 1. You need to have puppet agent installed on every node. yum -y >>>>> install >>>>> > > puppet will do the trick, or you need to download repo, or use >>>>> > > bigtop/bigtop_toolchain/bin/puppetize.sh directly. There's no >>>>> deploy effort >>>>> > > other than just install the package. >>>>> > > >>>>> > > 2. Masterless means each puppet agent takes commands and runs >>>>> > > independently. We DONT use puppet master, which is a centralized >>>>> master >>>>> > > daemon to dispatch works to all the registered puppet agent. >>>>> > > >>>>> > > 3. Next, you need to copy bigtop puppet recipes and bigtop hiera >>>>> > > configuration to all nodes, or at least make them available for >>>>> all nodes >>>>> > > via nfs. >>>>> > > >>>>> > > 4. Finally, execute the puppet apply command on all nodes and the >>>>> puppet >>>>> > > recipes will automatically figure out it's role. There no need to >>>>> deploy >>>>> > > head node first and then slaves. >>>>> > > >>>>> > > Ask if you still have questions. >>>>> > > >>>>> > > >>>>> > > Thanks >>>>> > > >>>>> > > Evans >>>>> > > 2016年3月15日 下午4:10於 "David Starina" <[email protected]>寫道： >>>>> > > >>>>> > >> Hi guys, >>>>> > >> >>>>> > >> me again. I haven't set up Puppet before, can you just point me >>>>> in the >>>>> > >> right direction? I know this is not a Puppet mailing list, but I >>>>> bet you >>>>> > >> have some tips for me :-) >>>>> > >> >>>>> > >> So what do I need to do? Install Puppet on every node? What does >>>>> it mean >>>>> > >> that the mode of Puppet is "*masterless"* - do I just have to do >>>>> a basic >>>>> > >> installation of Puppet and not care much about setting it up, and >>>>> then >>>>> > >> deploy bigtop on every node separately? And when setting the >>>>> Hadoop head >>>>> > >> node - does that mean that I need to deploy Bigtop on this node >>>>> first, and >>>>> > >> Bigtop will know this is the master node by the url or IP I >>>>> provide to it? >>>>> > >> And then deploy Bigtop via Puppet on all the other nodes and they >>>>> will >>>>> > >> connect to the head node (which is the NameNode, I suppose?). >>>>> > >> >>>>> > >> I can do this via trial and error, but if you guys can help me >>>>> reduce the >>>>> > >> "error" part, that will be great :) Sorry for such beginner >>>>> questions, I >>>>> > >> haven't used Puppet before ... The reason I'm asking is, I got >>>>> access to a >>>>> > >> "physical" (actually virtual, but it's the same) environment for >>>>> a limited >>>>> > >> amount of time (two weeks), and I need to start testing on it >>>>> asap - and >>>>> > >> since my day job takes a lot of my time, if I wanted to do this >>>>> without >>>>> > >> help it could take me two weeks just to set up the environment - >>>>> but I need >>>>> > >> to install it, which is not the central part of my project, and >>>>> then run >>>>> > >> some ML algorithms on a text corpus, which will be the main part >>>>> of the >>>>> > >> project ... >>>>> > >> >>>>> > >> --David >>>>> > >> >>>>> > >> On Thu, Mar 10, 2016 at 11:28 AM, David Starina < >>>>> [email protected]> >>>>> > >> wrote: >>>>> > >> >>>>> > >>> Thanks for the info! >>>>> > >>> >>>>> > >>> --David >>>>> > >>> >>>>> > >>> On Tue, Mar 8, 2016 at 7:15 PM, Jay Vyas < >>>>> [email protected]> >>>>> > >>> wrote: >>>>> > >>> >>>>> > >>>> I think there is enough info on youtube that you can cobble >>>>> together a >>>>> > >>>> syllabus if you need structured learning. >>>>> > >>>> >>>>> > >>>> See cos/Romans Bigtop talks which are on YouTube and then my >>>>> Bigtop >>>>> > >>>> internals hackathon video. That gives you both the high and >>>>> low level >>>>> > >>>> details to do whatever you want. >>>>> > >>>> >>>>> > >>>> > On Mar 8, 2016, at 1:11 PM, Konstantin Boudnik < >>>>> [email protected]> >>>>> > >>>> wrote: >>>>> > >>>> > >>>>> > >>>> > We also had this discussion to start producing "The Big Book >>>>> of Apache >>>>> > >>>> > Bigtop" like HBase does, but never got into the >>>>> materialization part >>>>> > >>>> of that >>>>> > >>>> > project. >>>>> > >>>> > >>>>> > >>>> > Cos >>>>> > >>>> > >>>>> > >>>> >> On Tue, Mar 08, 2016 at 11:44PM, Evans Ye wrote: >>>>> > >>>> >> Hi David, >>>>> > >>>> >> >>>>> > >>>> >> If you just want to take Bigtop binaries to deploy a >>>>> cluster, you >>>>> > >>>> can refer >>>>> > >>>> >> to our puppet management recipes: >>>>> > >>>> >> >>>>> https://github.com/apache/bigtop/tree/master/bigtop-deploy/puppet >>>>> > >>>> >> >>>>> > >>>> >> BTW bigtop provisioner is the best source to reference for >>>>> how bigtop >>>>> > >>>> >> provision a cluster: >>>>> > >>>> >> >>>>> > >>>> >>>>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Provisioner+User+Guide >>>>> > >>>> >> >>>>> > >>>> >> Please ask us if you find anything confusing. >>>>> > >>>> >> >>>>> > >>>> >> >>>>> > >>>> >> Evans >>>>> > >>>> >> >>>>> > >>>> >> >>>>> > >>>> >> >>>>> > >>>> >> 2016-03-08 0:48 GMT+08:00 David Starina < >>>>> [email protected]>: >>>>> > >>>> >> >>>>> > >>>> >>> Hi, >>>>> > >>>> >>> >>>>> > >>>> >>> Is there some version of Bigtop User Guide available? I >>>>> know it >>>>> > >>>> used to be >>>>> > >>>> >>> on the be on http://bigtop.apache.org/ under the "User >>>>> Guide" >>>>> > >>>> link, but >>>>> > >>>> >>> there's nothing there anymore ... >>>>> > >>>> >>> >>>>> > >>>> >>> I need some guide on how to install Bigtop on a physical >>>>> cluster >>>>> > >>>> ... It >>>>> > >>>> >>> was easy to create a virtual cluster on my machine using >>>>> Vagrant, >>>>> > >>>> but I >>>>> > >>>> >>> cannot find a manual to install on a physical cluster ... >>>>> > >>>> >>> >>>>> > >>>> >>> Cheers, >>>>> > >>>> >>> David >>>>> > >>>> >>> >>>>> > >>>> >>>>> > >>> >>>>> > >>> >>>>> > >> >>>>> >>>> >>> >> >
