Guys, what am I (still) doing wrong? I keep getting this error: Error: Execution of '/bin/yum -d 0 -e 0 -y list jdk' returned 1: Error: No matching Packages to list
This probably means I am not installing Java in a way Bigtop is expecting it. Do you have some idea what I should be doing differently? Of course, installing Oracle JDK manually won't work, because it won't be listed in yum. On the other hand, when I install OpenJDK via yum, "list jdk" doesn't list any package either. What am I missing? Should I be adding some other repo to yum beforehand? --David On Thu, Mar 17, 2016 at 7:52 PM, David Starina <[email protected]> wrote: > Some more problems ... Tried this on CentOS 6.6 now ... > > 1) this line in site.yaml: "bigtop::jdk_package_name: > "openjdk-7-jre-headless"" - not OK, you get the error: > > > > *Error: Execution of '/usr/bin/yum -d 0 -e 0 -y list > openjdk-7-jre-headless' returned 1: Error: No matching Packages to > listError: /Stage[main]/Main/Package[openjdk-7-jre-headless]/ensure: change > from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y list > openjdk-7-jre-headless' returned 1: Error: No matching Packages to list* > 2) so, I installed OpenJDK manually ... Got the following error: > > > > > > > *Error: Could not start Service[ignite-hadoop-service]: Execution of > '/sbin/service ignite-hadoop-service start' returned 1: > ignite-hadoop-service: unrecognized serviceError: > /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/Service[ignite-hadoop-service]/ensure: > change from stopped to running failed: Could not start > Service[ignite-hadoop-service]: Execution of '/sbin/service > ignite-hadoop-service start' returned 1: ignite-hadoop-service: > unrecognized serviceDebug: Executing '/bin/rpm -q jdk --nosignature > --nodigest --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} > %{ARCH}\n'Debug: Executing '/usr/bin/yum -d 0 -e 0 -y list jdk'Error: > Execution of '/usr/bin/yum -d 0 -e 0 -y list jdk' returned 1: Error: No > matching Packages to listError: /Stage[main]/Main/Package[jdk]/ensure: > change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 > -y list jdk' returned 1: Error: No matching Packages to listNotice: > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Dependency > Package[jdk] has failures: true* > ..... > > > So, '*/usr/bin/yum -d 0 -e 0 -y list jdk*' doesn't return anything ... > this is how Java is listed when I run 'yum list installed': > java-1.8.0-openjdk.x86_64 > java-1.8.0-openjdk-headless.x86_64 > > > So, still can't install Ignite for now ... > > --David > > On Thu, Mar 17, 2016 at 12:46 PM, David Starina <[email protected]> > wrote: > >> Konstantin, >> >> So I should be safer using CentOS 6? >> >> Also - was I right to include "hadoop" in site.yaml components? It's not >> listed in the basic site.yaml, but I am guessing it is not installed >> automatically? >> >> -David >> >> On Thu, Mar 17, 2016 at 7:17 AM, David Starina <[email protected]> >> wrote: >> >>> That's right, I'm using CentOS 7. >>> >>> --David >>> >>> >>> On Thursday, 17 March 2016, Konstantin Boudnik <[email protected]> wrote: >>> >>>> Hmm... that's plain nuts... Looks like latest ignite is pulling in some >>>> crap >>>> like osgi... OMG, WTF? >>>> >>>> Thanks for discovering this David! Somehow it never got stepped upon in >>>> my >>>> tests. Are you on centos7? >>>> >>>> Cos >>>> >>>> On Wed, Mar 16, 2016 at 11:39PM, David Starina wrote: >>>> > Hi, >>>> > >>>> > Roman, Evans, thank you for the tips - that helped a lot. Still have >>>> some >>>> > problems though, an error when running puppet apply ... >>>> > >>>> > This is a sample of errors I got: >>>> > >>>> > >>>> > *Debug: Executing '/bin/yum -d 0 -e 0 -y install >>>> > ignite-hadoop-service'Error: Could not update: Execution of '/bin/yum >>>> -d 0 >>>> > -e 0 -y install ignite-hadoop-service' returned 1: Error: Package: >>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>> Requires: >>>> > osgi(org.apache.servicemix.bundles.spring-beans)Error: Package: >>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>> Requires: >>>> > osgi(org.apache.servicemix.bundles.spring-context-support)Error: >>>> Package: >>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>> Requires: >>>> > osgi(org.apache.servicemix.bundles.spring-jdbc)Error: Package: >>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) >>>> Requires: >>>> > osgi(org.apache.servicemix.bundles.spring-core)* >>>> > ..... >>>> > >>>> > Also, after that: >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > *Notice: >>>> > >>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]: >>>> > Dependency Package[ignite-hadoop] has failures: trueWarning: >>>> > >>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]: >>>> > Skipping because of failed dependenciesNotice: >>>> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: >>>> Dependency >>>> > Package[openjdk-7-jre-headless] has failures: trueWarning: >>>> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Skipping >>>> > because of failed dependenciesNotice: >>>> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]: >>>> > Dependency Package[openjdk-7-jre-headless] has failures: trueWarning: >>>> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]: >>>> Skipping >>>> > because of failed dependenciesNotice: >>>> > >>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/ignite-hadoop/conf/default-config.xml]: >>>> > Dependency Package[ignite-hadoop] has failures: true* >>>> > ..... >>>> > >>>> > >>>> > This is the "site.yaml" I used ("hadoop" wasn't listed in components, >>>> but I >>>> > think it should be there?) >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > *bigtop::hadoop_head_node: "NameNode"hadoop::hadoop_storage_dirs: - >>>> > /data/1hadoop_cluster_node::cluster_components: - hadoop - >>>> ignite_hadoop >>>> > - hue - mahout - yarn - zookeeperbigtop::jdk_package_name: >>>> > "openjdk-7-jre-headless"bigtop::bigtop_repo_uri: >>>> > "http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64 >>>> > <http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64 >>>> >"* >>>> > >>>> > >>>> > >>>> > >>>> > Any idea what I am doing wrong? >>>> > >>>> > --David >>>> > >>>> > On Tue, Mar 15, 2016 at 6:57 PM, Evans Ye <[email protected]> wrote: >>>> > >>>> > > David >>>> > > >>>> > > Most of the time you need to run same command on all nodes. "pdsh" >>>> would >>>> > > be a good tool to use. >>>> > > >>>> > > 1. You need to have puppet agent installed on every node. yum -y >>>> install >>>> > > puppet will do the trick, or you need to download repo, or use >>>> > > bigtop/bigtop_toolchain/bin/puppetize.sh directly. There's no >>>> deploy effort >>>> > > other than just install the package. >>>> > > >>>> > > 2. Masterless means each puppet agent takes commands and runs >>>> > > independently. We DONT use puppet master, which is a centralized >>>> master >>>> > > daemon to dispatch works to all the registered puppet agent. >>>> > > >>>> > > 3. Next, you need to copy bigtop puppet recipes and bigtop hiera >>>> > > configuration to all nodes, or at least make them available for all >>>> nodes >>>> > > via nfs. >>>> > > >>>> > > 4. Finally, execute the puppet apply command on all nodes and the >>>> puppet >>>> > > recipes will automatically figure out it's role. There no need to >>>> deploy >>>> > > head node first and then slaves. >>>> > > >>>> > > Ask if you still have questions. >>>> > > >>>> > > >>>> > > Thanks >>>> > > >>>> > > Evans >>>> > > 2016年3月15日 下午4:10於 "David Starina" <[email protected]>寫道： >>>> > > >>>> > >> Hi guys, >>>> > >> >>>> > >> me again. I haven't set up Puppet before, can you just point me in >>>> the >>>> > >> right direction? I know this is not a Puppet mailing list, but I >>>> bet you >>>> > >> have some tips for me :-) >>>> > >> >>>> > >> So what do I need to do? Install Puppet on every node? What does >>>> it mean >>>> > >> that the mode of Puppet is "*masterless"* - do I just have to do a >>>> basic >>>> > >> installation of Puppet and not care much about setting it up, and >>>> then >>>> > >> deploy bigtop on every node separately? And when setting the >>>> Hadoop head >>>> > >> node - does that mean that I need to deploy Bigtop on this node >>>> first, and >>>> > >> Bigtop will know this is the master node by the url or IP I >>>> provide to it? >>>> > >> And then deploy Bigtop via Puppet on all the other nodes and they >>>> will >>>> > >> connect to the head node (which is the NameNode, I suppose?). >>>> > >> >>>> > >> I can do this via trial and error, but if you guys can help me >>>> reduce the >>>> > >> "error" part, that will be great :) Sorry for such beginner >>>> questions, I >>>> > >> haven't used Puppet before ... The reason I'm asking is, I got >>>> access to a >>>> > >> "physical" (actually virtual, but it's the same) environment for a >>>> limited >>>> > >> amount of time (two weeks), and I need to start testing on it asap >>>> - and >>>> > >> since my day job takes a lot of my time, if I wanted to do this >>>> without >>>> > >> help it could take me two weeks just to set up the environment - >>>> but I need >>>> > >> to install it, which is not the central part of my project, and >>>> then run >>>> > >> some ML algorithms on a text corpus, which will be the main part >>>> of the >>>> > >> project ... >>>> > >> >>>> > >> --David >>>> > >> >>>> > >> On Thu, Mar 10, 2016 at 11:28 AM, David Starina < >>>> [email protected]> >>>> > >> wrote: >>>> > >> >>>> > >>> Thanks for the info! >>>> > >>> >>>> > >>> --David >>>> > >>> >>>> > >>> On Tue, Mar 8, 2016 at 7:15 PM, Jay Vyas < >>>> [email protected]> >>>> > >>> wrote: >>>> > >>> >>>> > >>>> I think there is enough info on youtube that you can cobble >>>> together a >>>> > >>>> syllabus if you need structured learning. >>>> > >>>> >>>> > >>>> See cos/Romans Bigtop talks which are on YouTube and then my >>>> Bigtop >>>> > >>>> internals hackathon video. That gives you both the high and >>>> low level >>>> > >>>> details to do whatever you want. >>>> > >>>> >>>> > >>>> > On Mar 8, 2016, at 1:11 PM, Konstantin Boudnik <[email protected] >>>> > >>>> > >>>> wrote: >>>> > >>>> > >>>> > >>>> > We also had this discussion to start producing "The Big Book >>>> of Apache >>>> > >>>> > Bigtop" like HBase does, but never got into the >>>> materialization part >>>> > >>>> of that >>>> > >>>> > project. >>>> > >>>> > >>>> > >>>> > Cos >>>> > >>>> > >>>> > >>>> >> On Tue, Mar 08, 2016 at 11:44PM, Evans Ye wrote: >>>> > >>>> >> Hi David, >>>> > >>>> >> >>>> > >>>> >> If you just want to take Bigtop binaries to deploy a cluster, >>>> you >>>> > >>>> can refer >>>> > >>>> >> to our puppet management recipes: >>>> > >>>> >> >>>> https://github.com/apache/bigtop/tree/master/bigtop-deploy/puppet >>>> > >>>> >> >>>> > >>>> >> BTW bigtop provisioner is the best source to reference for >>>> how bigtop >>>> > >>>> >> provision a cluster: >>>> > >>>> >> >>>> > >>>> >>>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Provisioner+User+Guide >>>> > >>>> >> >>>> > >>>> >> Please ask us if you find anything confusing. >>>> > >>>> >> >>>> > >>>> >> >>>> > >>>> >> Evans >>>> > >>>> >> >>>> > >>>> >> >>>> > >>>> >> >>>> > >>>> >> 2016-03-08 0:48 GMT+08:00 David Starina < >>>> [email protected]>: >>>> > >>>> >> >>>> > >>>> >>> Hi, >>>> > >>>> >>> >>>> > >>>> >>> Is there some version of Bigtop User Guide available? I know >>>> it >>>> > >>>> used to be >>>> > >>>> >>> on the be on http://bigtop.apache.org/ under the "User >>>> Guide" >>>> > >>>> link, but >>>> > >>>> >>> there's nothing there anymore ... >>>> > >>>> >>> >>>> > >>>> >>> I need some guide on how to install Bigtop on a physical >>>> cluster >>>> > >>>> ... It >>>> > >>>> >>> was easy to create a virtual cluster on my machine using >>>> Vagrant, >>>> > >>>> but I >>>> > >>>> >>> cannot find a manual to install on a physical cluster ... >>>> > >>>> >>> >>>> > >>>> >>> Cheers, >>>> > >>>> >>> David >>>> > >>>> >>> >>>> > >>>> >>>> > >>> >>>> > >>> >>>> > >> >>>> >>> >> >
