On Thu, Mar 17, 2016 at 12:46PM, David Starina wrote: > Konstantin, > > So I should be safer using CentOS 6?
No, this isn't what I am saying. Centos7 is fine, it seems to be an issue with the Ignite's internal dependencies somehow... I will try to find some time over the weekend to check it out. > Also - was I right to include "hadoop" in site.yaml components? It's not > listed in the basic site.yaml, but I am guessing it is not installed > automatically? If it gives you what you want - why not? ;) Cos > -David > > On Thu, Mar 17, 2016 at 7:17 AM, David Starina <[email protected]> > wrote: > > > That's right, I'm using CentOS 7. > > > > --David > > > > > > On Thursday, 17 March 2016, Konstantin Boudnik <[email protected]> wrote: > > > >> Hmm... that's plain nuts... Looks like latest ignite is pulling in some > >> crap > >> like osgi... OMG, WTF? > >> > >> Thanks for discovering this David! Somehow it never got stepped upon in my > >> tests. Are you on centos7? > >> > >> Cos > >> > >> On Wed, Mar 16, 2016 at 11:39PM, David Starina wrote: > >> > Hi, > >> > > >> > Roman, Evans, thank you for the tips - that helped a lot. Still have > >> some > >> > problems though, an error when running puppet apply ... > >> > > >> > This is a sample of errors I got: > >> > > >> > > >> > *Debug: Executing '/bin/yum -d 0 -e 0 -y install > >> > ignite-hadoop-service'Error: Could not update: Execution of '/bin/yum > >> -d 0 > >> > -e 0 -y install ignite-hadoop-service' returned 1: Error: Package: > >> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) > >> Requires: > >> > osgi(org.apache.servicemix.bundles.spring-beans)Error: Package: > >> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) > >> Requires: > >> > osgi(org.apache.servicemix.bundles.spring-context-support)Error: > >> Package: > >> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) > >> Requires: > >> > osgi(org.apache.servicemix.bundles.spring-jdbc)Error: Package: > >> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) > >> Requires: > >> > osgi(org.apache.servicemix.bundles.spring-core)* > >> > ..... > >> > > >> > Also, after that: > >> > > >> > > >> > > >> > > >> > > >> > > >> > *Notice: > >> > > >> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]: > >> > Dependency Package[ignite-hadoop] has failures: trueWarning: > >> > > >> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]: > >> > Skipping because of failed dependenciesNotice: > >> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Dependency > >> > Package[openjdk-7-jre-headless] has failures: trueWarning: > >> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Skipping > >> > because of failed dependenciesNotice: > >> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]: > >> > Dependency Package[openjdk-7-jre-headless] has failures: trueWarning: > >> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]: > >> Skipping > >> > because of failed dependenciesNotice: > >> > > >> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/ignite-hadoop/conf/default-config.xml]: > >> > Dependency Package[ignite-hadoop] has failures: true* > >> > ..... > >> > > >> > > >> > This is the "site.yaml" I used ("hadoop" wasn't listed in components, > >> but I > >> > think it should be there?) > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > *bigtop::hadoop_head_node: "NameNode"hadoop::hadoop_storage_dirs: - > >> > /data/1hadoop_cluster_node::cluster_components: - hadoop - > >> ignite_hadoop > >> > - hue - mahout - yarn - zookeeperbigtop::jdk_package_name: > >> > "openjdk-7-jre-headless"bigtop::bigtop_repo_uri: > >> > "http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64 > >> > <http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64>"* > >> > > >> > > >> > > >> > > >> > Any idea what I am doing wrong? > >> > > >> > --David > >> > > >> > On Tue, Mar 15, 2016 at 6:57 PM, Evans Ye <[email protected]> wrote: > >> > > >> > > David > >> > > > >> > > Most of the time you need to run same command on all nodes. "pdsh" > >> would > >> > > be a good tool to use. > >> > > > >> > > 1. You need to have puppet agent installed on every node. yum -y > >> install > >> > > puppet will do the trick, or you need to download repo, or use > >> > > bigtop/bigtop_toolchain/bin/puppetize.sh directly. There's no deploy > >> effort > >> > > other than just install the package. > >> > > > >> > > 2. Masterless means each puppet agent takes commands and runs > >> > > independently. We DONT use puppet master, which is a centralized > >> master > >> > > daemon to dispatch works to all the registered puppet agent. > >> > > > >> > > 3. Next, you need to copy bigtop puppet recipes and bigtop hiera > >> > > configuration to all nodes, or at least make them available for all > >> nodes > >> > > via nfs. > >> > > > >> > > 4. Finally, execute the puppet apply command on all nodes and the > >> puppet > >> > > recipes will automatically figure out it's role. There no need to > >> deploy > >> > > head node first and then slaves. > >> > > > >> > > Ask if you still have questions. > >> > > > >> > > > >> > > Thanks > >> > > > >> > > Evans > >> > > 2016年3月15日 下午4:10於 "David Starina" <[email protected]>寫道： > >> > > > >> > >> Hi guys, > >> > >> > >> > >> me again. I haven't set up Puppet before, can you just point me in > >> the > >> > >> right direction? I know this is not a Puppet mailing list, but I bet > >> you > >> > >> have some tips for me :-) > >> > >> > >> > >> So what do I need to do? Install Puppet on every node? What does it > >> mean > >> > >> that the mode of Puppet is "*masterless"* - do I just have to do a > >> basic > >> > >> installation of Puppet and not care much about setting it up, and > >> then > >> > >> deploy bigtop on every node separately? And when setting the Hadoop > >> head > >> > >> node - does that mean that I need to deploy Bigtop on this node > >> first, and > >> > >> Bigtop will know this is the master node by the url or IP I provide > >> to it? > >> > >> And then deploy Bigtop via Puppet on all the other nodes and they > >> will > >> > >> connect to the head node (which is the NameNode, I suppose?). > >> > >> > >> > >> I can do this via trial and error, but if you guys can help me > >> reduce the > >> > >> "error" part, that will be great :) Sorry for such beginner > >> questions, I > >> > >> haven't used Puppet before ... The reason I'm asking is, I got > >> access to a > >> > >> "physical" (actually virtual, but it's the same) environment for a > >> limited > >> > >> amount of time (two weeks), and I need to start testing on it asap - > >> and > >> > >> since my day job takes a lot of my time, if I wanted to do this > >> without > >> > >> help it could take me two weeks just to set up the environment - but > >> I need > >> > >> to install it, which is not the central part of my project, and then > >> run > >> > >> some ML algorithms on a text corpus, which will be the main part of > >> the > >> > >> project ... > >> > >> > >> > >> --David > >> > >> > >> > >> On Thu, Mar 10, 2016 at 11:28 AM, David Starina < > >> [email protected]> > >> > >> wrote: > >> > >> > >> > >>> Thanks for the info! > >> > >>> > >> > >>> --David > >> > >>> > >> > >>> On Tue, Mar 8, 2016 at 7:15 PM, Jay Vyas < > >> [email protected]> > >> > >>> wrote: > >> > >>> > >> > >>>> I think there is enough info on youtube that you can cobble > >> together a > >> > >>>> syllabus if you need structured learning. > >> > >>>> > >> > >>>> See cos/Romans Bigtop talks which are on YouTube and then my Bigtop > >> > >>>> internals hackathon video. That gives you both the high and low > >> level > >> > >>>> details to do whatever you want. > >> > >>>> > >> > >>>> > On Mar 8, 2016, at 1:11 PM, Konstantin Boudnik <[email protected]> > >> > >>>> wrote: > >> > >>>> > > >> > >>>> > We also had this discussion to start producing "The Big Book of > >> Apache > >> > >>>> > Bigtop" like HBase does, but never got into the materialization > >> part > >> > >>>> of that > >> > >>>> > project. > >> > >>>> > > >> > >>>> > Cos > >> > >>>> > > >> > >>>> >> On Tue, Mar 08, 2016 at 11:44PM, Evans Ye wrote: > >> > >>>> >> Hi David, > >> > >>>> >> > >> > >>>> >> If you just want to take Bigtop binaries to deploy a cluster, > >> you > >> > >>>> can refer > >> > >>>> >> to our puppet management recipes: > >> > >>>> >> > >> https://github.com/apache/bigtop/tree/master/bigtop-deploy/puppet > >> > >>>> >> > >> > >>>> >> BTW bigtop provisioner is the best source to reference for how > >> bigtop > >> > >>>> >> provision a cluster: > >> > >>>> >> > >> > >>>> > >> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Provisioner+User+Guide > >> > >>>> >> > >> > >>>> >> Please ask us if you find anything confusing. > >> > >>>> >> > >> > >>>> >> > >> > >>>> >> Evans > >> > >>>> >> > >> > >>>> >> > >> > >>>> >> > >> > >>>> >> 2016-03-08 0:48 GMT+08:00 David Starina < > >> [email protected]>: > >> > >>>> >> > >> > >>>> >>> Hi, > >> > >>>> >>> > >> > >>>> >>> Is there some version of Bigtop User Guide available? I know it > >> > >>>> used to be > >> > >>>> >>> on the be on http://bigtop.apache.org/ under the "User Guide" > >> > >>>> link, but > >> > >>>> >>> there's nothing there anymore ... > >> > >>>> >>> > >> > >>>> >>> I need some guide on how to install Bigtop on a physical > >> cluster > >> > >>>> ... It > >> > >>>> >>> was easy to create a virtual cluster on my machine using > >> Vagrant, > >> > >>>> but I > >> > >>>> >>> cannot find a manual to install on a physical cluster ... > >> > >>>> >>> > >> > >>>> >>> Cheers, > >> > >>>> >>> David > >> > >>>> >>> > >> > >>>> > >> > >>> > >> > >>> > >> > >> > >> > >
signature.asc
Description: Digital signature
