@Krzysztof: I see the ShareMountPoint option, but it seemly does not support on Xen hypervisor, only KVM. Unfortunately, I'm using Xen.
2012/9/10 Krzysztof Janiszewski - ecenter sp. z o.o. < [email protected]> > You need Advanced Setup. Then you can setup Primary Storage as > SharedMountPoint and point it to, for example, /export/primary. Then, on > each Host you mount MooseFS storage to /export/primary and thats all. > > Secondary storage cannot be mounted as shared mount point. > > Best regards > Krzysztof Janiszewski > > > -----Original Message----- > From: Nguyen Anh Tu [mailto:[email protected]] > Sent: Monday, September 10, 2012 11:01 AM > To: [email protected] > Subject: Re: How to integrate Hadoop to CloudStack > > @Shanker: You right. We use CloudStack -> NFS -> GlusterFS. Bottle neck is > NFS. OneFS is seemly an enterprise solution, we prefer open source. > @Krzysztof Janiszewski: How can I mount MooseFS to CloudStack? > @both: you know that CloudStack supports some ways. As Secondary Storage, > you can only use NFS (maybe Swift if you enabled). As Primary Storage, you > have NFS, PreSetup and iSCSI. So, how can I use MooseFS or GlusterFS or > something else without NFS? > > 2012/9/10 Krzysztof Janiszewski - ecenter sp. z o.o. < > [email protected]> > > > Try MooseFS and mount it to CloudStack as shared mount point. Works > > fine for us. > > > > Best regards > > Krzysztof Janiszewski > > > > > > -----Original Message----- > > From: Shanker Balan [mailto:[email protected]] > > Sent: Monday, September 10, 2012 10:02 AM > > To: [email protected] > > Subject: Re: How to integrate Hadoop to CloudStack > > > > Hello, > > > > Nguyen Anh Tu wrote, > > > Hi Shanker, > > > > > > You right. I just want to use Hadoop HDFS for CS, no need any sub > > project. > > > Namenode is exactly a single point of failure. I found a good > > > solution instead. That's GlusterFS, which have no master node. All node > is peer. > > > However, CloudStack and Hadoop are both written by Java. They are > > > belong to ASF too. Personally, CloudStack and Hadoop are concerned. > > > > I had to do a bit of planning for the private cloud at my previous > > work and I pretty much went down the same line of thought for the > > storage as you did. > > NFS > > -> HDFS -> GlusterFS. Also considered DRBD. > > > > Unfortunately, CS does not support GlusterFS naively so you cant > > achieve HA. > > In order to use Gluster, you need to export the vols over NFS and you > > will run into all the limitations of NFS once more. > > > > I liked Isilon's OneFS solution as a replacement for Gluster though. > > > > Hth. > > > > > > > 2012/9/10 Hieu Le <[email protected]> > > > > > > > > > > > > > > > ---------- Forwarded message ---------- > > > > From: Shanker Balan <[email protected]> > > > > Date: Mon, Sep 10, 2012 at 1:53 PM > > > > Subject: Re: How to integrate Hadoop to CloudStack > > > > To: [email protected] > > > > > > > > > > > > Hello, > > > > > > > > Nguyen Anh Tu wrote, > > > > > > > > > Hello Shanker, > > > > > > > > > > I mean that with CS, I want to replace NFS to HDFS. You know NFS > > > > > is not a suitable solution for storage, because it has not > > fault-tolerant feature. > > > > > So I want to use HDFS for Secondary Storage in CS. I see this > > > > > http://www.slideshare.net/kkitase/cloudstack-architecture-future. > > > > > I > > > > think > > > > > in near future, Hadoop will be used as a storage solution in CS. > > > > > > > > You mean "HDFS" as a storage solution for CS? HDFS is just one of > > > > the components in the Hadoop project. Hadoop also includes non > > > > storage sub projects like MR, Pig, ZK etc. > > > > > > > > Regarding HDFS, The NameNode machine is a single point of failure > > > > for an HDFS cluster at this time. See > > > > > > > > http://hadoop.apache.org/common/docs/r0.20.2/hdfs_design.html#Meta > > > > da > > > > ta+Disk+Failure > > > > > > > > I have seen the NameNode fail many times at my previous $work > > > > place and its not fun. There is a HA NameNode solution in the > > > > works but I dont think its reached stable status. > > > > > > > > http://www.cloudera.com/blog/2012/03/high-availability-for-the-had > > > > oo > > > > p-distributed-file-system-hdfs/ > > > > > > > > In the end its about the scalability, availability, manageability > > > > and cost considerations you wish to achieve that decides every > > > > aspect of your cloud solution. > > > > > > > > Regards. > > > > > > > > > > > > > 2012/9/7 Shanker Balan <[email protected]> > > > > > > > > > > > (Moving to cloudstack-users@ with Bcc to > > > > > > [email protected]) > > > > > > > > > > > > Hello Nguyen, > > > > > > > > > > > > Nguyen Anh Tu wrote, > > > > > > > Hi guy, > > > > > > > > > > > > > > Anyone can help me to integrate Hadoop to CloudStack. I read > > > > > > > the > > > > article > > > > > > > "CloudStack and Hadoop: a match made in the cloud" but can > > > > > > > not find > > > > a way > > > > > > > to do this. > > > > > > > > > > > > Could you explain a bit more on what you mean by "Integrating > > > > > > Hadoop To Cloudstack"? I am not using CS yet, but I have a > > > > > > bunch of use cases I > > > > have > > > > > > been thinking about lately. > > > > > > > > > > > > You can use Cloudstack to provision Hadoop instances very easily. > > > > > > Cloudstack's > > > > > > bare metal provisioning capabilities allows you to build high > > > > performance > > > > > > clusters. > > > > > > > > > > > > > > > > > > > > > > http://www.cloudstack.org/blog/63-cloudstack-the-best-kept-secret- > > > > in > > > > -cloud-computing.html.html > > > > > > > > > > > > Cloudstack also provides an S3 compatible interface over > > > > > > supported > > > > object > > > > > > stores like Swift and Caringo. So instead of using HDFS, you > > > > > > can > > > > choose to > > > > > > store your data on CS backed by object store+s3 bridge. > > > > > > > > > > > > http://www.slideshare.net/sebastiengoasguen/cloudstack-s3 > > > > > > http://wiki.apache.org/hadoop/AmazonS3 > > > > > > > > > > > > On the other hand, if you are expecting a hosted Hadoop > > > > > > solution (like > > > > AWS > > > > > > EMR), I dont think that's quite ready yet (or if its even on > > > > > > the > > > > roadmap > > > > > > anytime soon). > > > > > > > > > > > > -- > > > > > > http://shankerbalan.net/ > > > > > > > > > > > > PS: cloudstack-users@ might be a more appropriate list to > > > > > > discuss this further. > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > N.g.U.y.e.N.A.n.H.t.U > > > > > > > > -- > > > > http://shankerbalan.net/ > > > > > > > > > > > > > > > > -- > > > > ..:: Hieu LE ::.. > > > > > > > > Class: Information System - Course 52 School of Information and > > > > Communication Technology Hanoi University of Technology No 1, Dai > > > > Co Viet street - Hai Ba Trung district - Hanoi > > > > > > > > High Performance Computing Center > > > > Cloud Computing Group > > > > Gmail: [email protected] > > > > > > > > > > > > > > > > -- > > > > > > N.g.U.y.e.N.A.n.H.t.U > > > > -- > > http://shankerbalan.net/ > > > > > > > -- > > N.g.U.y.e.N.A.n.H.t.U > > -- N.g.U.y.e.N.A.n.H.t.U
