On Tue, Oct 23, 2012 at 12:04 AM, <[email protected]> wrote: > > > Id suggest every one have a look at www.osnexus.com > supports fiber, > 10gb, infiniband using the SCST iSCSI code from > http://scst.sourceforge.net/ > has NFS and all the good stuff including a > pretty good GUI, replication (lumbering is not there yet) runs on > Ubuntu >
hahahahaha......... tiered pricing model, hardware not included....FAIL! License Licensed Capacity 128TB Support Tier $18,695.00 USD > On 23.10.2012 11:40, Andreas Huser wrote: >> Hi, >> >> for > Cloudstack i use Solaris 11 ZFS + GlusterFS over Infiniband >> (RDMA). > That gives the best performance and most scalable Storage. >> I have > tasted some different solutions for primary Storage but the >> most are > to expensive and for a CloudStack Cluster not economic or >> have a poor > performance. >> >> My Configuration: >> Storage Node: >> Supermicro Server > (Intel Hardware) with Solaris 11 with SSD write and >> read cache (read > crucial-m4, write ZeusIOPS) GlusterFS and dualport >> ConnectX 40Gbit/s > Infiniband adapter. >> >> I have installed GlusterFS direct on Solaris > with a modified code. >> Want you build bigger systems for more then 50 > VMs it is better you >> split the Solaris and GlusterFS with a separte > headnode for GlusterFS >> >> That looks like: >> Solaris ZFS > Backendstorage with a dataset Volume (Thin Provision) --> >> ( SRP Target > attached direct without Infiniband switch to GF Node) >> --> GlusterFS > Node the srp target formatted with xfs filesystem, >> create a GlusterFS > Volume --> ( Infiniband over a Mellanox Port >> Switch) --> Cloudstack > Node mount glusterFS Volume over RDMA >> >> For the Dataset Volume at the > ZFS Storage, disable atime and enable >> compression. >> (Space reclaim) > With compression you can shrink the ZFS Volume with >> command at Linux > dd /dev/zero or In a Windows VM with sdelete >> That gives you space left > on the Primary Storage for deleted Files in >> a VM or for deleted vhd's > or vm's in the cloudstack >> >> greeting Andreas >> >> >> >> >> Mit > freundlichen Grüßen >> >> Andreas Huser >> Geschäftsführer >> System > Engineer / Consultant >> (Cisco CSE, SMBAM, LCSE, ASAM) >> > --------------------------------------- >> Zellerstraße 28 - 77654 > Offenburg >> Tel: +49(781) 12786898 >> Mobil: +49(176) 10308549 >> > [email protected] >> >> >> >> >> ----- Ursprüngliche Mail ----- >> >> > Von: "Outback Dingo" >> An: [email protected] >> > Gesendet: Dienstag, 23. Oktober 2012 02:15:16 >> Betreff: Re: Primary > Storage >> >> On Mon, Oct 22, 2012 at 8:09 PM, Ivan Rodriguez wrote: >>> > Solaris 11 ZFS and yes we tried different setups, raids levels number > of >>> SSD cache, ARC zfs options etc etc etc. >>> >>> Cheers >>> >> >> > VMWare ?? >> >>> On Tue, Oct 23, 2012 at 11:05 AM, Outback Dingo > wrote: >>> >>>> On Mon, Oct 22, 2012 at 8:03 PM, Ivan Rodriguez wrote: >>>> >> We are using ZFS, with jbod, not in production yet exporting NFS > to >>>> > cloudstack, I'm not really happy about the performance >>>> > > but I think is related to the hardware itself rather than technology, > we >>>> > are using intel SR2625UR and Intel 320 SSD, we were evaluating > gluster as >>>> > well, but we decided to move away from that path since > gluster nfs is >>>> still >>>> > performing poorly, plus we would like to > see cloudstack integrating the >>>> > gluster-fuse module, we haven't > decided the final storage setup but at >>>> the >>>> > moment we had > better results with ZFS. >>>> > >>>> > >>>> >>>> question is whos ZFS and > have you "tweaked" the zfs / nfs config for >>>> performance >>>> >>>> >> >>>> > On Tue, Oct 23, 2012 at 10:44 AM, Nik Martin >> >wrote: >>>> >> >>>> >> On 10/22/2012 05:49 PM, Trevor Francis wrote: >>>> >> >>>> >>> > ZFS looks really interesting to me and I am leaning that way. I am >>>> >>>> considering using FreeNAS, as people seem to be having good luck > with >>>> >>> it. Can anyone weigh in here? >>>> >>> >>>> >>> >>>> >> My > personal opinion, I think FreeNAS and OpenFiler have horrible, >>>> > horrible >>>> >> User Interfaces - not very intuitive, and they both seem > to be file >>>> servers >>>> >> with things like iSCSI targets tacked on > as an afterthought. >>>> >> >>>> >> Nik >>>> >> >>>> >> >>>> >>> Trevor > Francis >>>> >>> Partner >>>> >>> 46 Labs | The PeerEdge Cloud >>>> >>> > http://www.46labs.com | >>>> http://www.peeredge.net >>>> >>> >>>> >>> > 405-362-0046 - Voice | 405-410-4980 - Cell >>>> >>> trevorgfrancis - > Skype >>>> >>> [email protected] >>>> >>> Solutions Provider for the > Telecom Industry >>>> >>> >>>> >>> >< >>>> >>> > http://www.twitter.**com/peeredge >< >>>> >>> > http://www.**twitter.com/peeredge >< >>>> >>> > http://**www.facebook.com/PeerEdge > >>>> >>> >>>> >>> On Oct 22, 2012, at > 2:30 PM, Jason Davis wrote: >>>> >>> >>>> >>> ZFS would be an interesting > setup as you can do the cache pools like >>>> you >>>> >>>> would do in > CacheCade. The problem with ZFS or CacheCade+DRBD is that >>>> >>>> > they >>>> >>>> really don't scale out well if you are looking for > something with a >>>> >>>> unified >>>> >>>> name space. I'll say however > that ZFS is a battle hardened FS with >>>> tons >>>> >>>> of >>>> >>>> > shops using it. A lot of the whiz-bang SSD+SATA disk SAN things > these >>>> >>>> smaller start up companies are hocking are just ZFS > appliances. >>>> >>>> >>>> >>>> RBD looks interesting but I'm not sure if > I would be willing to put >>>> >>>> production data on it, I'm not sure > how performant it is IRL. From a >>>> >>>> purely technical perspective, > it looks REALLY cool. >>>> >>>> >>>> >>>> I suppose anything is fast if > you put SSDs in it :) GlusterFS is >>>> another >>>> >>>> option although > historically small/random IO has not been it's strong >>>> >>>> > point. >>>> >>>> >>>> >>>> If you are ok spending money on software and > want a scale out block >>>> >>>> storage >>>> >>>> then you might want to > consider HP LeftHand's VSA product. I am >>>> >>>> personally >>>> >>>> > partial to NFS plays:) I went the exact opposite approach and > settled >>>> on >>>> >>>> Isilon for our primary storage for our CS > deployment. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Mon, Oct 22, > 2012 at 10:24 AM, Nik Martin >> >>>> >>wrote: >>>> >>>> >>>> >>>> On > 10/22/2012 10:16 AM, Trevor Francis wrote: >>>> >>>>> >>>> >>>>> We are > looking at building a Primary Storage solution for an >>>> >>>>>> > enterprise/carrier class application. However, we want to build it >>>> >>>>>>> using >>>> >>>>>> a FOSS solution and not a commercial solution. > Do you have a >>>> >>>>>> recommendation on platform? >>>> >>>>>> >>>> >>>>>>> >>>> >>>>>> Trevor, >>>> >>>>> >>>> >>>>> I got EXCELLENT results > builing a SAN from FOSS using: >>>> >>>>> OS: Centos >>>> >>>>> Hardware: > 2X storage servers, with 12x2TB 3.5 SATA drives. LSI >>>> MegaRAID >>>> >>>>>> with CacheCade Pro, with 240 GB Intel 520 SSDs configured to do > SSD >>>> >>>>> caching >>>> >>>>> (alternately, look at FlashCache from > Facebook) >>>> >>>>> intel 10GB dual port nics, one port for crossover, > on port for up >>>> link >>>> >>>>> to >>>> >>>>> storage network >>>> >>>>>> >>>> >>>>> DRBD for real time block replication to > active-active >>>> >>>>> Pacemaker+corosync for HA Resource > management >>>> >>>>> tgtd for iSCSI target >>>> >>>>> >>>> >>>>> If you > want file backed storage, XFS is a very good filesystem on >>>> Linux >>>> >>>>>> now. >>>> >>>>> >>>> >>>>> Pacemaker+Corosync can be difficult to > grok at the beginning, but >>>> that >>>> >>>>> setup gave me a VERY high > performance SAN. The downside is it is >>>> >>>>> entirely >>>> >>>>> > managed by CLI, no UI whatsoever. >>>> >>>>> >>>> >>>>> >>>> >>>>> Trevor > Francis >>>> >>>>>> Partner >>>> >>>>>> 46 Labs | The PeerEdge Cloud >>>> >>>>>>> http://www.46labs.com | >>>> >>>>>> http://www.peeredge.net >>>> >>>>>>> >>>> >>>>>> >>>> >>>>>> 405-362-0046 - Voice | 405-410-4980 - > Cell >>>> >>>>>> trevorgfrancis - Skype >>>> >>>>>> [email protected] >> > [email protected] >>>> >>>>>> > >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> > Solutions Provider for the Telecom Industry >>>> >>>>>> >>>> >>>>>> >> > http://www.twitter.com/**peeredge>< >>>> >>>>>> > http://www.twitter.com/**peeredge >> >>>< >>>> >>>>>> > http://www.twitter.**com/**peeredge < >>>> > http://www.twitter.com/**peeredge >>>> >>>>>> >>< >>>> >>>>>> > http://www.**twitter.com/**peeredge < >>>> >>>>>> > http://www.twitter.com/**peeredge >> >>>< >>>> >>>>>> > http://**www.facebook.com/**PeerEdge< >>>> > http://www.facebook.com/PeerEdge>< >>>> >>>>>> > http://www.facebook.com/**PeerEdge < >>>> > http://www.facebook.com/PeerEdge> >>>> >>>>>> >> >>>> >>>>>> >>>> >>>>>>> >>>> >>>>>> >>>> >>>>> >>>> >
