... keep it simple, make it robust ... use raid1 (or raidz if you can) for the bricks hth
On 19.02.2014, at 20:32, Targino Silveira <[email protected]> wrote: > Sure, > > I will use XFS, as I sayd before it's for old data, so we don't need a great > performance, we only need to store data. > > regards, > > Targino Silveira > +55-85-8626-7297 > www.twitter.com/targinosilveira > > > 2014-02-19 16:11 GMT-03:00 BGM <[email protected]>: >> well, note: >> - you don't need zfs on the hardeware machines, xfs or ext3 or ext4 would do >> it too >> - for production you wouldn't use a glusterfs on top of a glusterfs but >> rather giving the vm access to a real blockdevice, like a whole harddisk or >> at least a partition of it although migration of the vm wouldn't be possible >> than... >> therefor: a VM as a glusterserver might not be the best idea. >> - remember to peer probe the glusterserver partner from both sides! as >> mentioned below >> >> for a first setup you should be fine with that. >> >> regards >> >> On 19.02.2014, at 19:32, Targino Silveira <[email protected]> wrote: >> >>> Thanks Bernhard I will do this. >>> >>> Regards, >>> >>> >>> Targino Silveira >>> +55-85-8626-7297 >>> www.twitter.com/targinosilveira >>> >>> >>> 2014-02-19 14:43 GMT-03:00 Bernhard Glomm <[email protected]>: >>>> I would strongly recommend to restart fresh with gluster 3.2.4 from >>>> http://download.gluster.org/pub/gluster/glusterfs/3.4/ >>>> It works totally fine for me. >>>> (reinstall the vms as slim as possible if you can.) >>>> >>>> As a quick howto consider this: >>>> >>>> - We have 2 Hardware machines (just desktop machines for dev-env) >>>> - both running zol >>>> - create a zpool and zfs filesystem >>>> - create a gluster replica 2 volume between hostA and hostB >>>> - installe 3 VM vmachine0{4,5,6} >>>> - vmachine0{4,5} each have a 100GB diskimage file as /dev/vdb which also >>>> resides on the glustervolume >>>> - create ext3 filesystem on vmachine0{4,5}:/dev/vdb1 >>>> - create gluster replica 2 between vmachine04 and vmachine05 as shown below >>>> >>>> (!!!obviously nobody would do that in any serious environment, >>>> just to show that even a setup like that _would_ be possible!!!) >>>> >>>> - run some benchmarks on that volume and compare the results to other >>>> >>>> So: >>>> >>>> root@vmachine04[/0]:~ # mkdir -p /srv/vdb1/gf_brick >>>> root@vmachine04[/0]:~ # mount /dev/vdb1 /srv/vdb1/ >>>> root@vmachine04[/0]:~ # gluster peer probe vmachine05 >>>> peer probe: success >>>> >>>> # now switch over to vmachine05 and do >>>> >>>> root@vmachine05[/1]:~ # mkdir -p /srv/vdb1/gf_brick >>>> root@vmachine05[/1]:~ # mount /dev/vdb1 /srv/vdb1/ >>>> root@vmachine05[/1]:~ # gluster peer probe vmachine04 >>>> peer probe: success >>>> root@vmachine05[/1]:~ # gluster peer probe vmachine04 >>>> peer probe: success: host vmachine04 port 24007 already in peer list >>>> >>>> # the peer probe from BOTH sides ist often forgotten >>>> # switch back to vmachine04 and continue with >>>> >>>> root@vmachine04[/0]:~ # gluster peer status >>>> Number of Peers: 1 >>>> >>>> Hostname: vmachine05 >>>> Port: 24007 >>>> Uuid: 085a1489-dabf-40bb-90c1-fbfe66539953 >>>> State: Peer in Cluster (Connected) >>>> root@vmachine04[/0]:~ # gluster volume info layer_cake_volume >>>> >>>> Volume Name: layer_cake_volume >>>> Type: Replicate >>>> Volume ID: ef5299db-2896-4631-a2a8-d0082c1b25be >>>> Status: Started >>>> Number of Bricks: 1 x 2 = 2 >>>> Transport-type: tcp >>>> Bricks: >>>> Brick1: vmachine04:/srv/vdb1/gf_brick >>>> Brick2: vmachine05:/srv/vdb1/gf_brick >>>> root@vmachine04[/0]:~ # gluster volume status layer_cake_volume >>>> Status of volume: layer_cake_volume >>>> Gluster process Port Online Pid >>>> ------------------------------------------------------------------------------ >>>> Brick vmachine04:/srv/vdb1/gf_brick 49152 Y >>>> 12778 >>>> Brick vmachine05:/srv/vdb1/gf_brick 49152 Y >>>> 16307 >>>> NFS Server on localhost 2049 Y >>>> 12790 >>>> Self-heal Daemon on localhost N/A Y >>>> 12791 >>>> NFS Server on vmachine05 2049 Y >>>> 16320 >>>> Self-heal Daemon on vmachine05 N/A Y >>>> 16319 >>>> >>>> There are no active volume tasks >>>> >>>> # set any option you might like >>>> >>>> root@vmachine04[/1]:~ # gluster volume set layer_cake_volume >>>> network.remote-dio enable >>>> volume set: success >>>> >>>> # go to vmachine06 and mount the volume >>>> root@vmachine06[/1]:~ # mkdir /srv/layer_cake >>>> root@vmachine06[/1]:~ # mount -t glusterfs -o >>>> backupvolfile-server=vmachine05 vmachine04:/layer_cake_volume >>>> /srv/layer_cake >>>> root@vmachine06[/1]:~ # mount >>>> vmachine04:/layer_cake_volume on /srv/layer_cake type fuse.glusterfs >>>> (rw,default_permissions,allow_other,max_read=131072) >>>> root@vmachine06[/1]:~ # df -h >>>> Filesystem Size Used Avail Use% Mounted on >>>> ... >>>> vmachine04:/layer_cake_volume 97G 188M 92G 1% /srv/layer_cake >>>> >>>> All fine and stable >>>> >>>> >>>> >>>> # now let's see how it tastes >>>> # note this is postmark on / NOT on the glustermounted layer_cake_volume! >>>> # that postmark results might be available tomorrow ;-))) >>>> root@vmachine06[/1]:~ # postmark >>>> PostMark v1.51 : 8/14/01 >>>> pm>set transactions 500000 >>>> pm>set number 200000 >>>> pm>set subdirectories 10000 >>>> pm>run >>>> Creating subdirectories...Done >>>> Creating files...Done >>>> Performing transactions..........Done >>>> Deleting files...Done >>>> Deleting subdirectories...Done >>>> Time: >>>> 2314 seconds total >>>> 2214 seconds of transactions (225 per second) >>>> Files: >>>> 450096 created (194 per second) >>>> Creation alone: 200000 files (4166 per second) >>>> Mixed with transactions: 250096 files (112 per second) >>>> 249584 read (112 per second) >>>> 250081 appended (112 per second) >>>> 450096 deleted (194 per second) >>>> Deletion alone: 200192 files (3849 per second) >>>> Mixed with transactions: 249904 files (112 per second) >>>> >>>> Data: >>>> 1456.29 megabytes read (644.44 kilobytes per second) >>>> 2715.89 megabytes written (1.17 megabytes per second) >>>> >>>> # reference >>>> # running postmark on the hardware machine directly on zfs >>>> # >>>> # /test # postmark >>>> # PostMark v1.51 : 8/14/01 >>>> # pm>set transactions 500000 >>>> # pm>set number 200000 >>>> # pm>set subdirectories 10000 >>>> # pm>run >>>> # Creating subdirectories...Done >>>> # Creating files...Done >>>> # Performing transactions..........Done >>>> # Deleting files...Done >>>> # Deleting subdirectories...Done >>>> # Time: >>>> # 605 seconds total >>>> # 549 seconds of transactions (910 per second) >>>> # >>>> # Files: >>>> # 450096 created (743 per second) >>>> # Creation alone: 200000 files (4255 per second) >>>> # Mixed with transactions: 250096 files (455 per second) >>>> # 249584 read (454 per second) >>>> # 250081 appended (455 per second) >>>> # 450096 deleted (743 per second) >>>> # Deletion alone: 200192 files (22243 per second) >>>> # Mixed with transactions: 249904 files (455 per second) >>>> # >>>> # Data: >>>> # 1456.29 megabytes read (2.41 megabytes per second) >>>> # 2715.89 megabytes written (4.49 megabytes per second) >>>> >>>> dbench -D /srv/layer_cake 5 >>>> >>>> Operation Count AvgLat MaxLat >>>> ---------------------------------------- >>>> NTCreateX 195815 5.159 333.296 >>>> Close 143870 0.793 93.619 >>>> Rename 8310 10.922 123.096 >>>> Unlink 39525 2.428 203.753 >>>> Qpathinfo 177736 2.551 220.605 >>>> Qfileinfo 31030 2.057 175.565 >>>> Qfsinfo 32545 1.393 174.045 >>>> Sfileinfo 15967 2.691 129.028 >>>> Find 68664 9.629 185.739 >>>> WriteX 96860 0.841 108.863 >>>> ReadX 307834 0.511 213.602 >>>> LockX 642 1.511 10.578 >>>> UnlockX 642 1.541 10.137 >>>> Flush 13712 12.853 405.383 >>>> >>>> Throughput 10.1832 MB/sec 5 clients 5 procs max_latency=405.405 ms >>>> >>>> >>>> # reference >>>> dbench -D /tmp 5 >>>> # reference >>>> dbench -D /tmp 5 >>>> >>>> Operation Count AvgLat MaxLat >>>> ---------------------------------------- >>>> NTCreateX 3817455 0.119 499.847 >>>> Close 2804160 0.005 16.000 >>>> Rename 161655 0.322 459.790 >>>> Unlink 770906 0.556 762.314 >>>> Deltree 92 20.647 81.619 >>>> Mkdir 46 0.003 0.012 >>>> Qpathinfo 3460227 0.017 18.388 >>>> Qfileinfo 606258 0.003 11.652 >>>> Qfsinfo 634444 0.006 14.976 >>>> Sfileinfo 310990 0.155 604.585 >>>> Find 1337732 0.056 18.466 >>>> WriteX 1902611 0.245 503.604 >>>> ReadX 5984135 0.008 16.154 >>>> LockX 12430 0.008 9.111 >>>> UnlockX 12430 0.004 4.551 >>>> Flush 267557 4.505 902.093 >>>> >>>> Throughput 199.664 MB/sec 5 clients 5 procs max_latency=902.099 ms >
_______________________________________________ Gluster-users mailing list [email protected] http://supercolony.gluster.org/mailman/listinfo/gluster-users
