Hi,
Many thanks for your advice
zpool status
pool: pool1
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
pool1 ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
scsi-350000396f8429a0d ONLINE 0 0 0
scsi-350000396f8429b35 ONLINE 0 0 0
errors: No known data errors
I have 64 GB RAM and the following on each node
cat /etc/modprobe.d/zfs.conf
# Min 2048MB / Max 4096 MB Limit
options zfs zfs_arc_min=2147483648
options zfs zfs_arc_max=4294967296
I do have 4 nodes but decided to use only 2 to test/make sure functionality
is there
It seems to me that both glusterfs and zfs are working properly from OS
perspective but fail from Proxmox perspective
Should I mount the gluster volumes ?
Would you be able to share a sample of /etc/pve/storage.cfg and/or
/etc/fstab ?
Steven
On 24 October 2016 at 17:54, Lindsay Mathieson <[email protected]>
wrote:
> On 25/10/2016 7:35 AM, Steven Vacaroaia wrote:
>
>> I am using bricks
>> Here is the info
>>
>>
>
> Ok, thanks for the info. Could you post a zpool status as well? Also how
> much RAM do your nodes have?
>
> with zfs the following are essential for gluster usage:
> - sync=standard
> - xattr=sa
> - acltype=posixacl
>
> sync = disabled is very dangerous! you run a severe risk of gluster
> corruption if the node crashes or power goes out.
>
> Recommended:
> - compression=lz4
> - atime=off
>
> Compression can actually speed up io as less data is written/read to the
> disk.
>
>
> Gluster:
> I really recommend a replication factor of 3 for any replicated
> filesystem. You run a much reduced chance of split-brain issues. If you
> can't afford a 3rd disk set then gluster has "arbiter" nodes (
> http://staged-gluster-docs.readthedocs.io/en/release3.7.0be
> ta1/Features/afr-arbiter-volumes/)
>
> Also for VM Hosting the following are *required* settings for gluster
> volumes:
> cluster.quorum-type: auto
> cluster.server-quorum-type: server
> performance.quick-read: off
> performance.read-ahead: off
> performance.io-cache: off
> cluster.eager-lock: enable
> network.remote-dio: enable
>
> And I'd recommend the following:
> performance.strict-write-ordering: off
> performance.stat-prefetch: on
> cluster.locking-scheme: granular
> cluster.granular-entry-heal: yes
> performance.readdir-ahead: on
> cluster.data-self-heal: on
>
> Additionally I'd look at using sharding, it makes a huge difference to
> heal times:
> features.shard: on
> features.shard-block-size: 64MB
>
> It breaks the VM's up into shards which can be healed sperately, rather
> than the entire VM image.
>
>
>
> --
> Lindsay Mathieson
>
> _______________________________________________
> pve-user mailing list
> [email protected]
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>
_______________________________________________
pve-user mailing list
[email protected]
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
