Hello List, My use case, is to provide a large hardware-RAID backed volume (hence no ZFS) to PVE for continuous decent performance writes by a running container (cctv dvr).
I have 6 x 6TB SATA drives on a hardware RAID 10 configuration. It 's formated as a ~10TB XFS device: /dev/sdb on /10TB type xfs (rw,relatime,attr2,inode64,noquota) I created a local directory storage pool on my PVE host and assigned a 9.8TB mount point to a running container (DVR) pve:/10TB/images/101# ls -lah /10TB/images/101/ total 6.3T drwxr----- 2 root root 30 Feb 15 2017 . drwxr-xr-x 4 root root 38 Feb 15 2017 .. -rw-r----- 1 root root 9.8T Oct 18 14:03 vm-101-disk-1.raw After a couple of months, I started receiving the following XFS errors on the host: Oct 17 06:26:09 pve kernel: [5629844.259800] XFS: loop0(28500) possible memory allocation deadlock size 72832 in kmem_alloc (mode:0x2400240) leaving me with an unusable filesystem from inside the container (no writes), unless I drop the host's page cache: echo 1 > /proc/sys/vm/drop_caches or even need to drop all pagecache, dentries and inodes: echo 3 > /proc/sys/vm/drop_caches (remember to sync first!) After research, I concluded that the deadlock is caused from heavy fragmentation on the XFS image file which has 1.2 million extends (!) according to xfs_bmap. Of course, there is no way to defrag this filesystem with xfs_fsr, as the volume file occupies almost 100% of its space (yes, I won't use more than 80%). Also it wouldn't be practical to do so, as the volume is busy 24/7 (cctv recording) and the available capacity matters. Please correct me if I am wrong, but the above seemed to be a reasonably decent layout to employ a local XFS filesystem (which is supposed to have better handling of large files than EXT4) under PVE. But in practice it's not! Am I missing something? Is LVM a more suitable solution? Any ideas or suggestions would be greatly appreciated. Thanks, Markos _______________________________________________ pve-user mailing list [email protected] https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
