Re: [zfs-discuss] Validating alignment of NTFS/VMDK/ZFS blocks
That's a good idea, thanks. I get the feeling the remainder won't be zero, which will back up the misalignment theory. After a bit more digging, it seems the problem is just an NTFS issue and can be addressed irrespective of underlying storage system. I think I'm going to try the process in the following link: http://www.tuxyturvy.com/blog/index.php?/archives/59-Aligning-Windows-Partitions-Without-Losing-Data.html With any luck I'll then see a smaller dedup table, and better performance! Thanks to those for feedback, Chris -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Validating alignment of NTFS/VMDK/ZFS blocks
On Thu, Mar 18, 2010 at 14:44, Chris Murray wrote: > Good evening, > I understand that NTFS & VMDK do not relate to Solaris or ZFS, but I was > wondering if anyone has any experience of checking the alignment of data > blocks through that stack? It seems to me there's a simple way to check. Pick 4k of random data (say, dd if=/dev/urandom of=newfile bs=4k count=1) and copy that onto the VM filesystem. Now write a little program to read the .vmdk file and find that 4k of data. Report the offset, and check offset % 4096 == 0. This won't help you fix things, but it'll at least tell you that something is wrong. Will ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Validating alignment of NTFS/VMDK/ZFS blocks
I have only heard of alignment being discussed in reference to block-based storage (like DASD/iSCSI/FC). I'm not really sure how it would work out over NFS. I do see why you are asking though. My understanding is that VMDK files are basically 'aligned' but the partitions inside of them may not be. You don't state what OS you are using in your guests. Windows XP/2003 and older create mis-alligned partitions by default (within a VMDK). You would need to manually create/adjust NTFS partitions in those cases in order for them to properly fall on a 4k boundary. This could be a cause of the problem you are describing. This doc from VMware is aimed at block-based storage but it has some concepts that might be helpful as well as info on aligning guest OS partitions: http://www.vmware.com/pdf/esx3_partition_align.pdf -Brian Chris Murray wrote: Good evening, I understand that NTFS & VMDK do not relate to Solaris or ZFS, but I was wondering if anyone has any experience of checking the alignment of data blocks through that stack? I have a VMware ESX 4.0 host using storage presented over NFS from ZFS filesystems (recordsize 4KB). Within virtual machine VMDK files, I have formatted NTFS filesystems, block size 4KB. Dedup is turned on. When I run ZDB -DD, i see a figure of unique blocks which is higher than I expect, which makes me wonder whether any given 4KB in the NTFS filesystem is perfectly aligned with a 4KB block in ZFS? e.g. consider two virtual machines sharing lots of the same blocks. Assuming there /is/ a misalignment between NTFS & VMDK/VMDK & ZFS, if they're not in the same order within NTFS, they don't align, and will actually produce different blocks in ZFS: VM1 NTFS1---2---3--- ZFS 1---2---3---4--- ZFS blocks are " AA", "AABB" and so on ... Then in another virtual machine, the blocks are in a different order: VM2 NTFS1---2---3--- ZFS 1---2---3---4--- ZFS blocks for this VM would be " CC", "CCAA", "AABB" etc. So, no overlap between virtual machines, and no benefit from dedup. I may have it wrong, and there are indeed 30,785,627 unique blocks in my setup, but if there's a mechanism for checking alignment, I'd find that very helpful. Thanks, Chris ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Validating alignment of NTFS/VMDK/ZFS blocks
On Thu, Mar 18, 2010 at 2:44 PM, Chris Murray wrote: > Good evening, > I understand that NTFS & VMDK do not relate to Solaris or ZFS, but I was > wondering if anyone has any experience of checking the alignment of data > blocks through that stack? > NetApp has a great little tool called mbrscan/mbralignit's free, but I'm not sure if NetApp customers are supposed to distribute it. -marc ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Validating alignment of NTFS/VMDK/ZFS blocks
Not having specific knowledge of the VMDK format, I think what you are seeing is that there is extra data associated with maintaining the VMDK. If you are seeing lower dedup ratios than you would expect, it sounds like some of this extra data could be added to each block. The VMDK spec appears to be open, perhaps a read through the spec might help understand what VMware is doing to the NTFS data - http://en.wikipedia.org/wiki/VMDK --joe On 3/18/2010 11:44 AM, Chris Murray wrote: Good evening, I understand that NTFS& VMDK do not relate to Solaris or ZFS, but I was wondering if anyone has any experience of checking the alignment of data blocks through that stack? I have a VMware ESX 4.0 host using storage presented over NFS from ZFS filesystems (recordsize 4KB). Within virtual machine VMDK files, I have formatted NTFS filesystems, block size 4KB. Dedup is turned on. When I run ZDB -DD, i see a figure of unique blocks which is higher than I expect, which makes me wonder whether any given 4KB in the NTFS filesystem is perfectly aligned with a 4KB block in ZFS? e.g. consider two virtual machines sharing lots of the same blocks. Assuming there /is/ a misalignment between NTFS& VMDK/VMDK& ZFS, if they're not in the same order within NTFS, they don't align, and will actually produce different blocks in ZFS: VM1 NTFS1---2---3--- ZFS 1---2---3---4--- ZFS blocks are " AA", "AABB" and so on ... Then in another virtual machine, the blocks are in a different order: VM2 NTFS1---2---3--- ZFS 1---2---3---4--- ZFS blocks for this VM would be " CC", "CCAA", "AABB" etc. So, no overlap between virtual machines, and no benefit from dedup. I may have it wrong, and there are indeed 30,785,627 unique blocks in my setup, but if there's a mechanism for checking alignment, I'd find that very helpful. Thanks, Chris ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Validating alignment of NTFS/VMDK/ZFS blocks
Please excuse my pitiful example. :-) I meant to say "*less* overlap between virtual machines", as clearly block "AABB" occurs in both. -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Chris Murray Sent: 18 March 2010 18:45 To: zfs-discuss@opensolaris.org Subject: [zfs-discuss] Validating alignment of NTFS/VMDK/ZFS blocks Good evening, I understand that NTFS & VMDK do not relate to Solaris or ZFS, but I was wondering if anyone has any experience of checking the alignment of data blocks through that stack? I have a VMware ESX 4.0 host using storage presented over NFS from ZFS filesystems (recordsize 4KB). Within virtual machine VMDK files, I have formatted NTFS filesystems, block size 4KB. Dedup is turned on. When I run ZDB -DD, i see a figure of unique blocks which is higher than I expect, which makes me wonder whether any given 4KB in the NTFS filesystem is perfectly aligned with a 4KB block in ZFS? e.g. consider two virtual machines sharing lots of the same blocks. Assuming there /is/ a misalignment between NTFS & VMDK/VMDK & ZFS, if they're not in the same order within NTFS, they don't align, and will actually produce different blocks in ZFS: VM1 NTFS1---2---3--- ZFS 1---2---3---4--- ZFS blocks are " AA", "AABB" and so on ... Then in another virtual machine, the blocks are in a different order: VM2 NTFS1---2---3--- ZFS 1---2---3---4--- ZFS blocks for this VM would be " CC", "CCAA", "AABB" etc. So, no overlap between virtual machines, and no benefit from dedup. I may have it wrong, and there are indeed 30,785,627 unique blocks in my setup, but if there's a mechanism for checking alignment, I'd find that very helpful. Thanks, Chris -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Validating alignment of NTFS/VMDK/ZFS blocks
Good evening, I understand that NTFS & VMDK do not relate to Solaris or ZFS, but I was wondering if anyone has any experience of checking the alignment of data blocks through that stack? I have a VMware ESX 4.0 host using storage presented over NFS from ZFS filesystems (recordsize 4KB). Within virtual machine VMDK files, I have formatted NTFS filesystems, block size 4KB. Dedup is turned on. When I run ZDB -DD, i see a figure of unique blocks which is higher than I expect, which makes me wonder whether any given 4KB in the NTFS filesystem is perfectly aligned with a 4KB block in ZFS? e.g. consider two virtual machines sharing lots of the same blocks. Assuming there /is/ a misalignment between NTFS & VMDK/VMDK & ZFS, if they're not in the same order within NTFS, they don't align, and will actually produce different blocks in ZFS: VM1 NTFS1---2---3--- ZFS 1---2---3---4--- ZFS blocks are " AA", "AABB" and so on ... Then in another virtual machine, the blocks are in a different order: VM2 NTFS1---2---3--- ZFS 1---2---3---4--- ZFS blocks for this VM would be " CC", "CCAA", "AABB" etc. So, no overlap between virtual machines, and no benefit from dedup. I may have it wrong, and there are indeed 30,785,627 unique blocks in my setup, but if there's a mechanism for checking alignment, I'd find that very helpful. Thanks, Chris -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
