Added rjo...@redhat.com for having written a lot of the original code
involved here, thread from 2014 here:
https://sourceforge.net/p/ntfs-3g/mailman/ntfs-3g-devel/?viewmonth=201407
Replies below.
On Wed, Jan 9, 2019 at 11:29 AM Jean-Pierre André
<jean-pierre.an...@wanadoo.fr> wrote:
> james harvey wrote:
> > Not in the logs, but I ran more tests modifying the error message to
> > include discard_granularity and vol->cluster_size. On a thin volume,
> > 65536 and 4096. On a thick volume, 512 and 4096.
>
> So.
> To support this situation, I added some code to align
> the trimming bounds and remove the granularity check.
> This may of course leave some unused block un-trimmed,
> but the granularity has to be obeyed, even if the device
> block is smaller than the granularity imposed at some
> level.
>
> See the attached patch-v2
For when discard_alignment is 0, this patch seems good to me.
Test 1 - Using an LVM thin pool with discard_granularity/chunk_size
128MB, I used ntfs-3g to create a 5GB volume. Not using quick, after
writing zeros to drive, the volume showed 100% usage in LVM. Mounting
it and running fstrim showed 4.4 GiB trimmed. LVM showed a
corresponding drop in usage. With a 128MB discard_granularity, that
seems reasonable to me, not knowing the exact specifics of now ntfs is
laid out.
Test 2 - I made a copy of my win7 filesystem, on an LVM thin pool with
discard_granularity/chunk_size 1MB, so discard_alignment would be 0.
It's a 90G volume, showing 71G used. Mounting it and running fstrim
showed 13.5 GiB trimmed. LVM showed a corresponding drop in usage.
With a 1MB discard_granularity, it seems reasonable to me that only
13.5 of the 20G free is discarded. I then dropped caches (echo 3 >
/proc/sys/vm/drop_caches), mounted the original volume, and ran "diff
--recursive" in an attempt to make sure that nothing was discarded
improperly. It found no differences. This was on a Windows 7 system
volume with 225,461 files/directories, so that seems like a pretty
thorough test to me, given the topology.
Test 3 - Repeated test 2, but with a thin pool
discard_granularity/chunk_size 64KB. fstrim shows 18.3 GiB trimmed.
LVM shows a corresponding drop. I was expecting a bit more, but seems
reasonable enough that 8.5% of the free FS 4K chunks reside within a
discard 16K chunk with something else. Ran another diff, still no
differences. Here, I also changed the VM to use the LVM volume, and
was able to boot and use it without apparent problems.
> ...
> > if groups of consecutive clusters all being empty were looked for, to
> > make up an entire discard_granularity, and if so, discarding those,
> > the alignment would presumably need to be considered from the
> > beginning of the underlying device (the LVM thin volume), rather than
> > just within the partition.
>
> ntfs-3g does not reliably know where the ntfs partition
> sits on the device (there is a field in the boot sector for
> that, but it is not always fed in by formaters). The
> proposed patch assumes that the partition starts at
> a multiple of the granularity. Maybe this is where the
> discard_alignment comes into play, and maybe this
> should be taken into account when rounding the trim
> zones. I am not sure this is what discard_alignment is
> for ("The discard_alignment parameter indicates how
> many bytes the beginning of the partition is offset
> from the internal allocation unit's natural alignment").
It looks to me like that's what discard_alignment is for.
I emailed the dm-devel (LVM related) list to make sure all of this is
on the right track. See
https://www.redhat.com/archives/dm-devel/2019-January/msg00057.html --
One of their main dev's confirmed that LVM doesn't track smaller
amounts than discard_granularity, so only aligned chunks of
discard_granularity are discarded. He also said that if
discard_granularity is 64K, and an unaligned trim of 64K is sent at
position 32K, nothing happens. So, it's sounding to me like getting
this right is to efficiently discard, rather than prevent data loss.
I was thinking that LVM is calculating discard_alignment incorrectly.
I emailed the linux kernel list and tagged Martin Petersen in it,
being the contact for it. See https://lkml.org/lkml/2019/1/9/1190 --
His reply seems to confirm for me that LVM is calculating it
incorrectly. I reported that today on the dm-devel thread linked in
previous paragraph.
To me, and from my reading of what Martin said, discard_granularity
should be the offset from the beginning of the internal allocation
unit that holds the beginning of the block device (here, a partition),
to the beginning of the block device. But, LVM/dm/kpartx seems to be
calculating it in reverse, instead giving the offset from where the
block device (partition) starts to the beginning of the NEXT
allocation unit.
If an LVM volume has discard_granularity/chunk_size is 128MB, and
within it there's a partition table setting partition 1 at 1MB, I'd
expect discard_alignment to be 1MB, but LVM gives 127MB.
So, if I'm right on this, I won't be able to test a patch including
support for discard_alignment!=0 until LVM/dm/kpartx (not sure which)
is fixed.
_______________________________________________
ntfs-3g-devel mailing list
ntfs-3g-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ntfs-3g-devel
