james harvey wrote:
Replies to BOTH of your emails are below. Links to new output are at
the very bottom.
On Tue, Jan 8, 2019 at 4:04 AM Jean-Pierre André
<jean-pierre.an...@wanadoo.fr> wrote:
james harvey wrote:
[...]
Note : I personally do not have any ssd and I do not
use lvm, so I will have to rely on you for tests, and to
do that you may have to compile ntfs-3g with some
patches.
That's totally fine by me.
Arch uses the source from here:
https://tuxera.com/opensource/ntfs-3g_ntfsprogs-2017.3.23.tgz
The only modification Arch makes is applying your patch "[PATCH]
Stopped checking matches of MFTMirr against MFT at record 16" visible
here:
https://git.archlinux.org/svntogit/packages.git/tree/trunk/mftmirr.patch?h=packages/ntfs-3g
The steps Arch uses to make the package are visible here:
https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/ntfs-3g
An Arch PKGBUILD is easily read like a bash script, and its build
system call the provided bash functions "prepare()", "build()", and
"package_ntfs-3g()".
If you send any patches, please let me know if you'd like me to apply
them on top of this released source, or if you'd like me to use a
developmental version, such as from a git (or other VCS) repo. If a
developmental version, please let me know where it's at. I couldn't
find a repo for ntfs-3g. And, if any of the developmental version
commits require a change in the build process.
Running ntfs-3g 2017.3.23 on Arch Linux. fstrim (util-linux) 2.33.
I see ntfs-3g gained discard support quite some time ago, but haven't
been able to get it to work.
[...]
Also, on a thin device, I do not know how lvm reacts
to trimming a block which does not exist. ntfs-3g
forwards the trimming request irrespective of block
allocations on a thin device.
Understood, thanks for mentioning. I'm thinking fstrim with btrfs
would probably act the same way. I don't think btrfs is aware of
what's allocated on an underlying LVM thin volume, so I think however
LVM handles this is OK.
It's because of this ioctl that I'm starting here rather than
util-linux. It makes me think "fstrim" is at least attempting to give
"ntfs-3g" the trim command.
I saw that ntfs-3g doesn't support old Windows (before ~2008)
partitions.
Where did you get this from ? ntfs-3g is supposed to
support all ntfs partitions since NT4.
There are no recent changes in the trimming code, so you can
keep the version from Tuxera used by arch, or the latest code
from the repository on
https://sourceforge.net/p/ntfs-3g/ntfs-3g/ci/edge/tree/
For a test build, you may want to use the basic sequence
without installing (and without disturbing your environment)
./configure
make
Just use the script src/ntfs-3g for mounting.
Note : from the repository you first have to generate the
./configure by running ./autogen.sh If you are more familiar
with the arch building procedure, you probably do not have
to make any change to it.
[...]
So what does "stat /dev/lvm/ntfs" return ?
# stat /dev/lvm/ntfs
File: /dev/lvm/ntfs -> ../dm-12
Size: 8 Blocks: 0 IO Block: 4096 symbolic link
Device: 6h/6d Inode: 19698944 Links: 1
Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2019-01-08 19:24:02.738191553 -0500
Modify: 2019-01-08 19:24:02.738191553 -0500
Change: 2019-01-08 19:24:02.738191553 -0500
Birth: -
$ ls -la /dev/mapper/ | grep dm-12
lrwxrwxrwx 1 root root 8 Jan 8 19:24 lvm-ntfs -> ../dm-12
# stat /dev/dm-12
File: /dev/dm-12
Size: 0 Blocks: 0 IO Block: 4096 block special file
Device: 6h/6d Inode: 19689157 Links: 1 Device type: fe,c
Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 994/ disk)
Access: 2019-01-08 19:44:59.103746094 -0500
Modify: 2019-01-08 19:44:59.103746094 -0500
Change: 2019-01-08 19:44:59.103746094 -0500
Birth: -
I initially hoped this has shed light on what was causing the problem,
as LVM uses device mapper and then creates symlinks to them using the
user-provided names.
Unfortunately, mounting "/dev/dm-12" instead of "/dev/lvm/ntfs" still
failed. Links to the mounting and strace output are below in the next
sectional reply, both for using "/dev/lvm/ntfs" and "/dev/dm-12".
Actually ntfs-3g follows the links, and from the log, I see the actual
device is /dev/mapper/lvm-ntfs and I assume this is correct.
[...]
Sure, sorry for any confusion from my original message. From scratch:
# lvcreate --virtualsize 10G --thin lvm/thinpool --name ntfs
Logical volume "ntfs" created.
# mkfs.ntfs /dev/lvm/ntfs
Cluster size has been automatically set to 4096 bytes.
Initializing device with zeroes: 100% - Done.
Creating NTFS volume structures.
mkntfs completed successfully. Have a nice day.
# mkdir /tmp/ntfs
# lowntfs-3g -o debug /dev/lvm/ntfs /tmp/ntfs 2> ntfslog
# strace fstrim -v /tmp/ntfs 2> strace.fstrim
# umount /tmp/ntfs
$ cat ntfslog | netcat termbin.com 9999
https://termbin.com/bo5n
$ cat strace.fstrim | netcat termbin.com 9999
https://termbin.com/sbnm
# ntfs-3g -o debug /dev/dm-12 /tmp/ntfs 2> ntfslog.dm
# strace fstrim -v /tmp/ntfs 2> strace.fstrim.dm
# umount /tmp/ntfs
$ cat ntfslog.dm | netcat termbin.com 9999
https://termbin.com/8yf9l
$ cat strace.fstrim.dm | netcat termbin.com 9999
https://termbin.com/defd2
This is the correct procedure, just replace ntfs-3g by
src/ntfs-3g (src being the directory used for compiling),
and no need to use both ntfs-3g and lowntfs-3g : the log
has shown the error was issued by ntfs-3g.
Also the strace is probably not useful any more.
Attached is a patch to help understanding the issue, with
a more precise logging, and without stopping if a lower
level returns an error. The aim here is to determine if the
error has something to do with thin provisioning. For this
to be meaningful, you need to create a file and remove it
before trimming, for instance :
dd if=/dev/random of=/tmp/ntfs/random-data bs=16384 count=64
rm /tmp/ntfs/random-data
(using bs=16384 reduces the size of log).
Maybe a test without thin provisioning would be useful.
Jean-Pierre
--- libntfs-3g/ioctl.c.ref 2017-03-23 10:42:44.000000000 +0100
+++ libntfs-3g/ioctl.c 2019-01-09 09:23:01.998113900 +0100
@@ -243,7 +243,9 @@
u8 *buf = NULL;
LCN start_buf;
int ret;
+int successes, unsupported, other;
+successes = unsupported = other = 0;
ntfs_log_debug("fstrim: start=%llu len=%llu minlen=%llu\n",
(unsigned long long) start,
(unsigned long long) len,
@@ -255,11 +257,11 @@
* XXX We could fix these limitations in future.
*/
if (start != 0 || len != (uint64_t)-1) {
- ntfs_log_debug("fstrim: setting start or length is not supported\n");
+ ntfs_log_error("fstrim: setting start or length is not supported\n");
return -EINVAL;
}
if (minlen > vol->cluster_size) {
- ntfs_log_debug("fstrim: minlen > cluster size is not supported\n");
+ ntfs_log_error("fstrim: minlen > cluster size is not supported\n");
return -EINVAL;
}
@@ -269,7 +271,7 @@
* different.
*/
if (!NDevBlock(vol->dev)) {
- ntfs_log_debug("fstrim: not supported for non-block-device\n");
+ ntfs_log_error("fstrim: not supported for non-block-device\n");
return -EOPNOTSUPP;
}
@@ -278,15 +280,15 @@
if (ret)
return ret;
if (discard_alignment != 0) {
- ntfs_log_debug("fstrim: backing device is not aligned for discards\n");
+ ntfs_log_error("fstrim: backing device is not aligned for discards\n");
return -EOPNOTSUPP;
}
if (discard_granularity > vol->cluster_size) {
- ntfs_log_debug("fstrim: discard granularity of backing device is larger than cluster size\n");
+ ntfs_log_error("fstrim: discard granularity of backing device is larger than cluster size\n");
return -EOPNOTSUPP;
}
if (discard_max_bytes == 0) {
- ntfs_log_debug("fstrim: backing device does not support discard (discard_max_bytes == 0)\n");
+ ntfs_log_error("fstrim: backing device does not support discard (discard_max_bytes == 0)\n");
return -EOPNOTSUPP;
}
@@ -341,8 +343,13 @@
ret = fstrim_clusters(vol,
start_lcn, end_lcn-start_lcn);
+if (ret == -EOPNOTSUPP) unsupported++;
+else if (ret) other++;
+else successes++;
+/*
if (ret)
goto free_out;
+*/
*trimmed += (end_lcn - start_lcn)
<< vol->cluster_size_bits;
@@ -353,6 +360,7 @@
ret = 0;
free_out:
+ntfs_log_error("trim successes %d unsupported %d other %d\n", successes, unsupported, other);
free(buf);
return ret;
}
_______________________________________________
ntfs-3g-devel mailing list
ntfs-3g-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ntfs-3g-devel
