Dear Jean-Piere, Thank you! Here is the requested data:
http://paste.debian.net/plain/909296 I updated the plug-in by wget http://jp-andre.pagesperso-orange.fr/dedup.zip the file name was the same, so I hope the plugin was updated, the date stamp looked from the 16th.. -rwxr-xr-x 1 root root 17K Jan 16 15:47 ntfs-plugin-80000013.so Kind regards, Jelle de Jong On 17/01/17 14:08, Jean-Pierre André wrote: > Hi, > > Jelle de Jong wrote: >> Hi Jean-Piere, >> >> I managed to get the plug-in loaded with guestfs tools, but I could not >> read the files, so I went back to my basic ntfs mount test without >> guestmount and I could also not read the files! >> >> The plug-in is located at: >> /usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-80000013.so without the >> plug-in I get the "unsupported reparse point" messages, with the plug-in >> I do not get these messages, but I am still unable to read the files. :( >> >> I added the getfattr -h -e hex -n system.ntfs_reparse_data of a sample >> file as requested and the syslog messages. > > Ok. This shows a variation which only recently I have > been aware of : the Smap id is apparently incremented > when the Smap is updated. > > I have just updated the plugin, please load it again > and replace it. > >> https://paste.debian.net/hidden/4d358e9e/ > > This also shows the plugin was activated correctly. > >> Could you please help me out further? > > I can at least try, with some help from you. > > Please retry with the updated plugin, and please only > mount the partition as read-only, because due to > deduplication, when a file is updated, the layout of > other files is changed, which makes debugging difficult. > > It the updated plugin does not work with the same unchanged > file, please post a 1KB excerpt of the file 00120000.0000000a.ccc > from the directory whose path can be stated as : > /mnt/sr7-sdb2/System*/Dedup/ChunkStore/{0DECAE8D*/Stream > > To extract the needed data you can use : > > dd if=/mnt/--etc--/Stream/00120000.0000000a.ccc bs=512 skip=17862 > count=2 | od -t x1 > (I have put the full command as an attachment to avoid formatting > by the mailer). > > I am of course assuming the partition was not changed since > your earlier post. > > Note : in this excerpt there is no user data, you can > safely post it. > > Regards > > Jean-Pierre > >> Thank you in advance! >> >> Kind regards, >> >> Jelle de Jong >> >> >> lvcreate --snapshot --name sr7-disk2-snapshot-copy --size 50G >> /dev/lvm1-vol/sr7-disk2 >> lvchange -ay /dev/lvm1-vol/sr7-disk2-snapshot-copy >> blkid /dev/lvm1-vol/sr7-disk2-snapshot-copy >> kpartx -avg /dev/lvm1-vol/sr7-disk2-snapshot-copy >> blkid /dev/mapper/lvm1--vol-sr7--disk2--snapshot--copy2 >> mount -o ro -t ntfs-3g >> /dev/mapper/lvm1--vol-sr7--disk2--snapshot--copy2 /mnt/sr7-sdb2 >> >> Jan 17 11:04:14 backup ntfs-3g[5032]: Version 2016.2.22AR.1 integrated >> FUSE 28 >> Jan 17 11:04:14 backup ntfs-3g[5032]: Mounted >> /dev/mapper/lvm1--vol-sr7--disk2--snapshot--copy2 (Read-Only, label >> "DATA", NTFS 3.1) >> Jan 17 11:04:14 backup ntfs-3g[5032]: Cmdline options: ro >> Jan 17 11:04:14 backup ntfs-3g[5032]: Mount options: >> ro,allow_other,nonempty,relatime,fsname=/dev/mapper/lvm1--vol-sr7--disk2--snapshot--copy2,blkdev,blksize=4096 >> >> >> Jan 17 11:04:14 backup ntfs-3g[5032]: Ownership and permissions >> disabled, configuration type 7 >> Jan 17 11:04:33 backup ntfs-3g[5032]: Failed to open a dedup stream last >> try was System Volume >> Information/Dedup/ChunkStore/{0DECAE8D-71D2-4BDE-8798-530201C72D8D}.ddp/Stream/00120000.00000002.ccc >> >> >> Jan 17 11:04:55 backup ntfs-3g[5032]: Failed to open a dedup stream last >> try was System Volume >> Information/Dedup/ChunkStore/{0DECAE8D-71D2-4BDE-8798-530201C72D8D}.ddp/Stream/00120000.00000002.ccc >> >> >> Jan 17 11:04:55 backup ntfs-3g[5032]: Failed to open a dedup stream last >> try was System Volume >> Information/Dedup/ChunkStore/{0DECAE8D-71D2-4BDE-8798-530201C72D8D}.ddp/Stream/00120000.00000002.ccc >> >> >> Jan 17 11:04:57 backup ntfs-3g[5032]: Failed to open a dedup stream last >> try was System Volume >> Information/Dedup/ChunkStore/{0DECAE8D-71D2-4BDE-8798-530201C72D8D}.ddp/Stream/00120000.00000002.ccc >> >> >> Jan 17 11:04:57 backup ntfs-3g[5032]: Failed to open a dedup stream last >> try was System Volume >> Information/Dedup/ChunkStore/{0DECAE8D-71D2-4BDE-8798-530201C72D8D}.ddp/Stream/00120000.00000002.ccc >> >> >> >> >> root@backup:~# getfattr -h -e hex -n system.ntfs_reparse_data >> /mnt/sr7-sdb2/ALGEMEEN/2009-12-17\ Index\ mappenstructuur.txt >> getfattr: Removing leading '/' from absolute path names >> # file: mnt/sr7-sdb2/ALGEMEEN/2009-12-17 Index mappenstructuur.txt >> system.ntfs_reparse_data=0x1300008000010000020100010b0000000400000004006000030000000400640003000000040068000600000008006c00090000001000740005000000080084000600000008008c000a0000002000d40008000000400094000000000000000000050000000800f400000000000c0000000000000000000000000000008daeec0dd271de4b8798530201c72d8d60003fd1bb66d20127190000000012002719000000001200b08c8b0005000000010000004804000088010000000000009427b7d5fcd18238484f1121abc9175a202e08000000000000000000000000009f238a585b38f16276213e2cc9e63d3769232008d45ba70a03f02e356e463c680000000000000000d31118e8 >> >> >> >> >> On 10/01/17 17:45, Jean-Pierre André wrote: >>> Hi again, >>> >>> Jelle de Jong wrote: >>>> Dear Jean-Pierre André, >>>> >>>> root@backup:~# ls -hal >>>> /usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-80000013.so >>>> -rw-r--r-- 1 root root 16K Jan 10 13:34 >>>> /usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-80000013.so >>>> >>>> How do I know if ntfs-3g is using the plug-in? >>> >>> The plugin is dynamically loaded when you first access >>> a deduplicated file. This is recorded in the syslog. >>> >>> Note : you should probably make it executable (e.g. by chmod 755) >>> >>>> >>>> lvcreate --snapshot --name sr7-disk2-snapshot-copy --size 50G >>>> /dev/lvm1-vol/sr7-disk2 >>>> lvchange -ay /dev/lvm1-vol/sr7-disk2-snapshot-copy >>>> blkid /dev/lvm1-vol/sr7-disk2-snapshot-copy >>>> kpartx -avg /dev/lvm1-vol/sr7-disk2-snapshot-copy >>>> blkid /dev/mapper/lvm1--vol-sr7--disk2--snapshot--copy2 >>>> mount -o ro -t ntfs-3g >>>> /dev/mapper/lvm1--vol-sr7--disk2--snapshot--copy2 /mnt/sr7-sdb2 >>>> ls -hal /mnt/sr7-sdb2/ALGEMEEN/ | grep "unsupported reparse point" >>>> >>>> Shows no more unsupported reparse point! >>> >>> This might be a good indication that the plugin is used. >>> >>> Did you check whether the files are readable ? >>> >>>> umount /mnt/sr7-sdb2/ >>>> kpartx -dv /dev/lvm1-vol/sr7-disk2-snapshot-copy >>>> lvchange -an /dev/lvm1-vol/sr7-disk2-snapshot-copy >>>> lvremove /dev/lvm1-vol/sr7-disk2-snapshot-copy >>>> >>>> root@backup:~# fgrep ntfs-3g /var/log/syslog >>>> Jan 10 16:14:28 backup ntfs-3g[13082]: Version 2016.2.22AR.1 integrated >>>> FUSE 28 >>>> Jan 10 16:14:28 backup ntfs-3g[13082]: Mounted >>>> /dev/mapper/lvm1--vol-sr7--disk2--snapshot--copy2 (Read-Only, label >>>> "DATA", NTFS 3.1) >>>> Jan 10 16:14:28 backup ntfs-3g[13082]: Cmdline options: ro >>>> Jan 10 16:14:28 backup ntfs-3g[13082]: Mount options: >>>> ro,allow_other,nonempty,relatime,fsname=/dev/mapper/lvm1--vol-sr7--disk2--snapshot--copy2,blkdev,blksize=4096 >>>> >>>> >>>> >>>> >>>> Jan 10 16:14:28 backup ntfs-3g[13082]: Ownership and permissions >>>> disabled, configuration type 7 >>>> Jan 10 16:23:32 backup ntfs-3g[13082]: Unmounting >>> >>> In this syslog excerpt I do not see the plugin loading, >>> did you access a deduplicated file before unmounting ? >>> >>>> /dev/mapper/lvm1--vol-sr7--disk2--snapshot--copy2 (DATA) >>>> >>>> However when using with the virt-filesystems and guestmount tools it >>>> does not work: >>>> >>>> root@backup:~# lvcreate --snapshot --name sr7-disk2-snapshot-copy >>>> --size >>>> 50G /dev/lvm1-vol/sr7-disk2 >>>> root@backup:~# lvchange -ay /dev/lvm1-vol/sr7-disk2-snapshot-copy >>>> >>>> root@backup:~# blkid /dev/lvm1-vol/sr7-disk2-snapshot-copy >>>> /dev/lvm1-vol/sr7-disk2-snapshot-copy: >>>> PTUUID="8d00ec46-cb6d-457f-bfc8-703089c83fb9" PTTYPE="gpt" >>>> >>>> root@backup:~# virt-filesystems -a >>>> /dev/lvm1-vol/sr7-disk2-snapshot-copy >>>> /dev/sda2 >>>> >>>> root@backup:~# guestmount --ro -a /dev/lvm1-vol/sr7-disk2-snapshot-copy >>>> -m /dev/sda2 /mnt/sr7-sda2 >>>> >>>> root@backup:~# ls -hal /mnt/sr7-sda2/ALGEMEEN/ | grep "unsupported >>>> reparse point" >>>> lrwxrwxrwx 1 root root 26 May 19 2008 080514.Index I Directory >>>> algemeen.xls -> unsupported reparse point >>>> lrwxrwxrwx 1 root root 26 Dec 17 2009 2009-12-17 Index >>>> mappenstructuur.txt -> unsupported reparse point >>>> lrwxrwxrwx 1 root root 26 May 2 2013 ber_folders -> unsupported >>>> reparse point >>>> >>>> # verbose output: >>>> root@backup:~# guestmount -vvvv --ro -a >>>> /dev/lvm1-vol/sr7-disk2-snapshot-copy -m /dev/sda2 /mnt/sr7-sda2 >>>> >>>> http://paste.debian.net/hidden/be1c989f/ >>>> >>>> I will sent an email to libgues...@redhat.com to ask what they think. >>> >>> Indeed, as some extra layer could come across, but first >>> check you can actually read files from a plain partition. >>> >>> Regards >>> >>> Jean-Pierre >>> >>>> >>>> Kind regards, >>>> >>>> Jelle de Jong >>>> >>>> On 10/01/17 12:44, Jean-Pierre André wrote: >>>>> Jelle de Jong wrote: >>>>>> Dear Jean-Pierre, >>>>>> >>>>>> We created a test environment with Windows 2016 with data >>>>>> deduplication on. >>>>>> >>>>>> I upgraded the backup server to install ntfs-3g 2016.2.22AR.1. I >>>>>> tried >>>>>> to mount the volume to see if I could read the files, but they >>>>>> show up >>>>>> as unsupported reparse point. >>>>>> >>>>>> I listed the Dedup/ChunkStore in the following pastebin: >>>>>> http://paste.debian.net/hidden/ca51ee46/ >>>>>> >>>>>> I tried to use the ntfs-plugin-80000013.so but I am not sure if it is >>>>>> being used, the bellow is not working so far? >>>>> >>>>> Apparently, the expected location on Debian is >>>>> /usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-80000013.so >>>>> but you are getting libntfs-3g from /lib/x86_64-linux-gnu >>>>> maybe there is a symlink. Anyway the same directory is >>>>> expected (check possible errors in your syslog). >>>>> >>>>> If you get errors, please only mount as read-only until >>>>> the issue is solved, choose a sample file of moderate size >>>>> (say 100KB), and post its reparse data, which you can >>>>> get by : >>>>> >>>>> getfattr -h -e hex -n system.ntfs_reparse_data your-sample-file >>>>> >>>>> Regards >>>>> >>>>> Jean-Pierre >>>> >>> >>> >> > ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ ntfs-3g-devel mailing list ntfs-3g-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ntfs-3g-devel
