Jelle de Jong wrote: > Dear Jean-Pierre, > > I can now read the "algemeen.txt" file, thank you! > > However I still got an stream error on an other file.
You have a very interesting configuration for testing corner situations. I would not be able to get such a rich testbed without your cooperation. Unfortunately this leads to a lot of communication.... > http://paste.debian.net/plainh/cb081fa9 > > I did not get how you calculated that inode 1545290 was in cluster > 30817428, but assuming this is still the case here is the information: Normally, this calculation is not needed, just do an ntfsinfo, for example : ntfsinfo -fvi 1545290 /dev/... I asked you because you showed an inconsistency, and ntfsinfo could not display the contents (the inode had probably been freed). It is now apparently too late to catch the inconsistency. The explanation for the computation is : - divide 1545290 by 4, which gives 386322 or 0x5e513 - lookup for a run in mft which contains vcn 0x5e513 we find vcn 0x578ac lcn 0x1d5d02d length 0xc813 then lcn is 0x1d5d02d + (0x5e513 - 0x578ac) = 0x1d63c94 and 0x1d63c94 is 30817428 > > http://paste.debian.net/plainh/535a9df8 > > Thank you very much! > > If it is a quick fix I can still test it today, almost there. Probably not a quick fix. The reparse data points at offset 90622928, which is beyond the end of 00190000.00010002.ccc whose size is 87789056. Moreover the identification of the user file (D8A64E6D...) is not present in 00190000.00010002.ccc What comes to mind is that 00190000.00010002.ccc was being updated and a newer one should be used. Is there another 00190000.*.ccc in the same Stream directory ? Jean-Pierre > Kind regards, > > Jelle de Jong > > On 03/04/17 11:44, Jean-Pierre André wrote: >> Hi, >> >> Please try >> http://jp-andre.pagesperso-orange.fr/dedup123-beta.zip >> >> Regards >> >> Jean-Pierre >> >> Jean-Pierre André wrote: >>> Hi, >>> >>> I see what is happening here : this file "algemeen.txt" >>> is recorded the traditional way (like mine), not the >>> way most of your files are recorded. So I have to improve >>> the selection of modes. >>> >>> I will upload a fix next week. >>> >>> Was this file created recently ? Maybe, when a file is >>> updated, it is recorded differently from when it was created >>> initially. >>> >>> Regards >>> >>> Jean-Pierre >>> >>> Jelle de Jong wrote: >>>> Beste Jean-Pierre, >>>> >>>> #getfattr -h -e hex -n system.ntfs_reparse_data ... >>>> #dd if=/dev/mapper/.. skip=30817428 bs=4096 count=1 | od -t x1 >>>> #and the other commands outputs: >>>> https://powermail.nu/nextcloud/index.php/s/HwzpBqaIzOyZedg >>>> >>>> # stream.data.full.00190000.00010002.gz: >>>> https://powermail.nu/nextcloud/index.php/s/KKlByC8qEPKZFrC >>>> >>>> Thank you in advance, >>>> >>>> Kind regards, >>>> >>>> Jelle de Jong >>>> >>>> On 28/03/17 16:32, Jean-Pierre André wrote: >>>>> Jelle de Jong wrote: >>>>>> Hi Jean-Pierre, >>>>>> >>>>>> When trying to read algemeen.txt I get a reproducible "Bad stream for >>>>>> offset" (I got a few more files with the same issue) >>>>> >>>>> So, please post the reparse tag for this file algemeen.txt : >>>>> getfattr -h -e hex -n system.ntfs_reparse_data somepath/algemeen.txt >>>>> >>>>> Also please post the contents of file 1545290. I do not >>>>> know its name, but it must be in the Stream directory, >>>>> with a suffix .ccc. To get its name do : >>>>> ls -li *as*usual*/Stream/*.ccc | grep 1545290 >>>>> >>>>>> >>>>>> # ntfsinfo -fvi 0 /dev/mapper/lvm1--vol-sr7--disk2--snapshot--copy2 >>>>>> http://paste.debian.net/plainh/486b1a01 >>>>> >>>>> So inode 1582022 must be in cluster 30826611 and >>>>> inode 1545290 in cluster 30817428. >>>>> Please post the outputs of : >>>>> dd if=/dev/mapper/lvm1--vol* skip=30826611 bs=4096 count=1 | od -t x1 >>>>> dd if=/dev/mapper/lvm1--vol* skip=30817428 bs=4096 count=1 | od -t x1 >>>>> >>>>> The sequencing error which had been detected may have >>>>> disappeared with no evidence left, but it is worth >>>>> trying. >>>>> >>>>>> I feel we may be almost there (support for data deduplication), but >>>>>> still hitting a few unreadable files... >>>>> >>>>> I hope so, this is an uneasy situation for me, as I >>>>> cannot reproduce the errors. Thanks for you cooperation >>>>> to the bug hunt. >>>>> >>>>> Jean-Pierre >>>>> >>>>>> >>>>>> Thanks you again! >>>>>> >>>>>> Kind regards, >>>>>> >>>>>> Jelle de Jong >>>>>> >>>>>> >>>>>> On 23/03/17 22:03, Jean-Pierre André wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Jelle de Jong wrote: >>>>>>>> Hi Jean-Pierre, >>>>>>>> >>>>>>>> Thank you again! >>>>>>>> >>>>>>>> Could you take a look at the following output, I got some "Bad >>>>>>>> stream >>>>>>>> for offset" messages: http://paste.debian.net/plainh/c145e066 >>>>>>>> >>>>>>>> Follow-up on the previous mail >>>>>>>> >>>>>>>> I was not able to get any node information, and have not been >>>>>>>> able to >>>>>>>> reproduce the NTFS messages. >>>>>>>> >>>>>>>> ntfsinfo -fvi 1582022 >>>>>>>> /dev/mapper/lvm1--vol-sr7--disk2--snapshot--copy2 >>>>>>>> Error loading node: No such file or directory >>>>>>> >>>>>>> Before digging into the "Bad stream for offset" messages, >>>>>>> I would like to get details about those inconsistencies >>>>>>> before they are wiped out (hoping it is not too late). >>>>>>> Maybe the issues are related, maybe they are not. >>>>>>> >>>>>>> The ntfsinfo error means either that the faulty inodes >>>>>>> have been freed or they are extents to other inodes. >>>>>>> >>>>>>> I will have to do it in two steps : first determine where >>>>>>> they are located, then extract their contents. >>>>>>> >>>>>>> To get the mapping of inodes, please post the output of >>>>>>> ntfsinfo -fvi 0 /dev/mapper/lvm1--vol-sr7* >>>>>>> >>>>>>> Jean-Pierre >>>>>>> >>>>>>>> >>>>>>>> Kind regards, >>>>>>>> >>>>>>>> Jelle de Jong >>>>>>>> >>>>>>>> On 21/03/17 12:01, Jean-Pierre André wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Jelle de Jong wrote: >>>>>>>>>> Dear Jean-Pierre, >>>>>>>>>> >>>>>>>>>> I am still testing the dedup plugin, I have issues with >>>>>>>>>> rdiff-backup >>>>>>>>>> that the checksum of the source and destination are not the same, >>>>>>>>>> but >>>>>>>>>> this can be an issue with rdiff-backup. >>>>>>>>>> >>>>>>>>>> However I did get these messages, but I do not know if they are >>>>>>>>>> related >>>>>>>>>> to the dedup plugin? >>>>>>>>>> >>>>>>>>>> Mar 21 06:57:37 backup ntfs-3g[12620]: Record 1582022 has wrong >>>>>>>>>> SeqNo >>>>>>>>>> (159 <> 156) >>>>>>>>>> Mar 21 06:57:37 backup ntfs-3g[12620]: Could not decode the >>>>>>>>>> type of >>>>>>>>>> inode 1582022 >>>>>>>>>> Mar 21 06:57:37 backup ntfs-3g[12620]: Record 1545290 has wrong >>>>>>>>>> SeqNo >>>>>>>>>> (296 <> 295) >>>>>>>>>> Mar 21 06:57:37 backup ntfs-3g[12620]: Could not decode the >>>>>>>>>> type of >>>>>>>>>> inode 1545290 >>>>>>>>> >>>>>>>>> These errors are thrown by ntfs-3g proper, not by the >>>>>>>>> deduplication plugin. >>>>>>>>> >>>>>>>>> Inconsistencies have been found in the mentioned files, >>>>>>>>> and they are unlikely to be created by the plugin (which >>>>>>>>> does not have updating support), ... but one never knows. >>>>>>>>> Owing to the inconsistencies, ntfs-3g cannot tell if the >>>>>>>>> said files are plain files, directories, sockets, etc. >>>>>>>>> >>>>>>>>> You have to run chkdsk to fix these inconsistencies, but >>>>>>>>> this can lead to losing data, so before doing that, the >>>>>>>>> parameters of these files should be analyzed to assess >>>>>>>>> the possible damage. >>>>>>>>> >>>>>>>>> Please post the outputs of : >>>>>>>>> ntfsinfo -fvi 1582022 /dev/partition >>>>>>>>> ntfsinfo -fvi 1545290 /dev/partition >>>>>>>>> >>>>>>>>> This must be done as root, replacing /dev/partition by >>>>>>>>> the actual partition path (something like /dev/sdc2) >>>>>>>>> >>>>>>>>> Jean-Pierre >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Kind regards, >>>>>>>>>> >>>>>>>>>> Jelle de Jong >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >>> >>> >> >> > ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ ntfs-3g-devel mailing list ntfs-3g-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ntfs-3g-devel
