Dear Jean-Pierre, I thought version 1.2.1 of the plug-in was working so I took it further into production, but during backups with rdiff-backup and guestmount it created a 100% cpu load in qemu process that stayed there for days until I killed them, I tested this twice. So I went back to a xpart/mount -t ntfs command and found more "Bad stream for offset" and found that the /sbin/mount.ntfs-3g command was running at 100% cpu load and hanged there.
I have added the whole Stream directory here: (1.1GB) https://powermail.nu/nextcloud/index.php/s/vbq85qZ2wcVYxrG Separate stream file: stream.data.full.000c0000.00020001.gz https://powermail.nu/nextcloud/index.php/s/QinV51XE4jrAH7a All the commands I used: http://paste.debian.net/plainh/c0ea5950 I do not know how to get the reparse tags of all the files, maybe you can help me how to get all the information you need. Thank you for your help! Kind regards, Jelle de Jong On 14/02/17 15:55, Jean-Pierre André wrote: > Hi, > > Jelle de Jong wrote: >> Hi Jean-Pierre, >> >> If we have to switch to Windows 2012 and thereby having an environment >> similar to yours then we can switch to an other Windows version. > > I do not have any Windows Server, and my analysis > and tests are based on an unofficial deduplication > package which was adapted to Windows 10 Pro. > > A few months ago, following a bug report, I had to > make changes for Windows Server 2012 which uses an > older data format, and my only experience about this > format is related to this report. So switching to > Windows 2012 is not guaranteed to make debugging easier. > >> We are running out of disk space here so if switching Windows versions >> makes the process of having data deduplication working easer then me >> know. > > I have not yet analyzed your latest report, but it > would probably be useful I build a full copy of > non-user data from your partition : > - the reparse tags of all your files, > - all the "*.ccc" files in the Stream directory > > Do not do it now, I must first dig into the data you > posted. > > Regards > > Jean-Pierre > > >> Kind regards, >> >> Jelle de Jong >> >> On 09/02/17 13:46, Jelle de Jong wrote: >>> Hi Jean-Pierre, >>> >>> In case you are wondering: >>> >>> I am using data deduplication in Windows 2016 for my test environment >>> iso: >>> SW_DVD9_Win_Svr_STD_Core_and_DataCtr_Core_2016_64Bit_English_-2_MLF_X21-22843.ISO >>> >>> >>> Kind regards, >>> >>> Jelle de Jong >>> >>> On 09/02/17 11:41, Jean-Pierre André wrote: >>>> Hi, >>>> >>>> Jelle de Jong wrote: >>>>> Hi Jean-Pierre, >>>>> >>>>> Thank you! >>>>> >>>>> The new plug-in seems to work for now, I am moving it into testing >>>>> phase >>>>> with-in our production back-up scripts. >>>> >>>> Please wait a few hours, I have found a bug which >>>> I have fixed. I am currently inserting your data >>>> into my test base in order to rerun all my tests. >>>> >>>>> Will you release the source code eventually, would like to write a >>>>> blog >>>>> post about how to add the support. >>>> >>>> What exactly do you mean ? If it is about how to >>>> collect the data in a unsupported condition, it is >>>> difficult, because unsupported generally means >>>> unknown territory... >>>> >>>>> What do you think the changes are of the plug-in stop working again? >>>> >>>> (assuming a typo changes -> chances) >>>> Your files were in a condition not met before : data >>>> has been relocated according to a logic I do not fully >>>> understand. Maybe this is an intermediate step in the >>>> process of updating the files, anyway this can happen. >>>> >>>> The situation I am facing is that I have a single >>>> example from which it is difficult to derive the rules. >>>> So yes, the plugin may stop working again. >>>> >>>> Note : there are strict consistency checks in the plugin, >>>> so it is unlikely you read invalid data. Moreover if >>>> you only mount read-only you cannot damage the deduplicated >>>> partition. >>>> >>>>> We do not have an automatic test running to verify the back-ups at >>>>> this >>>>> moment _yet_, so if the plug-in stops working, incremental file-based >>>>> back-ups with empty files will slowly get in the back-ups this way :| >>>> >>>> Usually a deduplicated partition is only used for backups, >>>> and reading from backups is only for recovering former >>>> versions of files (on demand). >>>> >>>> If you access deduplicated files with no human control, >>>> you have to insert your own checks in the process. I >>>> would at least check whether the size of the recovered >>>> file is the same as the deduplicated one (also grep for >>>> messages in the syslog). >>>> >>>> Regards >>>> >>>> Jean-Pierre >>>> >>>>> Again thank you for all your help so far! >>>>> >>>>> Kind regards, >>>>> >>>>> Jelle de Jong >>>>> >>>>> >>>>> On 08/02/17 15:59, Jean-Pierre André wrote: >>>>>> Hi, >>>>>> >>>>>> Can you please make a try with : >>>>>> http://jp-andre.pagesperso-orange.fr/dedup120-beta.zip >>>>>> >>>>>> This is experimental and based on assumptions which have >>>>>> to be clarified, but it should work in your environment. >>>>>> >>>>>> Regards >>>>>> >>>>>> Jean-Pierre >>>> >>>> >>> >>> ------------------------------------------------------------------------------ >>> >>> Check out the vibrant tech community on one of the world's most >>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot >>> _______________________________________________ >>> ntfs-3g-devel mailing list >>> ntfs-3g-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/ntfs-3g-devel >>> >> >> ------------------------------------------------------------------------------ >> >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, SlashDot.org! http://sdm.link/slashdot >> _______________________________________________ >> ntfs-3g-devel mailing list >> ntfs-3g-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/ntfs-3g-devel >> > > ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ ntfs-3g-devel mailing list ntfs-3g-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ntfs-3g-devel
