Hi Daniel! Apologies for the late reply. I had seen your message back then but it somehow fell of the table and eventually forgot to answer. I just now found your report again since I resumed working on the hfsprogs package.
On Mon, 2021-09-27 at 12:45 +0200, Daniel Höpfl wrote: > Apple (the upstream) never finished the implementation of the directory > hardlink support > in fsck_hfs. Since, once again, my TimeMachine backup rendered unrepairable > due to directory > hardlink errors, I implemented what was required to fix the errors I > encountered. The current hfsprogs package in Debian is based on a patched version 540.1 of Apple's »diskdev_cmds« package [1] while Apple has actually continued their work on the HFS code. in that package. However, after version 557.3.1 Apple decided to move the HFS code into a separate package called just »hfs« [2] which makes sense given the fact that Apple switched their default filesystem to APFS instead and diskdev_cms is no longer strictly tied to HFS. I have started working on an updated Debian hfsprogs package which is based on the latest version 556.100.11 of the hfs package [3]. The code already compiles in openSUSE [4] but I don't consider the Linux port to be complete yet. However, you may want to try whether the latest version of Apple's own fsck_hfs has support for fixing hardlink problems. If not, you may want to rebase your patch on top of hfs 560.100.11. > My trust in Apple's feedback system is very low so I am sending the patch to > you in the hopes > that you can forward it to Apple. Try opening a pull request on Github [5]. Apple has actually moved their open source projects to Github and it seems they're actually accepting pull requests for some projects. > > The patch I added should work, with offsets, on both, Debian's sources as > well as Apple's sources. > > My changes: > > - Any hardlink inode with a parent other than the metadata directory is > changed > to have the correct parent. > I have seen these. I can understand how this happens when a normal directory > is hardlined for the first time. The journal SHOULD prevent this from being > a > problem but obviously this is not the case. > > - Directory hardlink inodes named "temp..." get deleted. > These directories were deleted while open when the volume was removed > without > unmounting. Since these directories have already been deleted, deleting them > in fsck is the right thing to do. (Documented like that for files in the > HFS+ > spec.) > > - Directory hardlink inodes with names other than "temp..." or "dir_..." are > moved > to "lost+found". > It would be better to rename them to "dir_..." in a first pass, there might > still > be valid links to them. These links are deleted by a different repair step > and I > did not see how to remove those other repair steps. Would probably lead to > a bigger > rewrite of the hardlink check code so I decided to just move these to l+f. > I have never seen this anyways. > > After all, none of the errors should happen in a journaled filesystem > (directory > hardlinks require the Journal to be active). My guess is that using HFS+ over > AFP > (as TimeMachine does) does not correctly persist the journal before > persisting changes. > Thus loosing the connection during directory hardlink operation can result in > partially > executed atomic operation. Sounds like a very useful and welcome improvement. I will definitely incorporate this patch once I'm ready with the new upstream version. But maybe we're lucky and you can get your changes merged upstream with Apple in the mean time. FWIW, I have also reached out to Apple Legal trying to convince them to relicense the hfs package to a more permissive license such as the Apache License like they have done for other projects such as mDNSResponder [6]. Apple's own APSL license is considered non-free by Debian [7] which makes redistribution of the package more complicated. > I will attach the patch and test cases for each of the three fixed errors. > These test > cases have been constructed in a hex editor, they might not be what a real > error looks > like. The first two repairs successfully repaired my broken backup file so I > guess > they work (sometimes). Thanks, having test cases will be very useful for verifying my patched hfs package! Adrian > [1] https://github.com/apple-opensource/diskdev_cmds/tree/540.1 > [2] https://github.com/apple-opensource/hfs > [3] https://github.com/glaubitz/hfs/tree/linux > [4] > https://build.opensuse.org/package/show/home:glaubitz:branches:filesystems/hfsplus-tools > [5] https://github.com/apple-opensource/hfs > [6] https://github.com/apple-oss-distributions/mDNSResponder/ > [7] > https://wiki.debian.org/DFSGLicenses#Apple_Public_Source_License_.28APSL.29 -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer `. `' Physicist `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
