I've received off-list help on this problem, which has been invaluable! Turns out it isn't really a problem of "safe save" - we already handle DOCUMENT_ID, and the related IOCTLs including transfer, as well as a version of "tombstones" that suits us.
It appears to be the HFS+ file compression "decmpfs". For a given source file, it appears that decmpfs will create a named stream (vnop_makenamedstream) named "com.apple.ResourceFork" where it saves the compressed version. It also saves a "compression header" (vnop_setxattr) as "com.apple.decmpfs". The next step is truncating the (real? data? source?) file to 0 byte length, and finally sets (chflags / vnop_setattr) UF_COMPRESSED. So for us, we would have an empty 0 byte length file and everything would fail, like Mail trying to send mail. In HFS, it has additional logic in vnop_read to detect that the file is compressed, and decompress it into UPL first. There is additional logic in vnop_getattr to handle returning the filesize, as well as in vnop_write to decompress the file completely should it be written to. This logic was/is missing for us. As we did not advertise VOL_CAP_FMT_DECMPFS_COMPRESSION, we ignored the compression paths in HFS (we totally should not have ignored that code) Indeed when using "ditto --hfsCompression" as a tester tool, ditto will skip compression if VOL_CAP_FMT_DECMPFS_COMPRESSION is not set. Unfortunately, this is not true of "Mail". It will compress no matter what. We tried returning errors in vnop_makenamedstream, vnop_setxattr, and vnop_setattr, but all errors are ignored and it will carrying on with compression and things will fail. For now, we have managed to give temporary support to the situation. As the last task it performs is to set UF_COMPRESSED, we now add these lines: cp = spl_decmpfs_cnode_alloc(); spl_decmpfs_cnode_init(cp); iscompressed = spl_decmpfs_file_is_compressed(ap->a_vp, cp); if (iscompressed == 1) error = spl_decmpfs_decompress_file(ap->a_vp, cp, /*tosize*/ -1, /*truncateok*/ 1, /*skiplock*/ 1 ); spl_decmpfs_cnode_destroy(cp); spl_decmpfs_cnode_free(cp); It is a shame to decompress it immediately, but it was the quickest solution "for now" (watch it become permanent!), while we ponder a proper implementation. Ultimately, we do not really want decmpfs, as the user can pick better compression, like that of gzip-9 (I mean, if they really wanted to..) None of the decmpfs_*() functions are exported, but that is a separate problem. I would never been able to figure this out on my own, so big thanks to the hints I received. We still have some incompatibilities to solve, why "bird" repeatedly crashes, and fonts are considered invalid, but we have to leave something to tinker with on a rainy day. Lund Jorgen Lundman wrote: > > Hello list, > > So with 10.13 we have now noticed that Mail also uses what I would call > "safe save" feature when sending email, I'm not sure what it is really > called? Versioning? AtomicFile? It is something we have not been able to > unravel for ZFS of yet, but so far we have lived without it. Mail makes > that a bit awkward. We just get a completely-empty email when you click > "send" and sending fails. Clicking Edit-Message then just shows empty message. > > So I'm wondering if I am allowed to know what underlying feature that > mechanism uses. I have for a while tried to pin it down with dtrace, > perhaps VNOP_EXCHANGE, or VNOP_CLONEFILE, or VNOP_COPYFILE, or renamex > (RENAME_SWAP), or ATTR_VOL_DOCUMENT_ID with UF_TRACKED, > HFSIOC_TRANSFER_DOCUMENT_ID, or ... I have checked quite a few other things... > > Dtrace has not given me any clues as to what we are missing. It could even > be my xmas present! > > Sincerely, > > Lund > > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Filesystem-dev mailing list (Filesystem-dev@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > https://lists.apple.com/mailman/options/filesystem-dev/lundman%40lundman.net > > This email sent to lund...@lundman.net > -- Jorgen Lundman | <lund...@lundman.net> Unix Administrator | +81 (0)90-5578-8500 Shibuya-ku, Tokyo | Japan _______________________________________________ Do not post admin requests to the list. They will be ignored. Filesystem-dev mailing list (Filesystem-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com