Hello list!
I was hoping to ask about any changes to decmpfs_file_is_compressed() in
Catalina as I've just spent 3 weeks chasing down a problem with random
memory corruption.
The stack looks like:
verify_events(NULL);
spl_decmpfs_cnode_init(cp);
verify_events(NULL);
/* _is_compressed() will getattr to make sure UF_COMPRESSED */
iscompressed = spl_decmpfs_file_is_compressed(ap->a_vp, cp);
~~~
_decmpfs_file_is_compressed + 0x263
_vnode_getattr + 0xae
zfs_vnop_getattr + 0x24
~~~
verify_events(NULL);
(lldb) if (curr == (void *)0x0000200721000000ULL)
==> panic("triggered");
The repeated call to `verify_events()` runs through all lists, and memory,
to check for corruption. The call just before
`decmpfs_file_is_compressed()` succeeds.
`decmpfs_file_is_compressed()` then calls `vnode_getattr()` to confirm
UF_COMPRESSED is set, and at the very start of our getattr
`zfs_vnop_getattr()` we call `verify_events()` and detect memory corruption.
Interestingly, it is always writing the value 0x0000200721000000 at a
(pseudo) random location.
Usually, I would simply check the sources for xnu-6153.0.139.161.2 - but I
don't think they are available yet. But perhaps there were changes to the
number/type of arguments it takes? It would be nice to be able to fix my
side before Catalina going public.
History (Feel free to skip):
* We do not advertise VOL_CAP_FMT_DECMPFS_COMPRESSION but unfortunately,
the AppleFSCompression library (as used by many Applications like 'Mail'
etc) totally ignores the lack of this property, and will attempt/write
decmpfs anyway.
* If we try to stop it, by returning error to setattr(UF_COMPRESSED) it
ignores the errors, and keeps going
* If we try to stop it, by returning error on XATTR create/write of
"com.apple.decmpfs" or "com.apple.ResourceFork" - it ignores the error, and
keeps going
* Then it truncates the file to 0. "Losing" the file data.
Currently, we handle this by playing-along with userland, but as soon as
the file is truncated it calls setattr(UF_COMPRESSED).
We then decmpfs decompress the file data back, and delete the xattr data.
Since the compression level is a user defined property for our filesystem,
and users can set it to disabled, lz4, gzip-9 and so on. So both to honour
the users' choice, and that some compression types have better performance.
This is rather frustrating, especially since there is a
VOL_CAP_FMT_DECMPFS_COMPRESSION property. (It is somewhat alarming that all
errors are just ignored as well) It forces us to support decmpfs, even if
we don't want to.
We felt it best to use XNU's built-in decmpfs, to avoid code duplication
and possible future case of Apple updating/changing it.
Unfortunately, all decmpfs function calls are not exported, so we aren't
really allowed to use them - we just do anyway - so that is a risk I took
and that's on me, it didn't work out this time.
So perhaps the only answer is to copy/paste the decmpfs code over :(
Lund
--
Jorgen Lundman | <lund...@lundman.net>
Unix Administrator | +81 (0)90-5578-8500
Shibuya-ku, Tokyo | Japan
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
