If you are using RBD, and want to avoid potential image corruption, add
filestore fiemap = false
to the [osd] section of your ceph.conf and restart your OSDs.
We've tracked down the source of some corruption to racy/buggy FIEMAP
ioctl behavior. The RBD client (when caching is diabled--the default)
uses a 'sparse read' operation that the OSD implements by doing an fsync
on the object file, mapping which extents are allocated, and sending only
that data over the wire. We have observed incorrect/changing FIEMAP on
both btrfs:
fsync
fiemap returns mapping
<time passes, no modifications to file>
fiemap returns different mapping
Josh is still tracking down which kernels and file system are affected;
fortunately it is relatively easy to reproduce with the test_librbd_fsx
tool. In the meantime, the (mis)feature can be safely disabled. It will
default to off in 0.48. It is unclear whether it's really much of a
performance win anyway.
Thanks!
sage
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
