Hi, I wonder what is the recommended way to do safe file delivery (the process which doesn't allow partial files and at the end the file is guaranteed to be in its place) with zfs?
Traditionally this is recommended: 1. write the file into a temporary file (possibly in a different directory than the destination, but on the same file system, because of rename) 2. fsync the file 3. rename the file into its final destination 4. fsync the final destination's directory Are there any better (in terms of performance) options here, given how zfs works? Also, if I remember correctly, zfs in the early years flushed everything in the currently open txg if an fsync was requested for any of the files/records involved in it. This was of course suboptimal for a lot of use cases, but when your only workload is to synchronously write files, it could be used as a major optimization: - you can write files in many threads/processes - do an fsync in any of them and you can be sure (?) that everything up to that point is persisted safely Theoretically (and not taking a lot of real world stuff into account): in the first case, if you have a drive which is capable of 100 IOPS, you can write around 50-100 files per second, each of them will have the latency of 10/20 ms. But in the second case you could do around 100 batches per second and if 10 files can fit into one batch, you'll have 1000 files per second, with around the same latency. (of course many things will affect this, but the throughput will be better) Is it possible to restore (or if I remember badly, achieve) that behaviour with current zfs versions? ------------------------------------------ openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/discussions/Tf07dc15aceb104fb-M618e18bbc5d80fe9c070e54f Delivery options: https://openzfs.topicbox.com/groups
