Hi all, Most of us would agree that way we fsync FS changes in FSFS and FSX is slow (~10 commits / sec on a SSD, YMMV) and not even all changes are fully fsync'ed (repo creation, upgrade).
>From a high-level perspective, a commit is are simple 3-step process: 1. Write rev contents & props to their final location. They are not accessible until 'current' gets bumped. Write a the new 'current' temporary contents to a temp file. 2. Fsync everything we wrote in step 1 to disk. Still not visible to any other FS API user. 3. Atomically switch to the new 'current' contents and fsync that change. Today, we fsync "locally" as part of whatever copy or move operation we need. That is inefficient because on POSIX platforms, we tend to fsync the same folder more than once, on Windows we would need to sync the files twice (content before and metadata after the rename). Basically, missing the context info, we need to play it safe and do much more work than we would actually have to. In the future, we should implement step 1 as simple non-fsync'ing file operations. Then explicitly sync every file, and on POSIX the folders, once. Step 2 does not have any atomicity requirements. Finally, do the 'current' rename. This also only requires a single fsync b/c the temp file will be in the same folder. On top of that, all operations in step 2 can be run concurrently. I did that for FSX on Linux using aio_fsync and it got 3x as fast. Windows can do something similar. I wrapped that functionality into a "batch_fsync" object with a few methods on it. You simply push paths into it, it drops duplicates, and finally you ask it to fsync all. To get even faster, a FS instance can piggy-back the allocate a new txn ID onto this process. In normal operation, the svn_fs_t will be cleaned up and the txn ID be wasted. In 'svnadmin load', however, we save the 2x fsync dance with the 'txn-current' file. So, I suggest to implement svn_io__batch_fsync_t and use that for all durable FS modification, explicitly excluding in-txn operations. Does that sound like a plan? -- Stefan^2.
