> On Oct. 15, 2018, 2:21 p.m., Benjamin Bannier wrote: > > 3rdparty/stout/include/stout/os/posix/rename.hpp > > Lines 41 (patched) > > <https://reviews.apache.org/r/69009/diff/2/?file=2097010#file2097010line41> > > > > While POSIX guarantees that `rename` e.g., does not see inconsistent > > state, there is nothing preventing `to` from being deleted once we execute > > the conditional code here. > > > > Since it is hard to know what semantics users expect _in general_, it > > might make more sense to not add the sync behavior to `rename`, but to > > e.g., ask users to perform `fsync` themself.
I considered this before. But `fsync`ing the directory is a POSIX-specific thing, then the caller (e.g. `slave.cpp`) needs to write two snippets for POSIX and Windows. Adding the `sync` option seems a better option for making the code cross-platform. Also I don't understand your concern about the deletion. Can you elaborate? > On Oct. 15, 2018, 2:21 p.m., Benjamin Bannier wrote: > > 3rdparty/stout/include/stout/os/write.hpp > > Lines 119 (patched) > > <https://reviews.apache.org/r/69009/diff/2/?file=2097012#file2097012line119> > > > > I personally find flag arguments hard to read (`write(.., .., false)` > > or `write(.., .., true)` are not obvious). I'd suggest to either introduce > > a dedicated `write_sync` function or let callers trigger the `fsync`. I agree that flag arguments sometimes are not easy to use but in this case it seems fine to me. Let me also discuss this with Jie. > On Oct. 15, 2018, 2:21 p.m., Benjamin Bannier wrote: > > 3rdparty/stout/include/stout/os/write.hpp > > Lines 133-134 (patched) > > <https://reviews.apache.org/r/69009/diff/2/?file=2097012#file2097012line133> > > > > Is the performance impact in the comment common knowledge? I would have > > expected that implementation asked to `write` with `O_SYNC` could > > internally perform the equivalent of what we are doing by hand here. > > > > If possible just use `O_SYNC` with `os::open` above. I've seen some benchmark about `O_SYNC` (which syncs for every write`) vs one single `fsync`. Let me find it later. - Chun-Hung ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69009/#review209538 ----------------------------------------------------------- On Oct. 12, 2018, 11:53 p.m., Chun-Hung Hsiao wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69009/ > ----------------------------------------------------------- > > (Updated Oct. 12, 2018, 11:53 p.m.) > > > Review request for mesos, Andrew Schwartzmeyer, Benjamin Bannier, Jie Yu, and > Jan Schlicht. > > > Bugs: MESOS-9281 > https://issues.apache.org/jira/browse/MESOS-9281 > > > Repository: mesos > > > Description > ------- > > This patch adds an option for the caller to sync the file and directory > contents to the disk after a write to prevent filesystem inconsistency > against reboots. > > > Diffs > ----- > > 3rdparty/stout/include/stout/os/posix/rename.hpp > 9cff6db16c8a4e5a79bf0082e18a1133bd287796 > 3rdparty/stout/include/stout/os/windows/rename.hpp > 523912ac3bf315f70f542e8eab7d2d02249909b4 > 3rdparty/stout/include/stout/os/write.hpp > cd35285a9595004bac627abf687588050aef8161 > 3rdparty/stout/include/stout/protobuf.hpp > 1d03e1e3a8dd642f7239d777fb04759caf100a8b > > > Diff: https://reviews.apache.org/r/69009/diff/2/ > > > Testing > ------- > > make check > > > Thanks, > > Chun-Hung Hsiao > >
