> On Oct. 15, 2018, 2:21 p.m., Benjamin Bannier wrote: > > 3rdparty/stout/include/stout/os/posix/rename.hpp > > Lines 41 (patched) > > <https://reviews.apache.org/r/69009/diff/2/?file=2097010#file2097010line41> > > > > While POSIX guarantees that `rename` e.g., does not see inconsistent > > state, there is nothing preventing `to` from being deleted once we execute > > the conditional code here. > > > > Since it is hard to know what semantics users expect _in general_, it > > might make more sense to not add the sync behavior to `rename`, but to > > e.g., ask users to perform `fsync` themself. > > Chun-Hung Hsiao wrote: > I considered this before. But `fsync`ing the directory is a > POSIX-specific thing, then the caller (e.g. `slave.cpp`) needs to write two > snippets for POSIX and Windows. Adding the `sync` option seems a better > option for making the code cross-platform. > > Also I don't understand your concern about the deletion. Can you > elaborate? > > Benjamin Bannier wrote: > Re:deletion, imagine us working in a temp dir which is aggressively > garbage-collected. We `::rename` `/tmp/mesos/a` to `/tmp/mesos/b`, but before > we `os::fsync` the directory `/tmp/mesos` gets garbage-collected; this would > make the `os::open` fail, even though the `::rename` finished successfully. > > It is not immediately obvious to me when such a scenario is a failure > when `sync=true`, which lead me to suggest to let callers handle the `sync` > (we could introduce a helper `sync(const& Path)` in addition to > `sync(int_fd)`. > > Jie Yu wrote: > I also prefer introducing sync behavior in rename, for two reasons: > 1) consistent with windows API > 2) force the caller to think if `sync` is needed or not. If we ask > callers to do sync as a second step, my worry is that most developers will > forget.
Re:deletion, how about adding a comment stating that `sync=true` is not thread-safe? - Chun-Hung ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69009/#review209538 ----------------------------------------------------------- On Oct. 12, 2018, 11:53 p.m., Chun-Hung Hsiao wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69009/ > ----------------------------------------------------------- > > (Updated Oct. 12, 2018, 11:53 p.m.) > > > Review request for mesos, Andrew Schwartzmeyer, Benjamin Bannier, Jie Yu, and > Jan Schlicht. > > > Bugs: MESOS-9281 > https://issues.apache.org/jira/browse/MESOS-9281 > > > Repository: mesos > > > Description > ------- > > This patch adds an option for the caller to sync the file and directory > contents to the disk after a write to prevent filesystem inconsistency > against reboots. > > > Diffs > ----- > > 3rdparty/stout/include/stout/os/posix/rename.hpp > 9cff6db16c8a4e5a79bf0082e18a1133bd287796 > 3rdparty/stout/include/stout/os/windows/rename.hpp > 523912ac3bf315f70f542e8eab7d2d02249909b4 > 3rdparty/stout/include/stout/os/write.hpp > cd35285a9595004bac627abf687588050aef8161 > 3rdparty/stout/include/stout/protobuf.hpp > 1d03e1e3a8dd642f7239d777fb04759caf100a8b > > > Diff: https://reviews.apache.org/r/69009/diff/2/ > > > Testing > ------- > > make check > > > Thanks, > > Chun-Hung Hsiao > >
