Hello Adar Dembo,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/9043
to review the following change.
Change subject: KUDU-2195 (part 1): always sync PBC-format metadata files
......................................................................
KUDU-2195 (part 1): always sync PBC-format metadata files
This changes the writing of metadata files to always fsync data before
renaming into place. Previously, we didn't do this in a few cases, most
notably for consensus metadata when WAL fsync is off (the default).
However, ext4 already has some automatic fsync when it detects the
commmon "write and rename-to-replace" paradigm used by a lot of
software. So, adding the explicit fsync isn't likely to slow down ext4
much.
More importantly, xfs does _not_ have this behavior, and we've recently
seen some issues on an xfs-enabled system where consensus metadata files
were unexpectedly zero-length or appear to have lost term changes,
causing tablets to fail to initialize. Initial investigation seems to
indicate that the hosts with these issues had experienced some hard
resets, lending further credence to the theory that it was due to lack
of syncing metadata.
Lots of good background reading can be found here (particularly in the
comments):
https://lwn.net/Articles/351422/
Change-Id: I4f0c911662b2ff35fcf3915790248ba85bf6026f
---
M src/kudu/consensus/consensus_meta.cc
M src/kudu/fs/block_manager_util.cc
M src/kudu/fs/data_dirs.cc
M src/kudu/fs/fs_manager.cc
M src/kudu/integration-tests/external_mini_cluster_fs_inspector.cc
M src/kudu/server/server_base.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/util/pb_util-test.cc
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
11 files changed, 48 insertions(+), 76 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/43/9043/1
--
To view, visit http://gerrit.cloudera.org:8080/9043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4f0c911662b2ff35fcf3915790248ba85bf6026f
Gerrit-Change-Number: 9043
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
