[
https://issues.apache.org/jira/browse/KUDU-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17461131#comment-17461131
]
ASF subversion and git services commented on KUDU-2195:
-------------------------------------------------------
Commit 6908620821ab93cb964f0f7a3674abf9843a9b5f in kudu's branch
refs/heads/master from Andrew Wong
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=6908620 ]
[consensus] force fsync metadata on XFS
We've seen issues like KUDU-2195 rear their heads more prominently on
XFS, sometimes requiring costly surgery to fix by restoring metadata if
all replicas lose their consensus metadata.
This patch adds to the band-aid flag --cmeta_force_fsync, checking if
the metadata is on an XFS mount, and if so, always fsyncing. This new
behavior is enabled by default with the --cmeta_fsync_override_on_xfs
flag.
Change-Id: Ie97138e3522f20325be104d3665d887693b79a10
Reviewed-on: http://gerrit.cloudera.org:8080/18101
Reviewed-by: Alexey Serbin <[email protected]>
Tested-by: Kudu Jenkins
Reviewed-by: Attila Bukor <[email protected]>
> Enforce durability happened before relationships on multiple disks
> ------------------------------------------------------------------
>
> Key: KUDU-2195
> URL: https://issues.apache.org/jira/browse/KUDU-2195
> Project: Kudu
> Issue Type: Bug
> Components: consensus, tablet
> Affects Versions: 1.9.0
> Reporter: David Alves
> Priority: Major
>
> When using weaker durability semantics (e.g. when log_force_fsync is off) we
> should still enforce certain happened before relationships which are not
> currently being enforced when using different disks for the wal and data.
> The two cases that come to mind where this is relevant are:
> 1) cmeta (c) -> wal (w) : We flush cmeta before flushing the wal (for
> instance on term change) with the intention that either {}, \{c} or \{c, w}
> were made durable.
> 2) wal (w) -> tablet meta (t): We flush the wal before tablet metadata to
> make sure that that all commit messages that refer to on disk row sets (and
> deltas) are on disk before the row sets they point to, i.e. with the
> intention that either {}, \{w} or \{w, t} were made durable.
> With strong durability semantics these are always made durable in the right
> order. With weaker semantics that is not the case though. If using the same
> disk for both the wal and data then the invariants are still preserved, as
> buffers will be flushed in the right order but if using different disks for
> the wal and data (and because cmeta is stored with the data) that is not
> always the case.
> 1) in ext4 is actually safe, because we perform an fsync (indirect, rename()
> implies fsync in ext4) when flushing cmeta. But it is not for xfs.
> 2) Is not safe in either filesystem.
> --- Possible solutions --
> For 1): Store cmeta with the wal; actually always fsync cmeta.
> For 2): Store tablet meta with the wal; always fsync the wal before flushing
> tablet meta.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
