Repository: kudu Updated Branches: refs/heads/master b848488d9 -> f1f192ca3
KUDU-1592 Make warnings about FBM more ominous The documentation warned about scalability issues with file block manager, but it was rather lightly worded and so was the warning message of a hole punch test failure. This commit removes the actual flag necessary to enable FBM from the hole punching tests' warning to force users look it up in the documentation instead of blindly applying it, adds a note about it being unsuitable for production use to the flag's description and expands the warning in the troubleshooting section. Change-Id: I15adf42dd7e6376a85cdb178ad460f239a4f87a1 Reviewed-on: http://gerrit.cloudera.org:8080/11579 Reviewed-by: Adar Dembo <a...@cloudera.com> Reviewed-by: Andrew Wong <aw...@cloudera.com> Tested-by: Attila Bukor <abu...@apache.org> Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/f1f192ca Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/f1f192ca Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/f1f192ca Branch: refs/heads/master Commit: f1f192ca3abead4055e7ab6e6bba84d3d6e9866b Parents: b848488 Author: Attila Bukor <abu...@apache.org> Authored: Fri Oct 5 19:53:38 2018 +0200 Committer: Attila Bukor <abu...@apache.org> Committed: Fri Oct 5 18:40:27 2018 +0000 ---------------------------------------------------------------------- docs/troubleshooting.adoc | 26 ++++++++++++++++++++------ src/kudu/fs/data_dirs.cc | 8 ++++++-- src/kudu/fs/fs_manager.cc | 3 ++- 3 files changed, 28 insertions(+), 9 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kudu/blob/f1f192ca/docs/troubleshooting.adoc ---------------------------------------------------------------------- diff --git a/docs/troubleshooting.adoc b/docs/troubleshooting.adoc index 1bebe07..c3f8fbf 100644 --- a/docs/troubleshooting.adoc +++ b/docs/troubleshooting.adoc @@ -48,8 +48,12 @@ Error during hole punch test. The log block manager requires a filesystem with hole punching support such as ext4 or xfs. On el6, kernel version 2.6.32-358 or newer is required. To run without hole punching (at the cost of some efficiency and scalability), reconfigure -Kudu with --block_manager=file. Refer to the Kudu documentation for more -details. Raw error message follows. +Kudu to use the file block manager. Refer to the Kudu documentation for +more details. WARNING: the file block manager is not suitable for +production use and should be used only for small-scale evaluation and +development on systems where hole-punching is not available. It's +impossible to switch between block managers after data is written to the +server. Raw error message follows ---- [NOTE] @@ -70,8 +74,18 @@ adding the `--block_manager=file` flag to the commands you use to start the mast and tablet servers. The file block manager does not scale as well as the log block manager. -WARNING: The file block manager is known to scale and perform poorly, and should -only be used for small-scale evaluation and development. +[WARNING] +==== +The file block manager is known to scale and perform poorly, and should +only be used for small-scale evaluation and development, and only on systems +where hole punching is unavailable. + +The file block manager uses one file per block. As multiple blocks are written +for each rowset, the number of blocks can be very high, especially for actively +written tablets. This can cause performance issues compared to the log block +manager even with a small amount of data and it's *impossible to switch between +block managers* without wiping and reinitializing the tablet servers. +==== [[disk_issues]] === Already present: FS layout already exists @@ -224,7 +238,7 @@ jitter, while `lpeers` lists dispersion. [NOTE] -**** +==== .Using `chrony` for time synchronization Some operating systems offer `chrony` as an alternative to `ntpd` for network time @@ -233,7 +247,7 @@ synchronization. Kudu has been tested most thoroughly using `ntpd` and use of In order to use `chrony` for synchronization, `chrony.conf` must be configured with the `rtcsync` option. -**** +==== ==== NTP Configuration Best Practices http://git-wip-us.apache.org/repos/asf/kudu/blob/f1f192ca/src/kudu/fs/data_dirs.cc ---------------------------------------------------------------------- diff --git a/src/kudu/fs/data_dirs.cc b/src/kudu/fs/data_dirs.cc index 49139ea..af0cbec 100644 --- a/src/kudu/fs/data_dirs.cc +++ b/src/kudu/fs/data_dirs.cc @@ -136,8 +136,12 @@ const char kHolePunchErrorMsg[] = "filesystem with hole punching support such as ext4 or xfs. On el6, " "kernel version 2.6.32-358 or newer is required. To run without hole " "punching (at the cost of some efficiency and scalability), reconfigure " - "Kudu with --block_manager=file. Refer to the Kudu documentation for more " - "details. Raw error message follows"; + "Kudu to use the file block manager. Refer to the Kudu documentation for " + "more details. WARNING: the file block manager is not suitable for " + "production use and should be used only for small-scale evaluation and " + "development on systems where hole-punching is not available. It's " + "impossible to switch between block managers after data is written to the " + "server. Raw error message follows"; Status CheckHolePunch(Env* env, const string& path) { // Arbitrary constants. http://git-wip-us.apache.org/repos/asf/kudu/blob/f1f192ca/src/kudu/fs/fs_manager.cc ---------------------------------------------------------------------- diff --git a/src/kudu/fs/fs_manager.cc b/src/kudu/fs/fs_manager.cc index a9b954d..1255888 100644 --- a/src/kudu/fs/fs_manager.cc +++ b/src/kudu/fs/fs_manager.cc @@ -67,7 +67,8 @@ TAG_FLAG(enable_data_block_fsync, unsafe); #if defined(__linux__) DEFINE_string(block_manager, "log", "Which block manager to use for storage. " - "Valid options are 'file' and 'log'."); + "Valid options are 'file' and 'log'. The file block manager is not suitable for " + "production use due to scaling limitations."); #else DEFINE_string(block_manager, "file", "Which block manager to use for storage. " "Only the file block manager is supported for non-Linux systems.");