Repository: kudu Updated Branches: refs/heads/master 12ae13b03 -> 17aceda39
KUDU-2377: cap GetResourceLimit return value at kint32max Resource limits on UNIX are 64-bit unsigned integers but we mistreat them: 1. GetResourceLimit returns the underlying unsigned long as a int64_t. 2. The values may be further truncated into 32-bit integers. For example, the LogBlockManager constructor truncates a int64_t value derived from the resource limit into a 32-bit integer when constructing its FileCache. Here's a simple workaround: let's enforce that the returned resource limit value never exceeds kint32max. Sure, that misrepresents the underlying system limit, but for the resources we currently define (max open files and max threads per euid), it's impossible to actually use that much. I manually tested this by running several unit tests in a root shell with ulimit -u set to some high values (2^32-1 and 2^32-2). Change-Id: I0d30ebd5447867d4227974e64786cc48fdbf2688 Reviewed-on: http://gerrit.cloudera.org:8080/9810 Tested-by: Kudu Jenkins Reviewed-by: Grant Henke <[email protected]> Reviewed-by: Todd Lipcon <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/17aceda3 Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/17aceda3 Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/17aceda3 Branch: refs/heads/master Commit: 17aceda39d486a8f3ed6ddf5ae0e32611154a5ef Parents: 12ae13b Author: Adar Dembo <[email protected]> Authored: Mon Mar 26 15:54:29 2018 -0700 Committer: Adar Dembo <[email protected]> Committed: Tue Mar 27 19:25:42 2018 +0000 ---------------------------------------------------------------------- src/kudu/util/env.h | 2 ++ src/kudu/util/env_posix.cc | 7 ++++++- 2 files changed, 8 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kudu/blob/17aceda3/src/kudu/util/env.h ---------------------------------------------------------------------- diff --git a/src/kudu/util/env.h b/src/kudu/util/env.h index 4ce3376..279dc18 100644 --- a/src/kudu/util/env.h +++ b/src/kudu/util/env.h @@ -321,6 +321,8 @@ class Env { // Gets the process' current limit for the given resource type. // + // Returns kint32max if the limit exceeds kint32max or if there is no limit. + // // On UNIX platforms, this is equivalent to the resource's soft limit. virtual int64_t GetResourceLimit(ResourceLimitType t) = 0; http://git-wip-us.apache.org/repos/asf/kudu/blob/17aceda3/src/kudu/util/env_posix.cc ---------------------------------------------------------------------- diff --git a/src/kudu/util/env_posix.cc b/src/kudu/util/env_posix.cc index 2b35805..a8c0c0e 100644 --- a/src/kudu/util/env_posix.cc +++ b/src/kudu/util/env_posix.cc @@ -39,6 +39,7 @@ #include "kudu/gutil/bind.h" #include "kudu/gutil/bind_helpers.h" #include "kudu/gutil/gscoped_ptr.h" +#include "kudu/gutil/integral_types.h" #include "kudu/gutil/macros.h" #include "kudu/gutil/map-util.h" #include "kudu/gutil/once.h" @@ -1588,7 +1589,11 @@ class PosixEnv : public Env { // There's no reason for this to ever fail. struct rlimit l; PCHECK(getrlimit(ResourceLimitTypeToUnixRlimit(t), &l) == 0); - return l.rlim_cur; + + // RLIM_INFINITY is defined to -1, which will be 2^64-1 when converted to + // rlim_cur's type (uint64_t). Thus it'll also be captured by the + // comparison and converted into kint32max. + return l.rlim_cur > kint32max ? kint32max : l.rlim_cur; } virtual void IncreaseResourceLimit(ResourceLimitType t) OVERRIDE {
