IMPALA-5297: Reduce free-pool-test mem requirement to avoid OOM On jenkins.impala.io gerrit-verify-dryrun jobs, free-pool-test started running out of memory after updating Kudu to a newer version which can have slightly higher memory requirements (Kudu will reject writes when mem is 80% full rather than 60% full). free-pool-test can allocate up to 12gb, and the VMs have a CommitLimit of only 16gb.
While larger VMs could be used, or the minicluster could be tuned further, free-pool-test can also be modified to reduce the actual RSS usage. Instead of memsetting the entire allocation in the test, we only scribble the first byte. This reduces the max RSS from 14gb to 88mb in some local tests. Change-Id: I31f03e7a4d5d237a1183277c988f85a992396a43 Reviewed-on: http://gerrit.cloudera.org:8080/6842 Reviewed-by: Dan Hecht <[email protected]> Tested-by: Impala Public Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/e7cb80b6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/e7cb80b6 Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/e7cb80b6 Branch: refs/heads/master Commit: e7cb80b66f6ff653fa41713c090a9284c5bbc1f7 Parents: 49b6af5 Author: Matthew Jacobs <[email protected]> Authored: Wed May 10 10:13:38 2017 -0700 Committer: Impala Public Jenkins <[email protected]> Committed: Wed May 10 23:08:48 2017 +0000 ---------------------------------------------------------------------- be/src/runtime/free-pool-test.cc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/e7cb80b6/be/src/runtime/free-pool-test.cc ---------------------------------------------------------------------- diff --git a/be/src/runtime/free-pool-test.cc b/be/src/runtime/free-pool-test.cc index 63153c9..07b676f 100644 --- a/be/src/runtime/free-pool-test.cc +++ b/be/src/runtime/free-pool-test.cc @@ -135,8 +135,9 @@ TEST(FreePoolTest, Loop) { EXPECT_TRUE(p1 != NULL); EXPECT_TRUE(p2 != NULL); EXPECT_TRUE(p1 != p2); - memset(p1, 1, size); // Scribble the expected value (used below). - memset(p2, 1, size); + // Scribble the expected value (used below). + *p1 = 1; + *p2 = 1; primed_allocations[size] = make_pair(p1, p2); pool.Free(p1); pool.Free(p2);
